2024-05-20T21:56:09.261 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-20T21:56:09.261 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/openfiletable} --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767 -- /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767/orig.config.yaml 2024-05-20T21:56:09.306 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-20T21:56:09.400 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767 branch: wip-pdonnell-testing-20240517.203521-squid-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/openfiletable} email: pdonnell@redhat.com first_in_suite: false job_id: '7715767' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240517.203521-squid-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240517.203521-squid-debug sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9749 sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d sleep_before_teardown: 0 subset: 272/512 suite: fs suite_branch: wip-pdonnell-testing-20240517.203521-squid-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi055.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHw6HAWdrgYYZB7wGfkSyqajCmebJ2L13MuGFBKYoqq8BznT+sq9XvWggFreDu94QVVUyPK+Lt3yAimoA6wRcig= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPjRcuToZueiuBO0foeE4lEP0eY7/VoO8+VRK6lAMBgw1lf/s0bi+wUHazN0iyPii5XbS8ms8l+T3mbr+dRdH/A= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_openfiletable teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-20_15:44:59 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-20T21:56:09.401 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa; will attempt to use it 2024-05-20T21:56:09.401 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks 2024-05-20T21:56:09.401 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-20T21:56:09.403 INFO:teuthology.task.internal:Checking packages... 2024-05-20T21:56:09.423 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2b507c724ef6c8295ee0a35d185cdab167fd61d' 2024-05-20T21:56:09.423 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-20T21:56:09.424 INFO:teuthology.packaging:ref: None 2024-05-20T21:56:09.424 INFO:teuthology.packaging:tag: None 2024-05-20T21:56:09.424 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-20T21:56:09.424 INFO:teuthology.packaging:sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T21:56:09.424 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-20T21:56:09.640 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2744.gd2b507c7 2024-05-20T21:56:09.642 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-20T21:56:09.648 INFO:teuthology.task.internal:no buildpackages task found 2024-05-20T21:56:09.648 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-20T21:56:09.673 INFO:teuthology.task.internal:Saving configuration 2024-05-20T21:56:09.689 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-20T21:56:09.697 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-20T21:56:09.723 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 21:51:17.699526', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHw6HAWdrgYYZB7wGfkSyqajCmebJ2L13MuGFBKYoqq8BznT+sq9XvWggFreDu94QVVUyPK+Lt3yAimoA6wRcig='} 2024-05-20T21:56:09.745 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 21:51:17.698271', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPjRcuToZueiuBO0foeE4lEP0eY7/VoO8+VRK6lAMBgw1lf/s0bi+wUHazN0iyPii5XbS8ms8l+T3mbr+dRdH/A='} 2024-05-20T21:56:09.745 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-20T21:56:09.752 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-20T21:56:09.752 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-20T21:56:09.752 INFO:teuthology.run_tasks:Running task console_log... 2024-05-20T21:56:09.815 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8ab0027ac0>, signals=[15]) 2024-05-20T21:56:09.815 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-20T21:56:09.820 INFO:teuthology.task.internal:Opening connections... 2024-05-20T21:56:09.821 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T21:56:09.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T21:56:09.910 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T21:56:09.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T21:56:10.002 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-20T21:56:10.009 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2024-05-20T21:56:10.029 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2024-05-20T21:56:10.030 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2024-05-20T21:56:10.087 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="9" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="9" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el9" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:LOGO="fedora-logo-icon" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-20T21:56:10.088 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-20T21:56:10.089 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-20T21:56:10.089 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2024-05-20T21:56:10.113 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-05-20T21:56:10.131 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-05-20T21:56:10.132 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-05-20T21:56:10.190 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-05-20T21:56:10.190 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-05-20T21:56:10.191 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-05-20T21:56:10.191 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-05-20T21:56:10.191 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-05-20T21:56:10.191 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-05-20T21:56:10.191 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-20T21:56:10.191 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-05-20T21:56:10.192 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-05-20T21:56:10.192 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-20T21:56:10.192 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-05-20T21:56:10.192 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-20T21:56:10.192 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-20T21:56:10.192 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-20T21:56:10.192 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-05-20T21:56:10.212 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-20T21:56:10.222 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-20T21:56:10.239 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-20T21:56:10.239 DEBUG:teuthology.orchestra.run.smithi055:> test '!' -e /home/ubuntu/cephtest 2024-05-20T21:56:10.241 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-05-20T21:56:10.257 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-20T21:56:10.263 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-20T21:56:10.263 DEBUG:teuthology.orchestra.run.smithi055:> test -z $(ls -A /var/lib/ceph) 2024-05-20T21:56:10.299 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-05-20T21:56:10.328 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-20T21:56:10.381 INFO:teuthology.run_tasks:Running task kernel... 2024-05-20T21:56:10.396 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-20T21:56:10.396 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-20T21:56:10.397 DEBUG:teuthology.orchestra.run.smithi055:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-20T21:56:10.398 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-20T21:56:10.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T21:56:10.415 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-05-20T21:56:10.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T21:56:10.416 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2024-05-20T21:56:10.473 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-437.el9.x86_64 2024-05-20T21:56:10.474 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.14.0-437.el9.x86_64 2024-05-20T21:56:10.474 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2024-05-20T21:56:10.475 INFO:teuthology.orchestra.run.smithi055.stdout:5.14.0-437.el9.x86_64 2024-05-20T21:56:10.476 INFO:teuthology.task.kernel:Running kernel on smithi055: 5.14.0-437.el9.x86_64 2024-05-20T21:56:10.476 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum install -y kernel 2024-05-20T21:56:12.976 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-20T21:56:14.781 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.1 MB 00:02 2024-05-20T21:56:19.415 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 19 MB 00:05 2024-05-20T21:56:26.116 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.2 MB 00:03 2024-05-20T21:56:27.900 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-05-20T21:56:31.255 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-20T21:56:33.129 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 21 MB 00:05 2024-05-20T21:56:38.317 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-20T21:56:38.348 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.2 MB 00:03 2024-05-20T21:56:39.731 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 16 kB 00:00 2024-05-20T21:56:40.140 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-20T21:56:40.141 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-20T21:56:40.230 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-20T21:56:40.250 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-20T21:56:40.250 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-20T21:56:40.250 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-20T21:56:40.250 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-20T21:56:40.250 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-20T21:56:40.250 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-20T21:56:40.250 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-20T21:56:40.250 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-20T21:56:40.251 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-20T21:56:40.251 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T21:56:40.251 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-20T21:56:40.251 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-20T21:56:40.251 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-05-20T21:56:40.251 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T21:56:40.251 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 78 M 2024-05-20T21:56:40.251 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 123 M 2024-05-20T21:56:40.251 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-20T21:56:41.074 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-20T21:56:41.227 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 817 kB/s | 576 kB 00:00 2024-05-20T21:56:46.316 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-20T21:56:47.702 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): kernel-modules-core-5.14.0-447.el9.x86_6 4.3 MB/s | 28 MB 00:06 2024-05-20T21:56:48.348 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-20T21:56:48.348 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-20T21:56:48.448 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-05-20T21:56:48.463 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-05-20T21:56:48.463 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2024-05-20T21:56:48.463 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-05-20T21:56:48.463 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2024-05-20T21:56:48.463 INFO:teuthology.orchestra.run.smithi055.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-20T21:56:48.463 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2024-05-20T21:56:48.463 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-20T21:56:48.463 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-20T21:56:48.463 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-20T21:56:48.464 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T21:56:48.464 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2024-05-20T21:56:48.464 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-05-20T21:56:48.464 INFO:teuthology.orchestra.run.smithi055.stdout:Install 4 Packages 2024-05-20T21:56:48.464 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T21:56:48.464 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 78 M 2024-05-20T21:56:48.464 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 123 M 2024-05-20T21:56:48.465 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2024-05-20T21:56:49.251 INFO:teuthology.orchestra.run.smithi055.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 862 kB/s | 576 kB 00:00 2024-05-20T21:56:55.384 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): kernel-core-5.14.0-447.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:14 2024-05-20T21:56:55.418 INFO:teuthology.orchestra.run.smithi055.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 2.3 MB/s | 16 MB 00:06 2024-05-20T21:57:19.374 INFO:teuthology.orchestra.run.smithi055.stdout:(3/4): kernel-modules-core-5.14.0-447.el9.x86_6 953 kB/s | 28 MB 00:30 2024-05-20T21:57:22.215 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 836 kB/s | 34 MB 00:41 2024-05-20T21:57:22.215 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-20T21:57:22.216 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.9 MB/s | 78 MB 00:41 2024-05-20T21:57:22.899 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-20T21:57:23.256 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-20T21:57:23.256 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-20T21:57:24.197 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-20T21:57:24.197 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-20T21:57:26.573 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-20T21:57:27.233 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-20T21:57:27.332 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T21:57:28.923 INFO:teuthology.orchestra.run.smithi055.stdout:(4/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 864 kB/s | 34 MB 00:40 2024-05-20T21:57:28.924 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-05-20T21:57:28.924 INFO:teuthology.orchestra.run.smithi055.stdout:Total 1.9 MB/s | 78 MB 00:40 2024-05-20T21:57:29.633 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2024-05-20T21:57:29.989 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T21:57:30.024 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2024-05-20T21:57:30.024 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2024-05-20T21:57:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T21:57:30.982 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2024-05-20T21:57:30.982 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2024-05-20T21:57:33.517 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2024-05-20T21:57:34.185 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-20T21:57:34.274 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T21:57:34.845 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T21:57:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:57:37.061 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T21:57:37.187 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T21:57:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:57:42.259 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T21:57:42.306 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:57:47.403 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:58:09.282 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:58:09.319 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:58:09.679 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:58:09.680 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-20T21:58:09.680 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T21:58:09.680 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T21:58:10.456 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:58:10.457 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T21:58:10.457 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-20T21:58:10.457 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-20T21:58:10.457 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-20T21:58:10.457 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-20T21:58:10.457 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-20T21:58:10.457 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T21:58:10.457 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-20T21:58:10.820 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T21:58:10.909 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T21:58:10.909 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T21:58:10.909 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-20T21:58:10.910 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-20T21:58:10.910 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-20T21:58:10.910 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-20T21:58:10.910 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=None, version=distro) 2024-05-20T21:58:10.910 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T21:58:10.962 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T21:58:10.962 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T21:58:10.962 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-05-20T21:58:10.993 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-05-20T21:58:10.993 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-05-20T21:58:10.993 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-05-20T21:58:10.993 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 68.el9 2024-05-20T21:58:10.993 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-05-20T21:58:10.993 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-20T21:58:10.993 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-05-20T21:58:10.993 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8264254 2024-05-20T21:58:10.993 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-05-20T21:58:10.993 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-05-20T21:58:10.994 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T21:58:10.995 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-05-20T21:58:10.995 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-20T21:58:10.995 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 5.14.0-447.el9.x86_64 2024-05-20T21:58:10.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-20T21:58:11.638 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-05-20T21:58:13.048 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-20T21:58:13.073 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-05-20T21:58:13.076 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-05-20T21:58:13.132 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.jZTSoKR14C 2024-05-20T21:58:13.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jZTSoKR14C 2024-05-20T21:58:13.199 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.jZTSoKR14C 2024-05-20T21:58:13.334 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.jZTSoKR14C is 6KB 2024-05-20T21:58:13.353 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.jZTSoKR14C 2024-05-20T21:58:13.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-05-20T21:58:13.437 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-20T21:58:13.437 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-20T21:58:13.437 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-20T21:58:13.437 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-20T21:58:13.439 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-20T21:58:13.581 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-05-20T21:58:18.406 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:58:18.446 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:58:18.833 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:58:18.833 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-20T21:58:18.834 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T21:58:18.834 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T21:58:19.665 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T21:58:19.665 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T21:58:19.665 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2024-05-20T21:58:19.665 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-20T21:58:19.666 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-20T21:58:19.666 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-20T21:58:19.666 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-20T21:58:19.666 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T21:58:19.666 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-05-20T21:58:20.030 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T21:58:20.129 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T21:58:20.130 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T21:58:20.130 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-20T21:58:20.130 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-20T21:58:20.130 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-20T21:58:20.130 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-20T21:58:20.130 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi055.front.sepia.ceph.com, path=None, version=distro) 2024-05-20T21:58:20.130 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T21:58:20.181 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T21:58:20.182 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T21:58:20.182 DEBUG:teuthology.orchestra.run.smithi055:> sudo rpm -qi grub2-tools 2024-05-20T21:58:20.212 INFO:teuthology.orchestra.run.smithi055.stdout:Name : grub2-tools 2024-05-20T21:58:20.212 INFO:teuthology.orchestra.run.smithi055.stdout:Epoch : 1 2024-05-20T21:58:20.212 INFO:teuthology.orchestra.run.smithi055.stdout:Version : 2.06 2024-05-20T21:58:20.212 INFO:teuthology.orchestra.run.smithi055.stdout:Release : 68.el9 2024-05-20T21:58:20.212 INFO:teuthology.orchestra.run.smithi055.stdout:Architecture: x86_64 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Group : Unspecified 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Size : 8264254 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:License : GPLv3+ 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Packager : builder@centos.org 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Vendor : CentOS 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Summary : Support tools for GRUB. 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout:Description : 2024-05-20T21:58:20.213 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T21:58:20.214 INFO:teuthology.orchestra.run.smithi055.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-20T21:58:20.214 INFO:teuthology.orchestra.run.smithi055.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-20T21:58:20.214 INFO:teuthology.orchestra.run.smithi055.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-20T21:58:20.214 INFO:teuthology.orchestra.run.smithi055.stdout:hardware devices. 2024-05-20T21:58:20.214 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T21:58:20.214 INFO:teuthology.orchestra.run.smithi055.stdout:This subpackage provides tools for support of all platforms. 2024-05-20T21:58:20.215 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-20T21:58:20.215 INFO:teuthology.task.kernel:Updating grub on smithi055 to boot 5.14.0-447.el9.x86_64 2024-05-20T21:58:20.215 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-20T21:58:20.868 INFO:teuthology.orchestra.run.smithi055.stderr:Generating grub configuration file ... 2024-05-20T21:58:22.276 INFO:teuthology.orchestra.run.smithi055.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-20T21:58:22.308 INFO:teuthology.orchestra.run.smithi055.stderr:done 2024-05-20T21:58:22.311 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2024-05-20T21:58:22.366 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.eLo33HvJDp 2024-05-20T21:58:22.367 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eLo33HvJDp 2024-05-20T21:58:22.433 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.eLo33HvJDp 2024-05-20T21:58:22.570 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.eLo33HvJDp is 6KB 2024-05-20T21:58:22.618 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.eLo33HvJDp 2024-05-20T21:58:22.633 DEBUG:teuthology.orchestra.run.smithi055:> sudo /bin/ls /boot/loader/entries || true 2024-05-20T21:58:22.696 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-20T21:58:22.696 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-20T21:58:22.696 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-20T21:58:22.696 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-20T21:58:22.697 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-20T21:58:22.844 DEBUG:teuthology.orchestra.run.smithi055:> sudo shutdown -r now 2024-05-20T21:58:43.603 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-20T21:58:43.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-20T21:58:43.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T21:58:52.856 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-20T21:58:52.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T21:58:52.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T21:59:02.178 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-05-20T21:59:11.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-20T21:59:11.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T21:59:11.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-05-20T21:59:14.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-05-20T21:59:20.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T21:59:20.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T21:59:26.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-20T21:59:26.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T21:59:38.786 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-05-20T21:59:50.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T21:59:50.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T21:59:51.294 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-05-20T21:59:51.703 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T21:59:51.703 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-447.el9.x86_64"... 2024-05-20T21:59:51.703 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2024-05-20T21:59:51.722 INFO:teuthology.orchestra.run.smithi055.stdout:5.14.0-447.el9.x86_64 2024-05-20T21:59:51.722 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-20T21:59:51.722 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-20T21:59:51.722 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-20T21:59:52.723 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-20T21:59:52.724 DEBUG:teuthology.orchestra.run.smithi055:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-20T21:59:52.786 INFO:teuthology.orchestra.run.smithi055.stdout:ttyS1 2024-05-20T21:59:52.826 DEBUG:teuthology.parallel:result is None 2024-05-20T21:59:58.016 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-05-20T21:59:58.487 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-20T21:59:58.488 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-447.el9.x86_64"... 2024-05-20T21:59:58.488 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-05-20T21:59:58.502 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-447.el9.x86_64 2024-05-20T21:59:58.503 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-20T21:59:58.503 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-20T21:59:58.503 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-20T21:59:59.504 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-20T21:59:59.504 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-20T21:59:59.618 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-05-20T21:59:59.683 DEBUG:teuthology.parallel:result is None 2024-05-20T21:59:59.684 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-20T21:59:59.691 INFO:teuthology.task.internal:Creating test directory... 2024-05-20T21:59:59.691 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-20T21:59:59.694 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-20T21:59:59.721 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-20T21:59:59.739 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-20T21:59:59.772 INFO:teuthology.task.internal:Creating archive directory... 2024-05-20T21:59:59.773 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-20T21:59:59.774 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-20T21:59:59.841 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-20T21:59:59.850 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-20T21:59:59.851 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-20T21:59:59.853 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-20T21:59:59.879 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T21:59:59.889 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T22:00:00.037 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T22:00:00.063 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T22:00:00.065 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-20T22:00:00.073 INFO:teuthology.task.internal:Configuring sudo... 2024-05-20T22:00:00.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-20T22:00:00.075 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-20T22:00:00.120 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-20T22:00:00.129 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-20T22:00:00.129 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-20T22:00:00.145 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-20T22:00:00.166 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T22:00:00.251 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T22:00:00.346 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T22:00:00.394 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T22:00:00.456 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:00:00.457 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-20T22:00:00.557 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T22:00:00.658 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T22:00:00.701 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T22:00:00.785 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T22:00:00.823 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:00:00.823 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-20T22:00:00.889 DEBUG:teuthology.orchestra.run.smithi055:> sudo service rsyslog restart 2024-05-20T22:00:00.892 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-05-20T22:00:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T22:00:01.156 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T22:00:01.691 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-20T22:00:01.700 INFO:teuthology.task.internal:Starting timer... 2024-05-20T22:00:01.701 INFO:teuthology.run_tasks:Running task pcp... 2024-05-20T22:00:01.736 INFO:teuthology.run_tasks:Running task selinux... 2024-05-20T22:00:01.770 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-20T22:00:01.770 DEBUG:teuthology.orchestra.run.smithi055:> sudo service auditd rotate 2024-05-20T22:00:01.864 INFO:teuthology.orchestra.run.smithi055.stdout:Rotating logs: 2024-05-20T22:00:01.866 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-05-20T22:00:01.903 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-05-20T22:00:01.906 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-20T22:00:01.906 DEBUG:teuthology.orchestra.run.smithi055:> /usr/sbin/getenforce 2024-05-20T22:00:01.988 INFO:teuthology.orchestra.run.smithi055.stdout:Permissive 2024-05-20T22:00:01.988 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-05-20T22:00:02.008 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-05-20T22:00:02.008 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi055.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-05-20T22:00:02.008 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T22:00:02.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:00:02.034 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T22:00:02.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:00:02.074 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-20T22:00:02.074 DEBUG:teuthology.orchestra.run.smithi055:> sudo /usr/sbin/setenforce permissive 2024-05-20T22:00:02.105 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-05-20T22:00:02.141 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-20T22:00:02.150 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-20T22:00:02.163 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-20T22:00:02.164 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi055.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-20T22:04:39.703 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi055.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-05-20T22:04:39.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T22:04:39.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T22:04:39.787 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-05-20T22:04:39.868 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-20T22:04:39.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-20T22:04:39.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T22:04:39.947 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-05-20T22:04:40.028 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-20T22:04:40.028 INFO:teuthology.run_tasks:Running task clock... 2024-05-20T22:04:40.037 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-20T22:04:40.037 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-20T22:04:40.038 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T22:04:40.040 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-20T22:04:40.041 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T22:04:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-20T22:04:40.090 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-20T22:04:40.099 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-20T22:04:40.114 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-20T22:04:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: ntpd: command not found 2024-05-20T22:04:40.129 INFO:teuthology.orchestra.run.smithi055.stdout:506 Cannot talk to daemon 2024-05-20T22:04:40.140 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-05-20T22:04:40.144 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-20T22:04:40.152 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-05-20T22:04:40.159 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-20T22:04:40.166 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-20T22:04:40.180 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-20T22:04:40.218 INFO:teuthology.orchestra.run.smithi055.stderr:bash: line 1: ntpq: command not found 2024-05-20T22:04:40.221 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T22:04:40.221 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2024-05-20T22:04:40.221 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T22:04:40.222 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T22:04:40.222 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T22:04:40.222 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T22:04:40.232 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-05-20T22:04:40.236 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T22:04:40.236 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-05-20T22:04:40.236 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T22:04:40.236 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T22:04:40.236 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T22:04:40.236 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T22:04:40.237 INFO:teuthology.run_tasks:Running task install... 2024-05-20T22:04:40.245 DEBUG:teuthology.task.install:project ceph 2024-05-20T22:04:40.245 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}} 2024-05-20T22:04:40.245 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'} 2024-05-20T22:04:40.246 INFO:teuthology.task.install:Using flavor: default 2024-05-20T22:04:40.251 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-20T22:04:40.251 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-20T22:04:40.252 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-20T22:04:40.252 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T22:04:40.254 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-20T22:04:40.254 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T22:04:40.396 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-20T22:04:40.396 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-20T22:04:40.399 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-20T22:04:40.400 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-20T22:04:40.510 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-20T22:04:40.510 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:04:40.510 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-20T22:04:40.517 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-20T22:04:40.517 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:04:40.517 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-20T22:04:40.536 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-20T22:04:40.536 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-20T22:04:40.545 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-20T22:04:40.545 DEBUG:teuthology.orchestra.run.smithi055:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-20T22:04:40.602 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-20T22:04:40.612 DEBUG:teuthology.orchestra.run.smithi055:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-20T22:04:40.677 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-20T22:04:40.689 DEBUG:teuthology.orchestra.run.smithi055:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-20T22:04:40.741 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-05-20T22:04:40.743 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-05-20T22:04:40.754 INFO:teuthology.orchestra.run.smithi055.stdout:check_obsoletes = 1 2024-05-20T22:04:40.756 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum clean all 2024-05-20T22:04:40.970 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-05-20T22:04:40.989 INFO:teuthology.orchestra.run.smithi055.stdout:45 files removed 2024-05-20T22:04:40.997 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-20T22:04:41.016 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-20T22:04:41.716 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 186 kB/s | 83 kB 00:00 2024-05-20T22:04:41.853 INFO:teuthology.orchestra.run.smithi055.stdout:ceph packages for x86_64 151 kB/s | 83 kB 00:00 2024-05-20T22:04:42.083 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-05-20T22:04:42.204 INFO:teuthology.orchestra.run.smithi055.stdout:ceph noarch packages 39 kB/s | 11 kB 00:00 2024-05-20T22:04:42.442 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-05-20T22:04:42.546 INFO:teuthology.orchestra.run.smithi055.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-05-20T22:04:44.563 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.1 MB 00:01 2024-05-20T22:04:45.192 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.1 MB 00:02 2024-05-20T22:04:46.522 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-20T22:04:53.348 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.2 MB 00:03 2024-05-20T22:04:54.807 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-05-20T22:05:02.250 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-20T22:05:03.667 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 2.4 MB/s | 21 MB 00:08 2024-05-20T22:05:07.720 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-20T22:05:08.992 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-20T22:05:10.187 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-05-20T22:05:11.244 INFO:teuthology.orchestra.run.smithi055.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T22:05:11.244 INFO:teuthology.orchestra.run.smithi055.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T22:05:11.256 INFO:teuthology.orchestra.run.smithi055.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-20T22:05:11.257 INFO:teuthology.orchestra.run.smithi055.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-20T22:05:11.257 INFO:teuthology.orchestra.run.smithi055.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-20T22:05:11.258 INFO:teuthology.orchestra.run.smithi055.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-20T22:05:11.260 INFO:teuthology.orchestra.run.smithi055.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-20T22:05:11.262 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-20T22:05:11.263 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-20T22:05:11.331 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-05-20T22:05:11.352 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-05-20T22:05:11.352 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2024-05-20T22:05:11.352 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-05-20T22:05:11.352 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2024-05-20T22:05:11.352 INFO:teuthology.orchestra.run.smithi055.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-20T22:05:11.352 INFO:teuthology.orchestra.run.smithi055.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-20T22:05:11.352 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-20T22:05:11.352 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-20T22:05:11.352 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-20T22:05:11.353 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-20T22:05:11.354 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrading: 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-20T22:05:11.355 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-20T22:05:11.356 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-20T22:05:11.357 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-20T22:05:11.358 INFO:teuthology.orchestra.run.smithi055.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-20T22:05:11.359 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-20T22:05:11.360 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-20T22:05:11.361 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-20T22:05:11.362 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-20T22:05:11.363 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout:Installing weak dependencies: 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-05-20T22:05:11.364 INFO:teuthology.orchestra.run.smithi055.stdout:Install 148 Packages 2024-05-20T22:05:11.365 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrade 11 Packages 2024-05-20T22:05:11.365 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:05:11.365 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 222 M 2024-05-20T22:05:11.365 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2024-05-20T22:05:11.917 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-20T22:05:12.010 INFO:teuthology.orchestra.run.smithi055.stdout:(1/159): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-05-20T22:05:12.685 INFO:teuthology.orchestra.run.smithi055.stdout:(2/159): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.2 MB/s | 863 kB 00:00 2024-05-20T22:05:12.769 INFO:teuthology.orchestra.run.smithi055.stdout:(3/159): ceph-immutable-object-cache-19.0.0-274 1.7 MB/s | 147 kB 00:00 2024-05-20T22:05:13.269 INFO:teuthology.orchestra.run.smithi055.stdout:(4/159): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 4.5 MB/s | 2.3 MB 00:00 2024-05-20T22:05:13.478 INFO:teuthology.orchestra.run.smithi055.stdout:(5/159): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 7.1 MB/s | 1.5 MB 00:00 2024-05-20T22:05:14.036 INFO:teuthology.orchestra.run.smithi055.stdout:(6/159): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 8.7 MB/s | 4.9 MB 00:00 2024-05-20T22:05:14.353 INFO:teuthology.orchestra.run.smithi055.stdout:(7/159): ceph-common-19.0.0-2744.gd2b507c7.el9. 7.6 MB/s | 19 MB 00:02 2024-05-20T22:05:14.987 INFO:teuthology.orchestra.run.smithi055.stdout:(8/159): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-05-20T22:05:15.154 INFO:teuthology.orchestra.run.smithi055.stdout:(9/159): ceph-radosgw-19.0.0-2744.gd2b507c7.el9 12 MB/s | 9.8 MB 00:00 2024-05-20T22:05:15.179 INFO:teuthology.orchestra.run.smithi055.stdout:(10/159): ceph-selinux-19.0.0-2744.gd2b507c7.el 131 kB/s | 25 kB 00:00 2024-05-20T22:05:15.321 INFO:teuthology.orchestra.run.smithi055.stdout:(11/159): cephfs-mirror-19.0.0-2744.gd2b507c7.e 1.5 MB/s | 221 kB 00:00 2024-05-20T22:05:15.421 INFO:teuthology.orchestra.run.smithi055.stdout:(12/159): cephfs-shell-19.0.0-2744.gd2b507c7.el 271 kB/s | 27 kB 00:00 2024-05-20T22:05:15.638 INFO:teuthology.orchestra.run.smithi055.stdout:(13/159): ceph-osd-19.0.0-2744.gd2b507c7.el9.x8 11 MB/s | 18 MB 00:01 2024-05-20T22:05:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:(14/159): libcephfs-devel-19.0.0-2744.gd2b507c7 131 kB/s | 32 kB 00:00 2024-05-20T22:05:15.705 INFO:teuthology.orchestra.run.smithi055.stdout:(15/159): libcephfs2-19.0.0-2744.gd2b507c7.el9. 11 MB/s | 756 kB 00:00 2024-05-20T22:05:15.730 INFO:teuthology.orchestra.run.smithi055.stdout:(16/159): libcephsqlite-19.0.0-2744.gd2b507c7.e 2.3 MB/s | 158 kB 00:00 2024-05-20T22:05:15.772 INFO:teuthology.orchestra.run.smithi055.stdout:(17/159): librados-devel-19.0.0-2744.gd2b507c7. 1.9 MB/s | 127 kB 00:00 2024-05-20T22:05:15.906 INFO:teuthology.orchestra.run.smithi055.stdout:(18/159): libradosstriper1-19.0.0-2744.gd2b507c 2.8 MB/s | 497 kB 00:00 2024-05-20T22:05:16.006 INFO:teuthology.orchestra.run.smithi055.stdout:(19/159): python3-ceph-argparse-19.0.0-2744.gd2 452 kB/s | 45 kB 00:00 2024-05-20T22:05:16.115 INFO:teuthology.orchestra.run.smithi055.stdout:(20/159): python3-ceph-common-19.0.0-2744.gd2b5 1.2 MB/s | 135 kB 00:00 2024-05-20T22:05:16.265 INFO:teuthology.orchestra.run.smithi055.stdout:(21/159): librgw2-19.0.0-2744.gd2b507c7.el9.x86 10 MB/s | 5.0 MB 00:00 2024-05-20T22:05:16.291 INFO:teuthology.orchestra.run.smithi055.stdout:(22/159): python3-cephfs-19.0.0-2744.gd2b507c7. 921 kB/s | 161 kB 00:00 2024-05-20T22:05:16.332 INFO:teuthology.orchestra.run.smithi055.stdout:(23/159): python3-rados-19.0.0-2744.gd2b507c7.e 4.7 MB/s | 320 kB 00:00 2024-05-20T22:05:16.399 INFO:teuthology.orchestra.run.smithi055.stdout:(24/159): python3-rgw-19.0.0-2744.gd2b507c7.el9 1.5 MB/s | 99 kB 00:00 2024-05-20T22:05:16.487 INFO:teuthology.orchestra.run.smithi055.stdout:(25/159): python3-rbd-19.0.0-2744.gd2b507c7.el9 1.5 MB/s | 299 kB 00:00 2024-05-20T22:05:16.517 INFO:teuthology.orchestra.run.smithi055.stdout:(26/159): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 731 kB/s | 86 kB 00:00 2024-05-20T22:05:16.584 INFO:teuthology.orchestra.run.smithi055.stdout:(27/159): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 2.5 MB/s | 169 kB 00:00 2024-05-20T22:05:16.651 INFO:teuthology.orchestra.run.smithi055.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2744.g 474 kB/s | 32 kB 00:00 2024-05-20T22:05:16.726 INFO:teuthology.orchestra.run.smithi055.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 1.9 MB/s | 142 kB 00:00 2024-05-20T22:05:17.023 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-20T22:05:17.135 INFO:teuthology.orchestra.run.smithi055.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2744.gd2b50 8.6 MB/s | 3.5 MB 00:00 2024-05-20T22:05:17.193 INFO:teuthology.orchestra.run.smithi055.stdout:(31/159): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 4.3 MB/s | 3.0 MB 00:00 2024-05-20T22:05:17.310 INFO:teuthology.orchestra.run.smithi055.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2744.gd2 2.1 MB/s | 249 kB 00:00 2024-05-20T22:05:17.411 INFO:teuthology.orchestra.run.smithi055.stdout:(33/159): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 494 kB/s | 49 kB 00:00 2024-05-20T22:05:17.519 INFO:teuthology.orchestra.run.smithi055.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2744.gd 152 kB/s | 16 kB 00:00 2024-05-20T22:05:17.645 INFO:teuthology.orchestra.run.smithi055.stdout:(35/159): ceph-volume-19.0.0-2744.gd2b507c7.el9 2.1 MB/s | 264 kB 00:00 2024-05-20T22:05:18.229 INFO:teuthology.orchestra.run.smithi055.stdout:(36/159): ceph-test-19.0.0-2744.gd2b507c7.el9.x 15 MB/s | 46 MB 00:03 2024-05-20T22:05:18.329 INFO:teuthology.orchestra.run.smithi055.stdout:(37/159): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-05-20T22:05:18.354 INFO:teuthology.orchestra.run.smithi055.stdout:(38/159): cephfs-top-19.0.0-2744.gd2b507c7.el9. 146 kB/s | 18 kB 00:00 2024-05-20T22:05:18.388 INFO:teuthology.orchestra.run.smithi055.stdout:(39/159): cephadm-19.0.0-2744.gd2b507c7.el9.noa 1.0 MB/s | 763 kB 00:00 2024-05-20T22:05:18.479 INFO:teuthology.orchestra.run.smithi055.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-05-20T22:05:18.513 INFO:teuthology.orchestra.run.smithi055.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-05-20T22:05:18.555 INFO:teuthology.orchestra.run.smithi055.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-05-20T22:05:18.588 INFO:teuthology.orchestra.run.smithi055.stdout:(43/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 803 kB 00:00 2024-05-20T22:05:18.739 INFO:teuthology.orchestra.run.smithi055.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 581 kB/s | 106 kB 00:00 2024-05-20T22:05:18.764 INFO:teuthology.orchestra.run.smithi055.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 771 kB/s | 135 kB 00:00 2024-05-20T22:05:18.798 INFO:teuthology.orchestra.run.smithi055.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-20T22:05:18.831 INFO:teuthology.orchestra.run.smithi055.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-05-20T22:05:18.865 INFO:teuthology.orchestra.run.smithi055.stdout:(48/159): python3-cryptography-36.0.1-4.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-05-20T22:05:18.899 INFO:teuthology.orchestra.run.smithi055.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-05-20T22:05:18.924 INFO:teuthology.orchestra.run.smithi055.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-20T22:05:18.925 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T22:05:18.966 INFO:teuthology.orchestra.run.smithi055.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 6.9 MB/s | 943 kB 00:00 2024-05-20T22:05:18.991 INFO:teuthology.orchestra.run.smithi055.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-05-20T22:05:19.016 INFO:teuthology.orchestra.run.smithi055.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-05-20T22:05:19.058 INFO:teuthology.orchestra.run.smithi055.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-05-20T22:05:19.117 INFO:teuthology.orchestra.run.smithi055.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-05-20T22:05:19.143 INFO:teuthology.orchestra.run.smithi055.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 132 kB/s | 11 kB 00:00 2024-05-20T22:05:19.168 INFO:teuthology.orchestra.run.smithi055.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-20T22:05:19.193 INFO:teuthology.orchestra.run.smithi055.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-05-20T22:05:19.219 INFO:teuthology.orchestra.run.smithi055.stdout:(59/159): elfutils-libelf-devel-0.191-4.el9.x86 68 kB/s | 22 kB 00:00 2024-05-20T22:05:19.241 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T22:05:19.241 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T22:05:19.252 INFO:teuthology.orchestra.run.smithi055.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-05-20T22:05:19.252 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-20T22:05:19.253 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-20T22:05:19.254 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-20T22:05:19.254 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-20T22:05:19.256 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-20T22:05:19.259 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-20T22:05:19.259 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-20T22:05:19.277 INFO:teuthology.orchestra.run.smithi055.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-05-20T22:05:19.302 INFO:teuthology.orchestra.run.smithi055.stdout:(62/159): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-05-20T22:05:19.319 INFO:teuthology.orchestra.run.smithi055.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-05-20T22:05:19.321 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-20T22:05:19.337 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-05-20T22:05:19.337 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-20T22:05:19.338 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-05-20T22:05:19.338 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-20T22:05:19.338 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-20T22:05:19.338 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-20T22:05:19.338 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-20T22:05:19.339 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-20T22:05:19.339 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-20T22:05:19.339 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-20T22:05:19.339 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-20T22:05:19.339 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-20T22:05:19.339 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-20T22:05:19.340 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-20T22:05:19.340 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-20T22:05:19.340 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-20T22:05:19.340 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-20T22:05:19.340 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-20T22:05:19.340 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-20T22:05:19.341 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-20T22:05:19.341 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-20T22:05:19.341 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-20T22:05:19.341 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-20T22:05:19.341 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-20T22:05:19.341 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-20T22:05:19.342 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-20T22:05:19.342 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-20T22:05:19.342 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-20T22:05:19.342 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-20T22:05:19.343 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-20T22:05:19.343 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-20T22:05:19.343 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-20T22:05:19.343 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-20T22:05:19.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-20T22:05:19.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-20T22:05:19.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-20T22:05:19.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-20T22:05:19.344 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-20T22:05:19.344 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-20T22:05:19.344 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-20T22:05:19.345 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-20T22:05:19.345 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-05-20T22:05:19.345 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-20T22:05:19.345 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-20T22:05:19.345 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-20T22:05:19.345 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-20T22:05:19.345 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-20T22:05:19.346 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-20T22:05:19.346 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-20T22:05:19.346 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-20T22:05:19.346 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-20T22:05:19.346 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-20T22:05:19.346 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-20T22:05:19.346 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-20T22:05:19.347 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-20T22:05:19.347 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-20T22:05:19.347 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-20T22:05:19.347 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-20T22:05:19.347 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-20T22:05:19.347 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-20T22:05:19.347 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-20T22:05:19.347 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-20T22:05:19.348 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-20T22:05:19.349 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-20T22:05:19.350 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-20T22:05:19.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-20T22:05:19.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-20T22:05:19.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-20T22:05:19.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-20T22:05:19.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-20T22:05:19.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-20T22:05:19.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-20T22:05:19.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-20T22:05:19.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-20T22:05:19.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-20T22:05:19.352 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-20T22:05:19.353 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-20T22:05:19.353 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-20T22:05:19.353 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-20T22:05:19.353 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-20T22:05:19.353 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-20T22:05:19.353 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-20T22:05:19.353 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-20T22:05:19.353 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-20T22:05:19.353 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-20T22:05:19.353 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-20T22:05:19.354 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-20T22:05:19.355 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-20T22:05:19.356 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-20T22:05:19.356 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-05-20T22:05:19.356 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-20T22:05:19.356 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:05:19.356 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-20T22:05:19.356 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-05-20T22:05:19.356 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-05-20T22:05:19.356 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 11 Packages 2024-05-20T22:05:19.356 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:05:19.357 INFO:teuthology.orchestra.run.smithi055.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-05-20T22:05:19.357 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 222 M 2024-05-20T22:05:19.358 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-20T22:05:19.361 INFO:teuthology.orchestra.run.smithi055.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 842 kB/s | 35 kB 00:00 2024-05-20T22:05:19.387 INFO:teuthology.orchestra.run.smithi055.stdout:(66/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-20T22:05:19.479 INFO:teuthology.orchestra.run.smithi055.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-05-20T22:05:19.521 INFO:teuthology.orchestra.run.smithi055.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-20T22:05:19.588 INFO:teuthology.orchestra.run.smithi055.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-20T22:05:19.613 INFO:teuthology.orchestra.run.smithi055.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-20T22:05:19.647 INFO:teuthology.orchestra.run.smithi055.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-05-20T22:05:19.681 INFO:teuthology.orchestra.run.smithi055.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-20T22:05:19.706 INFO:teuthology.orchestra.run.smithi055.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-20T22:05:19.723 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T22:05:19.798 INFO:teuthology.orchestra.run.smithi055.stdout:(74/159): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-05-20T22:05:19.881 INFO:teuthology.orchestra.run.smithi055.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-05-20T22:05:19.906 INFO:teuthology.orchestra.run.smithi055.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-05-20T22:05:19.907 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T22:05:19.948 INFO:teuthology.orchestra.run.smithi055.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-05-20T22:05:19.982 INFO:teuthology.orchestra.run.smithi055.stdout:(78/159): python3-colorama-0.4.6-3.el9.noarch.r 222 kB/s | 61 kB 00:00 2024-05-20T22:05:20.007 INFO:teuthology.orchestra.run.smithi055.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 326 kB/s | 19 kB 00:00 2024-05-20T22:05:20.033 INFO:teuthology.orchestra.run.smithi055.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-20T22:05:20.058 INFO:teuthology.orchestra.run.smithi055.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 689 kB/s | 35 kB 00:00 2024-05-20T22:05:20.100 INFO:teuthology.orchestra.run.smithi055.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-05-20T22:05:20.167 INFO:teuthology.orchestra.run.smithi055.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-05-20T22:05:20.201 INFO:teuthology.orchestra.run.smithi055.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-20T22:05:20.226 INFO:teuthology.orchestra.run.smithi055.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-20T22:05:20.252 INFO:teuthology.orchestra.run.smithi055.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-05-20T22:05:20.277 INFO:teuthology.orchestra.run.smithi055.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-20T22:05:20.302 INFO:teuthology.orchestra.run.smithi055.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-20T22:05:20.327 INFO:teuthology.orchestra.run.smithi055.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-05-20T22:05:20.353 INFO:teuthology.orchestra.run.smithi055.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-20T22:05:20.378 INFO:teuthology.orchestra.run.smithi055.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-20T22:05:20.420 INFO:teuthology.orchestra.run.smithi055.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-20T22:05:20.462 INFO:teuthology.orchestra.run.smithi055.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-20T22:05:20.487 INFO:teuthology.orchestra.run.smithi055.stdout:(94/159): python3-jinja2-2.11.3-6.el9.noarch.rp 410 kB/s | 248 kB 00:00 2024-05-20T22:05:20.513 INFO:teuthology.orchestra.run.smithi055.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-20T22:05:20.676 INFO:teuthology.orchestra.run.smithi138.stdout:(1/159): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-05-20T22:05:20.713 INFO:teuthology.orchestra.run.smithi055.stdout:(96/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-05-20T22:05:20.738 INFO:teuthology.orchestra.run.smithi055.stdout:(97/159): qatlib-service-24.02.0-1.el9.x86_64.r 143 kB/s | 36 kB 00:00 2024-05-20T22:05:20.758 INFO:teuthology.orchestra.run.smithi055.stdout:(98/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 196 kB/s | 47 kB 00:00 2024-05-20T22:05:20.864 INFO:teuthology.orchestra.run.smithi055.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 864 kB/s | 94 kB 00:00 2024-05-20T22:05:21.005 INFO:teuthology.orchestra.run.smithi055.stdout:(100/159): uuid-1.6.2-55.el9.x86_64.rpm 231 kB/s | 58 kB 00:00 2024-05-20T22:05:21.031 INFO:teuthology.orchestra.run.smithi055.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 382 kB/s | 64 kB 00:00 2024-05-20T22:05:21.106 INFO:teuthology.orchestra.run.smithi055.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 446 kB/s | 45 kB 00:00 2024-05-20T22:05:21.131 INFO:teuthology.orchestra.run.smithi055.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-05-20T22:05:21.381 INFO:teuthology.orchestra.run.smithi055.stdout:(104/159): socat-1.7.4.1-5.el9.x86_64.rpm 456 kB/s | 305 kB 00:00 2024-05-20T22:05:21.532 INFO:teuthology.orchestra.run.smithi055.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-05-20T22:05:21.724 INFO:teuthology.orchestra.run.smithi055.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 925 kB/s | 548 kB 00:00 2024-05-20T22:05:21.774 INFO:teuthology.orchestra.run.smithi055.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-05-20T22:05:21.908 INFO:teuthology.orchestra.run.smithi055.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 587 kB/s | 309 kB 00:00 2024-05-20T22:05:21.933 INFO:teuthology.orchestra.run.smithi055.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 164 kB/s | 26 kB 00:00 2024-05-20T22:05:22.033 INFO:teuthology.orchestra.run.smithi055.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-05-20T22:05:22.117 INFO:teuthology.orchestra.run.smithi055.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-20T22:05:22.142 INFO:teuthology.orchestra.run.smithi055.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 322 kB/s | 67 kB 00:00 2024-05-20T22:05:22.168 INFO:teuthology.orchestra.run.smithi055.stdout:(113/159): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-05-20T22:05:22.235 INFO:teuthology.orchestra.run.smithi055.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-05-20T22:05:22.268 INFO:teuthology.orchestra.run.smithi055.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 432 kB/s | 43 kB 00:00 2024-05-20T22:05:22.350 INFO:teuthology.orchestra.run.smithi138.stdout:(2/159): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 515 kB/s | 863 kB 00:01 2024-05-20T22:05:22.352 INFO:teuthology.orchestra.run.smithi055.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-05-20T22:05:22.377 INFO:teuthology.orchestra.run.smithi055.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-05-20T22:05:22.444 INFO:teuthology.orchestra.run.smithi055.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-20T22:05:22.478 INFO:teuthology.orchestra.run.smithi055.stdout:(119/159): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-05-20T22:05:22.528 INFO:teuthology.orchestra.run.smithi055.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-05-20T22:05:22.553 INFO:teuthology.orchestra.run.smithi055.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-05-20T22:05:22.598 INFO:teuthology.orchestra.run.smithi055.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-20T22:05:22.633 INFO:teuthology.orchestra.run.smithi138.stdout:(3/159): ceph-immutable-object-cache-19.0.0-274 519 kB/s | 147 kB 00:00 2024-05-20T22:05:22.655 INFO:teuthology.orchestra.run.smithi055.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-20T22:05:22.688 INFO:teuthology.orchestra.run.smithi055.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-20T22:05:22.746 INFO:teuthology.orchestra.run.smithi055.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-20T22:05:22.780 INFO:teuthology.orchestra.run.smithi055.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-05-20T22:05:22.830 INFO:teuthology.orchestra.run.smithi055.stdout:(127/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-05-20T22:05:22.855 INFO:teuthology.orchestra.run.smithi055.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 178 kB/s | 19 kB 00:00 2024-05-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi055.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-05-20T22:05:22.948 INFO:teuthology.orchestra.run.smithi055.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-05-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi055.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-05-20T22:05:23.006 INFO:teuthology.orchestra.run.smithi055.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 628 kB/s | 79 kB 00:00 2024-05-20T22:05:23.065 INFO:teuthology.orchestra.run.smithi055.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 492 kB/s | 58 kB 00:00 2024-05-20T22:05:23.107 INFO:teuthology.orchestra.run.smithi055.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-20T22:05:23.124 INFO:teuthology.orchestra.run.smithi055.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-05-20T22:05:23.158 INFO:teuthology.orchestra.run.smithi055.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-05-20T22:05:23.210 INFO:teuthology.orchestra.run.smithi055.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-05-20T22:05:23.225 INFO:teuthology.orchestra.run.smithi055.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-05-20T22:05:23.258 INFO:teuthology.orchestra.run.smithi055.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-20T22:05:23.309 INFO:teuthology.orchestra.run.smithi055.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 587 kB/s | 59 kB 00:00 2024-05-20T22:05:23.334 INFO:teuthology.orchestra.run.smithi055.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-05-20T22:05:23.359 INFO:teuthology.orchestra.run.smithi055.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 506 kB/s | 51 kB 00:00 2024-05-20T22:05:23.427 INFO:teuthology.orchestra.run.smithi055.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-20T22:05:23.460 INFO:teuthology.orchestra.run.smithi055.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-20T22:05:23.486 INFO:teuthology.orchestra.run.smithi055.stdout:(145/159): python3-websocket-client-1.2.3-2.el9 593 kB/s | 90 kB 00:00 2024-05-20T22:05:23.519 INFO:teuthology.orchestra.run.smithi055.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-05-20T22:05:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:(4/159): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-05-20T22:05:23.569 INFO:teuthology.orchestra.run.smithi055.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-20T22:05:23.753 INFO:teuthology.orchestra.run.smithi055.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 5.9 MB/s | 1.6 MB 00:00 2024-05-20T22:05:23.928 INFO:teuthology.orchestra.run.smithi055.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-20T22:05:23.987 INFO:teuthology.orchestra.run.smithi055.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 31 MB/s | 1.8 MB 00:00 2024-05-20T22:05:24.029 INFO:teuthology.orchestra.run.smithi055.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 996 kB/s | 41 kB 00:00 2024-05-20T22:05:24.070 INFO:teuthology.orchestra.run.smithi055.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-05-20T22:05:24.087 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T22:05:24.279 INFO:teuthology.orchestra.run.smithi055.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 176 kB/s | 37 kB 00:00 2024-05-20T22:05:24.509 INFO:teuthology.orchestra.run.smithi138.stdout:(5/159): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-05-20T22:05:24.538 INFO:teuthology.orchestra.run.smithi055.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 801 kB/s | 207 kB 00:00 2024-05-20T22:05:24.551 INFO:teuthology.orchestra.run.smithi138.stdout:(6/159): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 1.2 MB/s | 2.3 MB 00:01 2024-05-20T22:05:24.721 INFO:teuthology.orchestra.run.smithi055.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.4 MB/s | 260 kB 00:00 2024-05-20T22:05:24.930 INFO:teuthology.orchestra.run.smithi055.stdout:(156/159): librbd1-19.0.0-2744.gd2b507c7.el9.x8 2.3 MB/s | 3.1 MB 00:01 2024-05-20T22:05:24.997 INFO:teuthology.orchestra.run.smithi055.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 1.3 MB 00:00 2024-05-20T22:05:26.842 INFO:teuthology.orchestra.run.smithi138.stdout:(7/159): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 2.1 MB/s | 4.9 MB 00:02 2024-05-20T22:05:27.056 INFO:teuthology.orchestra.run.smithi055.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 2.4 MB 00:02 2024-05-20T22:05:27.564 INFO:teuthology.orchestra.run.smithi055.stdout:(159/159): librados2-19.0.0-2744.gd2b507c7.el9. 838 kB/s | 3.3 MB 00:04 2024-05-20T22:05:27.567 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-05-20T22:05:27.567 INFO:teuthology.orchestra.run.smithi055.stdout:Total 14 MB/s | 222 MB 00:16 2024-05-20T22:05:27.692 INFO:teuthology.orchestra.run.smithi138.stdout:(8/159): ceph-osd-19.0.0-2744.gd2b507c7.el9.x86 5.6 MB/s | 18 MB 00:03 2024-05-20T22:05:27.801 INFO:teuthology.orchestra.run.smithi138.stdout:(9/159): ceph-selinux-19.0.0-2744.gd2b507c7.el9 232 kB/s | 25 kB 00:00 2024-05-20T22:05:28.818 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2024-05-20T22:05:28.995 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2024-05-20T22:05:28.995 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2024-05-20T22:05:30.251 INFO:teuthology.orchestra.run.smithi138.stdout:(10/159): ceph-common-19.0.0-2744.gd2b507c7.el9 1.9 MB/s | 19 MB 00:10 2024-05-20T22:05:30.426 INFO:teuthology.orchestra.run.smithi138.stdout:(11/159): cephfs-mirror-19.0.0-2744.gd2b507c7.e 1.2 MB/s | 221 kB 00:00 2024-05-20T22:05:30.493 INFO:teuthology.orchestra.run.smithi138.stdout:(12/159): cephfs-shell-19.0.0-2744.gd2b507c7.el 406 kB/s | 27 kB 00:00 2024-05-20T22:05:30.578 INFO:teuthology.orchestra.run.smithi138.stdout:(13/159): libcephfs-devel-19.0.0-2744.gd2b507c7 381 kB/s | 32 kB 00:00 2024-05-20T22:05:30.919 INFO:teuthology.orchestra.run.smithi138.stdout:(14/159): libcephfs2-19.0.0-2744.gd2b507c7.el9. 2.2 MB/s | 756 kB 00:00 2024-05-20T22:05:31.052 INFO:teuthology.orchestra.run.smithi138.stdout:(15/159): libcephsqlite-19.0.0-2744.gd2b507c7.e 1.2 MB/s | 158 kB 00:00 2024-05-20T22:05:31.153 INFO:teuthology.orchestra.run.smithi138.stdout:(16/159): librados-devel-19.0.0-2744.gd2b507c7. 1.2 MB/s | 127 kB 00:00 2024-05-20T22:05:31.271 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2024-05-20T22:05:31.271 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2024-05-20T22:05:31.420 INFO:teuthology.orchestra.run.smithi138.stdout:(17/159): libradosstriper1-19.0.0-2744.gd2b507c 1.8 MB/s | 497 kB 00:00 2024-05-20T22:05:31.795 INFO:teuthology.orchestra.run.smithi138.stdout:(18/159): ceph-radosgw-19.0.0-2744.gd2b507c7.el 2.0 MB/s | 9.8 MB 00:04 2024-05-20T22:05:31.870 INFO:teuthology.orchestra.run.smithi138.stdout:(19/159): python3-ceph-argparse-19.0.0-2744.gd2 605 kB/s | 45 kB 00:00 2024-05-20T22:05:31.954 INFO:teuthology.orchestra.run.smithi138.stdout:(20/159): python3-ceph-common-19.0.0-2744.gd2b5 1.6 MB/s | 135 kB 00:00 2024-05-20T22:05:32.429 INFO:teuthology.orchestra.run.smithi138.stdout:(21/159): ceph-test-19.0.0-2744.gd2b507c7.el9.x 10 MB/s | 46 MB 00:04 2024-05-20T22:05:32.454 INFO:teuthology.orchestra.run.smithi138.stdout:(22/159): python3-cephfs-19.0.0-2744.gd2b507c7. 322 kB/s | 161 kB 00:00 2024-05-20T22:05:32.505 INFO:teuthology.orchestra.run.smithi138.stdout:(23/159): python3-rados-19.0.0-2744.gd2b507c7.e 4.2 MB/s | 320 kB 00:00 2024-05-20T22:05:32.605 INFO:teuthology.orchestra.run.smithi138.stdout:(24/159): python3-rgw-19.0.0-2744.gd2b507c7.el9 993 kB/s | 99 kB 00:00 2024-05-20T22:05:32.630 INFO:teuthology.orchestra.run.smithi138.stdout:(25/159): python3-rbd-19.0.0-2744.gd2b507c7.el9 1.7 MB/s | 299 kB 00:00 2024-05-20T22:05:32.681 INFO:teuthology.orchestra.run.smithi138.stdout:(26/159): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-20T22:05:32.773 INFO:teuthology.orchestra.run.smithi138.stdout:(27/159): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 1.8 MB/s | 169 kB 00:00 2024-05-20T22:05:32.848 INFO:teuthology.orchestra.run.smithi138.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2744.g 424 kB/s | 32 kB 00:00 2024-05-20T22:05:32.932 INFO:teuthology.orchestra.run.smithi138.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 1.7 MB/s | 142 kB 00:00 2024-05-20T22:05:33.232 INFO:teuthology.orchestra.run.smithi138.stdout:(30/159): librgw2-19.0.0-2744.gd2b507c7.el9.x86 2.8 MB/s | 5.0 MB 00:01 2024-05-20T22:05:33.357 INFO:teuthology.orchestra.run.smithi138.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2744.gd2b50 8.3 MB/s | 3.5 MB 00:00 2024-05-20T22:05:33.449 INFO:teuthology.orchestra.run.smithi138.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2744.gd2 2.7 MB/s | 249 kB 00:00 2024-05-20T22:05:33.525 INFO:teuthology.orchestra.run.smithi138.stdout:(33/159): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 656 kB/s | 49 kB 00:00 2024-05-20T22:05:33.600 INFO:teuthology.orchestra.run.smithi138.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2744.gd 220 kB/s | 16 kB 00:00 2024-05-20T22:05:33.700 INFO:teuthology.orchestra.run.smithi138.stdout:(35/159): ceph-volume-19.0.0-2744.gd2b507c7.el9 2.6 MB/s | 264 kB 00:00 2024-05-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi138.stdout:(36/159): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 2.5 MB/s | 3.0 MB 00:01 2024-05-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi138.stdout:(37/159): cephadm-19.0.0-2744.gd2b507c7.el9.noa 4.7 MB/s | 763 kB 00:00 2024-05-20T22:05:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:(38/159): cephfs-top-19.0.0-2744.gd2b507c7.el9. 362 kB/s | 18 kB 00:00 2024-05-20T22:05:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 642 kB/s | 80 kB 00:00 2024-05-20T22:05:34.052 INFO:teuthology.orchestra.run.smithi138.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-05-20T22:05:34.077 INFO:teuthology.orchestra.run.smithi138.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-20T22:05:34.102 INFO:teuthology.orchestra.run.smithi138.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-20T22:05:34.144 INFO:teuthology.orchestra.run.smithi138.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-20T22:05:34.186 INFO:teuthology.orchestra.run.smithi138.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-05-20T22:05:34.220 INFO:teuthology.orchestra.run.smithi138.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-05-20T22:05:34.245 INFO:teuthology.orchestra.run.smithi138.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-20T22:05:34.270 INFO:teuthology.orchestra.run.smithi138.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-20T22:05:34.304 INFO:teuthology.orchestra.run.smithi138.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 11 MB/s | 689 kB 00:00 2024-05-20T22:05:34.346 INFO:teuthology.orchestra.run.smithi138.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-05-20T22:05:34.371 INFO:teuthology.orchestra.run.smithi138.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-20T22:05:34.372 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T22:05:34.422 INFO:teuthology.orchestra.run.smithi138.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-20T22:05:34.456 INFO:teuthology.orchestra.run.smithi138.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 984 kB/s | 33 kB 00:00 2024-05-20T22:05:34.548 INFO:teuthology.orchestra.run.smithi138.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-05-20T22:05:34.590 INFO:teuthology.orchestra.run.smithi138.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-05-20T22:05:34.631 INFO:teuthology.orchestra.run.smithi138.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-05-20T22:05:34.657 INFO:teuthology.orchestra.run.smithi138.stdout:(56/159): elfutils-libelf-devel-0.191-4.el9.x86 70 kB/s | 22 kB 00:00 2024-05-20T22:05:34.682 INFO:teuthology.orchestra.run.smithi138.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-20T22:05:34.726 INFO:teuthology.orchestra.run.smithi138.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 656 kB/s | 45 kB 00:00 2024-05-20T22:05:34.895 INFO:teuthology.orchestra.run.smithi138.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-05-20T22:05:34.925 INFO:teuthology.orchestra.run.smithi138.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 2.9 MB/s | 582 kB 00:00 2024-05-20T22:05:34.950 INFO:teuthology.orchestra.run.smithi138.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 462 kB/s | 25 kB 00:00 2024-05-20T22:05:34.975 INFO:teuthology.orchestra.run.smithi138.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-05-20T22:05:35.076 INFO:teuthology.orchestra.run.smithi138.stdout:(63/159): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-05-20T22:05:35.101 INFO:teuthology.orchestra.run.smithi138.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-05-20T22:05:35.126 INFO:teuthology.orchestra.run.smithi138.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-20T22:05:35.168 INFO:teuthology.orchestra.run.smithi138.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 382 kB/s | 35 kB 00:00 2024-05-20T22:05:35.269 INFO:teuthology.orchestra.run.smithi138.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-05-20T22:05:35.344 INFO:teuthology.orchestra.run.smithi138.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-20T22:05:35.386 INFO:teuthology.orchestra.run.smithi138.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-05-20T22:05:35.411 INFO:teuthology.orchestra.run.smithi138.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-20T22:05:35.453 INFO:teuthology.orchestra.run.smithi138.stdout:(71/159): postgresql-13.14-1.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-05-20T22:05:35.495 INFO:teuthology.orchestra.run.smithi138.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-05-20T22:05:35.520 INFO:teuthology.orchestra.run.smithi138.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-20T22:05:35.537 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T22:05:35.612 INFO:teuthology.orchestra.run.smithi138.stdout:(74/159): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-05-20T22:05:35.704 INFO:teuthology.orchestra.run.smithi138.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 6.0 MB 00:00 2024-05-20T22:05:35.730 INFO:teuthology.orchestra.run.smithi138.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-05-20T22:05:35.730 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T22:05:35.763 INFO:teuthology.orchestra.run.smithi138.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 779 kB/s | 26 kB 00:00 2024-05-20T22:05:35.789 INFO:teuthology.orchestra.run.smithi138.stdout:(78/159): python3-colorama-0.4.6-3.el9.noarch.r 228 kB/s | 61 kB 00:00 2024-05-20T22:05:35.806 INFO:teuthology.orchestra.run.smithi138.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 451 kB/s | 19 kB 00:00 2024-05-20T22:05:35.831 INFO:teuthology.orchestra.run.smithi138.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-20T22:05:35.848 INFO:teuthology.orchestra.run.smithi138.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 833 kB/s | 35 kB 00:00 2024-05-20T22:05:35.890 INFO:teuthology.orchestra.run.smithi138.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.8 MB/s | 250 kB 00:00 2024-05-20T22:05:35.965 INFO:teuthology.orchestra.run.smithi138.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-05-20T22:05:35.991 INFO:teuthology.orchestra.run.smithi138.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-20T22:05:36.016 INFO:teuthology.orchestra.run.smithi138.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-20T22:05:36.033 INFO:teuthology.orchestra.run.smithi138.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-20T22:05:36.066 INFO:teuthology.orchestra.run.smithi138.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-20T22:05:36.092 INFO:teuthology.orchestra.run.smithi138.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-05-20T22:05:36.117 INFO:teuthology.orchestra.run.smithi138.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-20T22:05:36.142 INFO:teuthology.orchestra.run.smithi138.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-20T22:05:36.168 INFO:teuthology.orchestra.run.smithi138.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-20T22:05:36.210 INFO:teuthology.orchestra.run.smithi138.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-20T22:05:36.252 INFO:teuthology.orchestra.run.smithi138.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-20T22:05:36.294 INFO:teuthology.orchestra.run.smithi138.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-05-20T22:05:36.327 INFO:teuthology.orchestra.run.smithi138.stdout:(95/159): python3-jinja2-2.11.3-6.el9.noarch.rp 399 kB/s | 248 kB 00:00 2024-05-20T22:05:36.352 INFO:teuthology.orchestra.run.smithi138.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-05-20T22:05:36.511 INFO:teuthology.orchestra.run.smithi138.stdout:(97/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-05-20T22:05:36.536 INFO:teuthology.orchestra.run.smithi138.stdout:(98/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 227 kB/s | 47 kB 00:00 2024-05-20T22:05:36.562 INFO:teuthology.orchestra.run.smithi138.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 305 kB 00:00 2024-05-20T22:05:36.662 INFO:teuthology.orchestra.run.smithi138.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-05-20T22:05:36.721 INFO:teuthology.orchestra.run.smithi138.stdout:(101/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 509 kB/s | 94 kB 00:00 2024-05-20T22:05:36.902 INFO:teuthology.orchestra.run.smithi138.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 187 kB/s | 45 kB 00:00 2024-05-20T22:05:36.930 INFO:teuthology.orchestra.run.smithi138.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 107 kB/s | 22 kB 00:00 2024-05-20T22:05:36.955 INFO:teuthology.orchestra.run.smithi138.stdout:(104/159): uuid-1.6.2-55.el9.x86_64.rpm 130 kB/s | 58 kB 00:00 2024-05-20T22:05:37.181 INFO:teuthology.orchestra.run.smithi138.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-05-20T22:05:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-05-20T22:05:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 841 kB/s | 309 kB 00:00 2024-05-20T22:05:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:(108/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-20T22:05:37.373 INFO:teuthology.orchestra.run.smithi138.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 517 kB/s | 26 kB 00:00 2024-05-20T22:05:37.407 INFO:teuthology.orchestra.run.smithi138.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-05-20T22:05:37.432 INFO:teuthology.orchestra.run.smithi138.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-20T22:05:37.474 INFO:teuthology.orchestra.run.smithi138.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-05-20T22:05:37.546 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2024-05-20T22:05:37.558 INFO:teuthology.orchestra.run.smithi138.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-05-20T22:05:37.583 INFO:teuthology.orchestra.run.smithi138.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-05-20T22:05:37.625 INFO:teuthology.orchestra.run.smithi138.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 650 kB/s | 43 kB 00:00 2024-05-20T22:05:37.644 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-20T22:05:37.650 INFO:teuthology.orchestra.run.smithi138.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-05-20T22:05:37.676 INFO:teuthology.orchestra.run.smithi138.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-05-20T22:05:37.685 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-20T22:05:37.751 INFO:teuthology.orchestra.run.smithi138.stdout:(118/159): libarrow-9.0.0-11.el9.x86_64.rpm 9.7 MB/s | 4.4 MB 00:00 2024-05-20T22:05:37.754 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-20T22:05:37.776 INFO:teuthology.orchestra.run.smithi138.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-20T22:05:37.795 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-20T22:05:37.802 INFO:teuthology.orchestra.run.smithi138.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-20T22:05:37.820 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-20T22:05:37.827 INFO:teuthology.orchestra.run.smithi138.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-05-20T22:05:37.852 INFO:teuthology.orchestra.run.smithi138.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-20T22:05:37.871 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-20T22:05:37.886 INFO:teuthology.orchestra.run.smithi138.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-20T22:05:37.903 INFO:teuthology.orchestra.run.smithi138.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-05-20T22:05:37.953 INFO:teuthology.orchestra.run.smithi138.stdout:(125/159): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-05-20T22:05:37.970 INFO:teuthology.orchestra.run.smithi138.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 210 kB/s | 18 kB 00:00 2024-05-20T22:05:37.987 INFO:teuthology.orchestra.run.smithi138.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-05-20T22:05:38.012 INFO:teuthology.orchestra.run.smithi138.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-05-20T22:05:38.038 INFO:teuthology.orchestra.run.smithi138.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-05-20T22:05:38.079 INFO:teuthology.orchestra.run.smithi138.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-05-20T22:05:38.105 INFO:teuthology.orchestra.run.smithi138.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-05-20T22:05:38.130 INFO:teuthology.orchestra.run.smithi138.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 852 kB/s | 79 kB 00:00 2024-05-20T22:05:38.156 INFO:teuthology.orchestra.run.smithi138.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 761 kB/s | 58 kB 00:00 2024-05-20T22:05:38.181 INFO:teuthology.orchestra.run.smithi138.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-20T22:05:38.206 INFO:teuthology.orchestra.run.smithi138.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-20T22:05:38.232 INFO:teuthology.orchestra.run.smithi138.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-20T22:05:38.257 INFO:teuthology.orchestra.run.smithi138.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-20T22:05:38.274 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-20T22:05:38.282 INFO:teuthology.orchestra.run.smithi138.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-05-20T22:05:38.307 INFO:teuthology.orchestra.run.smithi138.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-20T22:05:38.333 INFO:teuthology.orchestra.run.smithi138.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-05-20T22:05:38.346 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T22:05:38.358 INFO:teuthology.orchestra.run.smithi138.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-20T22:05:38.383 INFO:teuthology.orchestra.run.smithi138.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-05-20T22:05:38.408 INFO:teuthology.orchestra.run.smithi138.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-20T22:05:38.442 INFO:teuthology.orchestra.run.smithi138.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-05-20T22:05:38.467 INFO:teuthology.orchestra.run.smithi138.stdout:(145/159): python3-websocket-client-1.2.3-2.el9 821 kB/s | 90 kB 00:00 2024-05-20T22:05:38.474 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T22:05:38.493 INFO:teuthology.orchestra.run.smithi138.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-20T22:05:38.505 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T22:05:38.526 INFO:teuthology.orchestra.run.smithi138.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-20T22:05:38.568 INFO:teuthology.orchestra.run.smithi138.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-20T22:05:38.607 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T22:05:38.636 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-20T22:05:38.685 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-20T22:05:38.752 INFO:teuthology.orchestra.run.smithi138.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-20T22:05:38.811 INFO:teuthology.orchestra.run.smithi138.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 31 MB/s | 1.8 MB 00:00 2024-05-20T22:05:38.852 INFO:teuthology.orchestra.run.smithi138.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-20T22:05:38.895 INFO:teuthology.orchestra.run.smithi138.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-05-20T22:05:38.900 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-20T22:05:38.911 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T22:05:38.951 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-20T22:05:38.978 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T22:05:39.003 INFO:teuthology.orchestra.run.smithi138.stdout:(153/159): librados2-19.0.0-2744.gd2b507c7.el9. 6.5 MB/s | 3.3 MB 00:00 2024-05-20T22:05:39.085 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T22:05:39.104 INFO:teuthology.orchestra.run.smithi138.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 176 kB/s | 37 kB 00:00 2024-05-20T22:05:39.112 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-20T22:05:39.175 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-20T22:05:39.197 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T22:05:39.229 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T22:05:39.304 INFO:teuthology.orchestra.run.smithi138.stdout:(155/159): librbd1-19.0.0-2744.gd2b507c7.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-05-20T22:05:39.379 INFO:teuthology.orchestra.run.smithi138.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 942 kB/s | 260 kB 00:00 2024-05-20T22:05:39.425 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T22:05:39.479 INFO:teuthology.orchestra.run.smithi138.stdout:(157/159): elfutils-libelf-0.191-4.el9.x86_64.r 435 kB/s | 207 kB 00:00 2024-05-20T22:05:39.573 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T22:05:39.778 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T22:05:39.839 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T22:05:39.871 INFO:teuthology.orchestra.run.smithi138.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 2.4 MB 00:00 2024-05-20T22:05:39.948 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T22:05:40.039 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T22:05:40.064 INFO:teuthology.orchestra.run.smithi138.stdout:(159/159): openssl-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 1.3 MB 00:00 2024-05-20T22:05:40.066 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-20T22:05:40.067 INFO:teuthology.orchestra.run.smithi138.stdout:Total 11 MB/s | 222 MB 00:20 2024-05-20T22:05:40.257 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T22:05:40.365 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-20T22:05:40.469 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-20T22:05:40.513 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-20T22:05:40.547 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-20T22:05:40.622 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-20T22:05:40.663 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-20T22:05:40.706 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-20T22:05:40.739 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-20T22:05:40.766 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-20T22:05:40.838 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-20T22:05:40.890 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-20T22:05:40.919 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-20T22:05:40.963 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-20T22:05:41.042 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-20T22:05:41.074 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 34/170 2024-05-20T22:05:41.118 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 35/170 2024-05-20T22:05:41.146 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 36/170 2024-05-20T22:05:41.219 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-20T22:05:41.261 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-20T22:05:41.288 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-20T22:05:41.338 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-20T22:05:41.445 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-20T22:05:41.445 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-20T22:05:41.475 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-20T22:05:41.573 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-20T22:05:42.125 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-20T22:05:42.211 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-20T22:05:42.247 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-20T22:05:42.284 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-20T22:05:42.332 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-20T22:05:42.370 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 47/170 2024-05-20T22:05:42.413 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-20T22:05:42.575 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-20T22:05:42.640 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-20T22:05:42.696 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-20T22:05:42.769 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-20T22:05:42.807 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-20T22:05:42.843 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-20T22:05:43.000 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-20T22:05:43.035 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-20T22:05:43.171 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-20T22:05:43.213 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-20T22:05:43.356 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-20T22:05:43.405 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-20T22:05:43.447 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-20T22:05:43.544 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-20T22:05:43.579 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-20T22:05:43.621 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-20T22:05:43.625 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-20T22:05:43.626 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-20T22:05:43.653 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-20T22:05:43.678 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-20T22:05:43.765 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-20T22:05:43.902 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-20T22:05:43.944 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-20T22:05:43.973 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-20T22:05:44.006 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-20T22:05:44.046 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-20T22:05:44.109 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-20T22:05:44.142 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-20T22:05:44.177 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-20T22:05:44.210 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-20T22:05:44.244 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-20T22:05:44.270 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-20T22:05:44.360 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-20T22:05:44.410 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-20T22:05:44.440 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-20T22:05:44.489 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-20T22:05:44.598 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-20T22:05:44.660 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-20T22:05:45.419 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-20T22:05:45.615 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-20T22:05:45.649 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-20T22:05:45.752 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-20T22:05:45.805 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-20T22:05:45.851 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-20T22:05:45.940 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-20T22:05:45.985 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-20T22:05:46.168 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-20T22:05:46.199 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-20T22:05:46.282 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-20T22:05:47.111 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-20T22:05:47.228 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-20T22:05:48.545 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-20T22:05:48.639 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-20T22:05:48.668 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-20T22:05:48.701 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-20T22:05:49.037 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-20T22:05:49.080 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-20T22:05:49.155 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-20T22:05:49.185 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 104/170 2024-05-20T22:05:49.222 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-20T22:05:49.724 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-20T22:05:49.750 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-20T22:05:49.822 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-20T22:05:49.852 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/170 2024-05-20T22:05:50.222 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-20T22:05:50.294 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-20T22:05:50.331 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-20T22:05:50.395 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-20T22:05:50.435 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-20T22:05:50.461 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-20T22:05:50.506 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-20T22:05:50.928 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-20T22:05:50.971 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T22:05:51.095 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T22:05:51.122 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T22:05:51.219 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T22:05:51.243 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-20T22:05:51.298 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-20T22:05:51.500 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-20T22:05:51.550 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-20T22:05:51.578 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T22:05:51.707 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T22:05:51.729 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-20T22:05:51.792 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-20T22:05:51.814 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T22:05:51.838 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T22:05:51.990 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T22:05:52.169 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T22:05:52.198 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T22:05:52.233 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T22:05:52.303 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T22:05:52.363 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T22:05:52.468 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T22:05:52.539 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T22:05:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T22:05:52.821 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T22:05:52.853 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-20T22:05:52.869 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-20T22:05:52.945 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-20T22:05:52.945 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-20T22:05:52.946 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-20T22:05:52.946 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:05:52.956 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-20T22:05:52.980 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-20T22:05:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-20T22:05:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-20T22:05:53.094 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-20T22:05:53.129 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-20T22:05:53.172 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-20T22:05:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-20T22:05:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-20T22:05:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-20T22:05:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-20T22:05:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-20T22:05:53.437 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-20T22:05:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-20T22:05:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 34/170 2024-05-20T22:05:53.591 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 35/170 2024-05-20T22:05:53.620 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 36/170 2024-05-20T22:05:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-20T22:05:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-20T22:05:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-20T22:05:53.945 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-20T22:05:53.997 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-20T22:05:54.530 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-20T22:05:54.609 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-20T22:05:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-20T22:05:54.690 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-20T22:05:54.733 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-20T22:05:54.774 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 47/170 2024-05-20T22:05:54.813 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-20T22:05:54.968 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-20T22:05:55.033 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-20T22:05:55.088 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-20T22:05:55.162 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-20T22:05:55.200 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-20T22:05:55.237 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-20T22:05:55.284 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-20T22:05:55.328 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-20T22:05:55.371 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-20T22:05:55.413 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-20T22:05:55.550 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-20T22:05:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-20T22:05:55.640 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-20T22:05:55.735 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-20T22:05:55.773 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-20T22:05:55.815 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-20T22:05:55.847 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-20T22:05:55.872 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-20T22:05:55.965 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-20T22:05:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-20T22:05:56.120 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-20T22:05:56.155 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-20T22:05:56.183 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-20T22:05:56.230 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-20T22:05:56.262 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-20T22:05:56.291 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-20T22:05:56.351 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-20T22:05:56.384 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-20T22:05:56.426 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-20T22:05:56.460 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-20T22:05:56.548 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-20T22:05:56.591 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-20T22:05:56.630 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-20T22:05:56.678 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-20T22:05:56.776 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-20T22:05:56.832 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-20T22:05:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-20T22:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-20T22:05:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-20T22:05:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-20T22:05:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-20T22:05:57.846 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-20T22:05:57.934 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-20T22:05:57.982 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-20T22:05:58.160 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-20T22:05:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-20T22:05:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-20T22:05:58.932 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-20T22:05:59.044 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-20T22:06:00.313 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-20T22:06:00.391 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-20T22:06:00.419 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-20T22:06:00.452 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-20T22:06:00.771 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-20T22:06:00.815 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-20T22:06:00.906 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-20T22:06:00.937 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 104/170 2024-05-20T22:06:00.973 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-20T22:06:01.452 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-20T22:06:01.485 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-20T22:06:01.556 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-20T22:06:01.586 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/170 2024-05-20T22:06:03.772 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T22:06:03.816 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T22:06:04.376 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T22:06:04.419 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-20T22:06:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-20T22:06:04.495 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-20T22:06:04.495 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-20T22:06:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-20T22:06:05.534 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-20T22:06:05.534 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /sys 2024-05-20T22:06:05.534 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /proc 2024-05-20T22:06:05.535 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /mnt 2024-05-20T22:06:05.535 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /var/tmp 2024-05-20T22:06:05.535 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /home 2024-05-20T22:06:05.535 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /root 2024-05-20T22:06:05.535 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /tmp 2024-05-20T22:06:05.535 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:05.708 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-20T22:06:05.725 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-20T22:06:05.725 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:05.725 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-20T22:06:05.725 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:07.446 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-20T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-20T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-20T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:07.714 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-20T22:06:07.738 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-20T22:06:07.738 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:07.738 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-20T22:06:07.738 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T22:06:07.738 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T22:06:07.738 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:08.387 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-20T22:06:08.446 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-20T22:06:08.509 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-20T22:06:08.713 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-20T22:06:08.767 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-20T22:06:08.848 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-20T22:06:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-20T22:06:09.089 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-20T22:06:09.126 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-20T22:06:09.163 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-20T22:06:09.202 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-20T22:06:09.650 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-20T22:06:09.702 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-20T22:06:09.733 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 128/170 2024-05-20T22:06:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 129/170 2024-05-20T22:06:09.825 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-20T22:06:10.913 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-20T22:06:10.951 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-20T22:06:11.967 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-20T22:06:11.994 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-20T22:06:12.103 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-20T22:06:12.274 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 133/170 2024-05-20T22:06:12.313 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-20T22:06:12.339 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-20T22:06:12.339 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:12.339 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-20T22:06:12.339 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T22:06:12.339 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T22:06:12.339 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:12.389 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-20T22:06:12.401 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-20T22:06:12.502 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 136/170 2024-05-20T22:06:12.559 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/170 2024-05-20T22:06:13.516 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-20T22:06:13.550 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-20T22:06:13.575 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-20T22:06:13.575 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:13.575 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-20T22:06:13.575 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T22:06:13.575 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T22:06:13.576 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:13.617 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-20T22:06:13.641 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-20T22:06:13.642 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:13.642 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-20T22:06:13.642 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:13.693 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-20T22:06:13.718 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-20T22:06:13.718 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:13.718 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-20T22:06:13.718 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T22:06:13.719 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T22:06:13.719 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:14.065 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-20T22:06:14.089 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-20T22:06:14.089 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:14.090 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-20T22:06:14.090 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T22:06:14.090 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T22:06:14.090 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:16.188 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-20T22:06:16.189 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-05-20T22:06:16.189 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-05-20T22:06:16.189 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-05-20T22:06:16.189 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-05-20T22:06:16.189 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-05-20T22:06:16.189 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-05-20T22:06:16.189 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-05-20T22:06:16.189 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:16.359 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-20T22:06:16.374 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-20T22:06:16.374 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:16.374 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-20T22:06:16.375 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:17.962 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-20T22:06:17.988 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-20T22:06:17.988 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:17.988 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-20T22:06:17.989 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T22:06:17.989 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T22:06:17.989 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:18.212 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-20T22:06:18.236 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-20T22:06:18.236 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:18.236 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-20T22:06:18.236 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T22:06:18.236 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T22:06:18.236 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:18.713 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-20T22:06:18.736 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-20T22:06:18.736 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:18.736 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-20T22:06:18.736 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T22:06:18.736 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T22:06:18.736 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:18.843 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-20T22:06:18.901 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-20T22:06:18.929 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/170 2024-05-20T22:06:18.956 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-20T22:06:18.973 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 144/170 2024-05-20T22:06:19.022 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 145/170 2024-05-20T22:06:19.053 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-20T22:06:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-20T22:06:19.143 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-20T22:06:19.197 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-20T22:06:19.276 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-20T22:06:19.278 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 148/170 2024-05-20T22:06:19.327 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-20T22:06:19.364 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-20T22:06:19.366 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 150/170 2024-05-20T22:06:19.410 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-20T22:06:19.438 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-20T22:06:19.492 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T22:06:19.512 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-20T22:06:19.555 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-20T22:06:19.592 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-20T22:06:19.623 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-20T22:06:20.084 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-20T22:06:20.114 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-20T22:06:20.145 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 128/170 2024-05-20T22:06:20.145 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T22:06:20.181 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T22:06:20.203 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T22:06:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 129/170 2024-05-20T22:06:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-20T22:06:21.109 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 155/170 2024-05-20T22:06:21.203 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-20T22:06:21.246 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-20T22:06:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-20T22:06:21.330 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-20T22:06:21.347 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-20T22:06:21.381 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-20T22:06:21.406 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T22:06:21.432 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-20T22:06:21.458 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-20T22:06:21.458 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T22:06:21.535 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T22:06:21.567 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-20T22:06:21.568 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T22:06:21.652 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T22:06:21.677 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-20T22:06:21.683 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T22:06:21.683 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T22:06:21.871 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T22:06:21.871 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T22:06:21.955 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T22:06:21.988 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-20T22:06:22.057 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T22:06:22.301 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-20T22:06:22.330 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-20T22:06:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-20T22:06:22.595 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 133/170 2024-05-20T22:06:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-20T22:06:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-20T22:06:22.648 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:22.648 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-20T22:06:22.648 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T22:06:22.648 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T22:06:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:22.691 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-20T22:06:22.702 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-20T22:06:22.802 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 136/170 2024-05-20T22:06:22.862 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/170 2024-05-20T22:06:23.778 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-20T22:06:23.810 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-20T22:06:23.833 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-20T22:06:23.833 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:23.833 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-20T22:06:23.833 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T22:06:23.834 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T22:06:23.834 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:23.877 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-20T22:06:23.900 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-20T22:06:23.900 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:23.900 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-20T22:06:23.901 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:23.953 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-20T22:06:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-20T22:06:23.976 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:23.976 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-20T22:06:23.976 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T22:06:23.976 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T22:06:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:24.287 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-20T22:06:24.309 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-20T22:06:24.310 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T22:06:24.310 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-20T22:06:24.310 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T22:06:24.310 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T22:06:24.310 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:27.896 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T22:06:27.896 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/170 2024-05-20T22:06:27.896 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/170 2024-05-20T22:06:27.896 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/170 2024-05-20T22:06:27.896 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/170 2024-05-20T22:06:27.896 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/170 2024-05-20T22:06:27.896 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/170 2024-05-20T22:06:27.896 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/170 2024-05-20T22:06:27.896 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/170 2024-05-20T22:06:27.897 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/170 2024-05-20T22:06:27.898 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-20T22:06:27.899 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-20T22:06:27.900 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-20T22:06:27.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-20T22:06:27.902 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-20T22:06:27.903 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-20T22:06:27.904 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-20T22:06:27.905 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-20T22:06:27.906 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-20T22:06:27.907 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-20T22:06:27.908 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-20T22:06:27.909 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-20T22:06:28.894 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/170 2024-05-20T22:06:28.935 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 144/170 2024-05-20T22:06:28.976 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 145/170 2024-05-20T22:06:29.009 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-20T22:06:29.044 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-20T22:06:29.245 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 148/170 2024-05-20T22:06:29.291 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-20T22:06:29.333 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 150/170 2024-05-20T22:06:29.374 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-20T22:06:29.403 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-20T22:06:29.441 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T22:06:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T22:06:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T22:06:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T22:06:30.637 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-20T22:06:30.637 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:30.637 INFO:teuthology.orchestra.run.smithi055.stdout:Upgraded: 2024-05-20T22:06:30.637 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-20T22:06:30.637 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-20T22:06:30.638 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-20T22:06:30.638 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-20T22:06:30.638 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-20T22:06:30.638 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-20T22:06:30.638 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-20T22:06:30.638 INFO:teuthology.orchestra.run.smithi055.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.638 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.638 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout: automake-1.16.2-8.el9.noarch 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout: bison-3.7.4-5.el9.x86_64 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.639 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.640 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: flex-2.6.4-9.el9.x86_64 2024-05-20T22:06:30.641 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.642 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-20T22:06:30.643 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: lua-5.4.4-4.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-13.14-1.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-20T22:06:30.644 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.645 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-20T22:06:30.646 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-20T22:06:30.647 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ply-3.11-14.el9.noarch 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-20T22:06:30.648 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-20T22:06:30.649 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-20T22:06:30.650 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: re2-1:20211101-3.el9.x86_64 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-20T22:06:30.651 INFO:teuthology.orchestra.run.smithi055.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-20T22:06:30.652 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-20T22:06:30.652 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-20T22:06:30.652 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-20T22:06:30.652 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:30.652 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-05-20T22:06:30.981 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 155/170 2024-05-20T22:06:31.065 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-20T22:06:31.107 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-20T22:06:31.186 DEBUG:teuthology.parallel:result is None 2024-05-20T22:06:31.209 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-20T22:06:31.244 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-20T22:06:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T22:06:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-20T22:06:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-20T22:06:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T22:06:31.406 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T22:06:31.439 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-20T22:06:31.440 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T22:06:31.524 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T22:06:31.582 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-20T22:06:31.588 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T22:06:31.588 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T22:06:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T22:06:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T22:06:31.853 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T22:06:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-20T22:06:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T22:06:38.071 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T22:06:38.071 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/170 2024-05-20T22:06:38.071 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/170 2024-05-20T22:06:38.071 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/170 2024-05-20T22:06:38.071 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/170 2024-05-20T22:06:38.071 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/170 2024-05-20T22:06:38.072 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/170 2024-05-20T22:06:38.073 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/170 2024-05-20T22:06:38.073 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/170 2024-05-20T22:06:38.073 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/170 2024-05-20T22:06:38.073 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/170 2024-05-20T22:06:38.073 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/170 2024-05-20T22:06:38.073 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/170 2024-05-20T22:06:38.073 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/170 2024-05-20T22:06:38.073 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/170 2024-05-20T22:06:38.073 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/170 2024-05-20T22:06:38.074 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/170 2024-05-20T22:06:38.074 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/170 2024-05-20T22:06:38.091 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-20T22:06:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-20T22:06:38.093 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-20T22:06:38.094 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-20T22:06:38.095 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-20T22:06:38.096 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-20T22:06:38.097 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-20T22:06:38.098 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-20T22:06:38.099 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-20T22:06:38.100 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-20T22:06:38.101 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-20T22:06:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-20T22:06:40.784 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-20T22:06:40.784 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:40.784 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-05-20T22:06:40.784 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-20T22:06:40.784 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-20T22:06:40.784 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-20T22:06:40.784 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-20T22:06:40.784 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-20T22:06:40.785 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:40.786 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-20T22:06:40.787 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-20T22:06:40.788 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-20T22:06:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-20T22:06:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-20T22:06:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-20T22:06:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-20T22:06:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-20T22:06:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-20T22:06:40.789 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.14-1.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-20T22:06:40.790 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-20T22:06:40.791 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-20T22:06:40.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-20T22:06:40.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-20T22:06:40.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-20T22:06:40.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-20T22:06:40.796 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-20T22:06:40.797 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-20T22:06:40.797 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:40.797 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-20T22:06:41.248 DEBUG:teuthology.parallel:result is None 2024-05-20T22:06:41.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T22:06:41.464 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-20T22:06:41.491 INFO:teuthology.orchestra.run.smithi055.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-20T22:06:41.492 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-20T22:06:41.492 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-20T22:06:41.494 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T22:06:41.709 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-20T22:06:41.736 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-20T22:06:41.736 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-20T22:06:41.736 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-20T22:06:41.738 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-20T22:06:41.738 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:41.739 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-20T22:06:41.772 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:06:41.772 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-20T22:06:41.807 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-20T22:06:41.807 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:41.808 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/daemon-helper 2024-05-20T22:06:41.842 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-20T22:06:41.912 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:06:41.912 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-05-20T22:06:41.940 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-20T22:06:42.008 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-20T22:06:42.008 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:42.009 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-20T22:06:42.039 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-20T22:06:42.109 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:06:42.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-20T22:06:42.137 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-20T22:06:42.205 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-20T22:06:42.205 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:42.206 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/stdin-killer 2024-05-20T22:06:42.231 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-20T22:06:42.298 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:06:42.298 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-05-20T22:06:42.325 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-20T22:06:42.391 INFO:teuthology.run_tasks:Running task ceph... 2024-05-20T22:06:42.474 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-20T22:06:42.474 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 777 /var/log/ceph 2024-05-20T22:06:42.477 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-05-20T22:06:42.503 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-20T22:06:42.503 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-20T22:06:42.543 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-20T22:06:42.568 INFO:tasks.ceph:Creating extra log directories... 2024-05-20T22:06:42.569 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-20T22:06:42.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-20T22:06:42.640 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-20T22:06:42.640 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-20T22:06:42.640 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-20T22:06:42.640 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:42.640 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-20T22:06:42.688 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-20T22:06:42.804 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-20T22:06:42.831 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:06:42.831 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-20T22:06:42.846 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-20T22:06:42.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-20T22:06:42.987 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-20T22:06:42.988 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-20T22:06:42.988 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767', 'branch': 'wip-pdonnell-testing-20240517.203521-squid-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/openfiletable}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7715767', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240517.203521-squid-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240517.203521-squid-debug', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9749, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'sleep_before_teardown': 0, 'subset': '272/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240517.203521-squid-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi055.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHw6HAWdrgYYZB7wGfkSyqajCmebJ2L13MuGFBKYoqq8BznT+sq9XvWggFreDu94QVVUyPK+Lt3yAimoA6wRcig=', 'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPjRcuToZueiuBO0foeE4lEP0eY7/VoO8+VRK6lAMBgw1lf/s0bi+wUHazN0iyPii5XbS8ms8l+T3mbr+dRdH/A='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_openfiletable']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a', 'timestamp': '2024-05-20_15:44:59', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-20T22:06:42.988 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-20T22:06:42.990 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-20T22:06:43.038 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m0777 -- /var/run/ceph 2024-05-20T22:06:43.046 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-05-20T22:06:43.102 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:43.102 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2024-05-20T22:06:43.126 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-20T22:06:43.126 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_1 2024-05-20T22:06:43.182 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-20T22:06:43.182 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T22:06:43.182 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-20T22:06:43.182 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T22:06:43.182 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T22:06:43.183 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-20 22:06:22.389942296 +0000 2024-05-20T22:06:43.183 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-20 22:03:46.028852737 +0000 2024-05-20T22:06:43.183 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-20 22:03:46.028852737 +0000 2024-05-20T22:06:43.183 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-20 22:03:46.028852737 +0000 2024-05-20T22:06:43.183 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-20T22:06:43.246 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:06:43.246 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:06:43.247 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000286399 s, 1.8 MB/s 2024-05-20T22:06:43.248 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-20T22:06:43.305 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_2 2024-05-20T22:06:43.362 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-20T22:06:43.362 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T22:06:43.362 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-20T22:06:43.362 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T22:06:43.362 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T22:06:43.362 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-20 22:06:22.389942296 +0000 2024-05-20T22:06:43.363 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-20 22:03:46.367846214 +0000 2024-05-20T22:06:43.363 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-20 22:03:46.367846214 +0000 2024-05-20T22:06:43.363 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-20 22:03:46.367846214 +0000 2024-05-20T22:06:43.363 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-20T22:06:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:06:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:06:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000242086 s, 2.1 MB/s 2024-05-20T22:06:43.428 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-20T22:06:43.485 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_3 2024-05-20T22:06:43.541 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-20T22:06:43.542 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T22:06:43.542 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-20T22:06:43.542 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T22:06:43.542 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T22:06:43.542 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-20 22:06:22.389942296 +0000 2024-05-20T22:06:43.542 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-20 22:03:46.714839536 +0000 2024-05-20T22:06:43.542 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-20 22:03:46.714839536 +0000 2024-05-20T22:06:43.542 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-20 22:03:46.714839536 +0000 2024-05-20T22:06:43.542 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-20T22:06:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:06:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:06:43.606 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000267365 s, 1.9 MB/s 2024-05-20T22:06:43.607 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-20T22:06:43.663 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_4 2024-05-20T22:06:43.719 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-20T22:06:43.719 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T22:06:43.720 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-20T22:06:43.720 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T22:06:43.720 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T22:06:43.720 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-20 22:06:22.390942277 +0000 2024-05-20T22:06:43.720 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-20 22:03:47.070832685 +0000 2024-05-20T22:06:43.720 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-20 22:03:47.070832685 +0000 2024-05-20T22:06:43.720 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-20 22:03:47.070832685 +0000 2024-05-20T22:06:43.720 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-20T22:06:43.783 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:06:43.783 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:06:43.783 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000262855 s, 1.9 MB/s 2024-05-20T22:06:43.785 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-20T22:06:43.842 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-20T22:06:43.842 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:06:43.842 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-05-20T22:06:43.859 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-20T22:06:43.859 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-05-20T22:06:43.915 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-20T22:06:43.915 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T22:06:43.915 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-20T22:06:43.915 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T22:06:43.915 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T22:06:43.915 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-20 22:06:32.308137945 +0000 2024-05-20T22:06:43.915 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-20 22:03:46.025588421 +0000 2024-05-20T22:06:43.915 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-20 22:03:46.025588421 +0000 2024-05-20T22:06:43.915 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-20 22:03:46.025588421 +0000 2024-05-20T22:06:43.916 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-20T22:06:43.980 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-20T22:06:43.980 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-20T22:06:43.980 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000264996 s, 1.9 MB/s 2024-05-20T22:06:43.981 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-20T22:06:44.038 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-05-20T22:06:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-20T22:06:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T22:06:44.095 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-20T22:06:44.095 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T22:06:44.095 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T22:06:44.095 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-20 22:06:32.308137945 +0000 2024-05-20T22:06:44.095 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-20 22:03:46.364581109 +0000 2024-05-20T22:06:44.095 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-20 22:03:46.364581109 +0000 2024-05-20T22:06:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-20 22:03:46.364581109 +0000 2024-05-20T22:06:44.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-20T22:06:44.158 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-20T22:06:44.158 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-20T22:06:44.158 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000242439 s, 2.1 MB/s 2024-05-20T22:06:44.159 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-20T22:06:44.216 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-05-20T22:06:44.271 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-20T22:06:44.272 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T22:06:44.272 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-20T22:06:44.272 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T22:06:44.272 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T22:06:44.272 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-20 22:06:32.309137922 +0000 2024-05-20T22:06:44.272 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-20 22:03:46.711573624 +0000 2024-05-20T22:06:44.272 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-20 22:03:46.711573624 +0000 2024-05-20T22:06:44.272 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-20 22:03:46.711573624 +0000 2024-05-20T22:06:44.272 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-20T22:06:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-20T22:06:44.335 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-20T22:06:44.335 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000201695 s, 2.5 MB/s 2024-05-20T22:06:44.336 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-20T22:06:44.392 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-05-20T22:06:44.447 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-20T22:06:44.447 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T22:06:44.447 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-20T22:06:44.447 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T22:06:44.447 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T22:06:44.447 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-20 22:06:32.309137922 +0000 2024-05-20T22:06:44.447 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-20 22:03:47.040566528 +0000 2024-05-20T22:06:44.447 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-20 22:03:47.040566528 +0000 2024-05-20T22:06:44.447 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-20 22:03:47.040566528 +0000 2024-05-20T22:06:44.448 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-20T22:06:44.511 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-20T22:06:44.511 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-20T22:06:44.511 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000213549 s, 2.4 MB/s 2024-05-20T22:06:44.512 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-20T22:06:44.569 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-20T22:06:44.569 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi055.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-20T22:06:44.569 INFO:tasks.ceph:Generating config... 2024-05-20T22:06:44.570 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-20T22:06:44.570 INFO:tasks.ceph:[client] debug client = 20 2024-05-20T22:06:44.570 INFO:tasks.ceph:[client] debug ms = 1 2024-05-20T22:06:44.570 INFO:tasks.ceph:[client] pid file = 2024-05-20T22:06:44.571 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-20T22:06:44.571 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-20T22:06:44.571 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-20T22:06:44.572 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-20T22:06:44.572 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-20T22:06:44.572 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-20T22:06:44.572 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-20T22:06:44.572 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-20T22:06:44.572 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-20T22:06:44.572 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-20T22:06:44.572 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-20T22:06:44.572 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-20T22:06:44.572 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-20T22:06:44.572 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-20T22:06:44.572 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-20T22:06:44.572 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-20T22:06:44.572 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-20T22:06:44.573 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-20T22:06:44.573 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-20T22:06:44.573 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-20T22:06:44.573 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-20T22:06:44.573 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-20T22:06:44.573 INFO:tasks.ceph:Setting up mon.a... 2024-05-20T22:06:44.573 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-20T22:06:44.659 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.keyring 2024-05-20T22:06:44.663 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-20T22:06:44.723 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-20T22:06:44.750 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.55'), ('mon.b', '172.21.15.138'), ('mon.c', '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]')] 2024-05-20T22:06:44.750 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-20T22:06:44.751 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:44.751 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-20T22:06:44.809 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.55 --add b 172.21.15.138 --addv c '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:06:44.908 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:06:44.908 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: generated fsid de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:06:44.908 INFO:teuthology.orchestra.run.smithi055.stdout:setting min_mon_release = quincy 2024-05-20T22:06:44.908 INFO:teuthology.orchestra.run.smithi055.stdout:epoch 0 2024-05-20T22:06:44.908 INFO:teuthology.orchestra.run.smithi055.stdout:fsid de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:06:44.908 INFO:teuthology.orchestra.run.smithi055.stdout:last_changed 2024-05-20T22:06:44.908172+0000 2024-05-20T22:06:44.908 INFO:teuthology.orchestra.run.smithi055.stdout:created 2024-05-20T22:06:44.908172+0000 2024-05-20T22:06:44.908 INFO:teuthology.orchestra.run.smithi055.stdout:min_mon_release 17 (quincy) 2024-05-20T22:06:44.908 INFO:teuthology.orchestra.run.smithi055.stdout:election_strategy: 1 2024-05-20T22:06:44.909 INFO:teuthology.orchestra.run.smithi055.stdout:0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2024-05-20T22:06:44.909 INFO:teuthology.orchestra.run.smithi055.stdout:1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-05-20T22:06:44.909 INFO:teuthology.orchestra.run.smithi055.stdout:2: [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] mon.c 2024-05-20T22:06:44.909 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-20T22:06:44.910 DEBUG:teuthology.orchestra.run.smithi055:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-20T22:06:44.925 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID de8ca569-3fd1-4cb2-8bdc-99f367745927... 2024-05-20T22:06:44.926 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T22:06:44.968 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T22:06:45.010 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2024-05-20T22:06:45.010 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2024-05-20T22:06:45.010 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T22:06:45.010 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2024-05-20T22:06:45.010 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.010 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.011 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: mon_allow_pool_size_one = true 2024-05-20T22:06:45.012 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T22:06:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2024-05-20T22:06:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2024-05-20T22:06:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2024-05-20T22:06:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2024-05-20T22:06:45.013 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: osd mclock profile = high_recovery_ops 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2024-05-20T22:06:45.014 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug trim objects = true 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2024-05-20T22:06:45.015 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:06:45.016 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-20T22:06:45.017 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2024-05-20T22:06:45.018 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: # 1m isn't quite enough 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: mon_down_mkfs_grace = 2m 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_filestore_osds = false 2024-05-20T22:06:45.019 INFO:teuthology.orchestra.run.smithi055.stdout: mon down mkfs grace = 300 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = "" 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-20T22:06:45.020 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds balancer = 20 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2024-05-20T22:06:45.021 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2024-05-20T22:06:45.022 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2024-05-20T22:06:45.022 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-20T22:06:45.022 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-20T22:06:45.022 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-20T22:06:45.023 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-05-20T22:06:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-05-20T22:06:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-05-20T22:06:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-05-20T22:06:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-05-20T22:06:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-05-20T22:06:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-20T22:06:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:06:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: mon down mkfs grace = 300 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-05-20T22:06:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-05-20T22:06:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-05-20T22:06:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-05-20T22:06:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:06:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-05-20T22:06:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-05-20T22:06:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-05-20T22:06:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-05-20T22:06:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-20T22:06:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-20T22:06:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-20T22:06:45.035 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-20T22:06:45.036 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-20T22:06:45.130 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-20T22:06:45.130 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:45.130 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-20T22:06:45.146 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:45.146 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-20T22:06:45.202 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:06:45.202 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:45.202 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-20T22:06:45.203 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-20T22:06:45.279 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:06:45.279 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:06:45.334 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:06:45.334 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:06:45.335 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-20T22:06:45.335 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-20T22:06:45.370 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:06:45.370 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:06:45.425 INFO:tasks.ceph:Setting up mon nodes... 2024-05-20T22:06:45.425 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-20T22:06:45.425 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-20T22:06:45.487 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-20T22:06:45.491 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-20T22:06:45.568 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-20T22:06:45.571 INFO:tasks.ceph:Setting up mds nodes... 2024-05-20T22:06:45.571 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-20T22:06:45.634 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-20T22:06:45.637 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-20T22:06:45.662 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-20T22:06:45.764 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-20T22:06:45.767 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-20T22:06:45.792 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-20T22:06:45.852 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-20T22:06:45.855 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-20T22:06:45.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-20T22:06:45.979 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-20T22:06:45.982 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-20T22:06:46.007 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-20T22:06:46.007 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-20T22:06:46.059 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-20T22:06:46.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-20T22:06:46.125 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-20T22:06:46.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-20T22:06:46.190 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-20T22:06:46.204 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-20T22:06:46.254 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-20T22:06:46.267 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-20T22:06:46.268 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi055.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-20T22:06:46.268 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-20T22:06:46.293 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-20T22:06:46.293 INFO:tasks.ceph:role: osd.0 2024-05-20T22:06:46.293 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:06:46.294 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-20T22:06:46.388 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T22:06:46.389 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T22:06:46.389 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T22:06:46.389 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T22:06:46.389 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T22:06:46.389 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2024-05-20T22:06:46.389 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T22:06:46.389 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T22:06:46.389 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T22:06:46.389 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T22:06:47.703 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2024-05-20T22:06:47.706 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2024-05-20T22:06:47.706 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-20T22:06:47.743 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-20T22:06:47.818 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-20T22:06:47.887 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-20T22:06:47.887 INFO:tasks.ceph:role: osd.1 2024-05-20T22:06:47.887 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:06:47.887 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-20T22:06:47.983 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T22:06:47.983 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T22:06:47.983 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T22:06:47.983 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T22:06:47.983 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T22:06:47.984 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2024-05-20T22:06:47.984 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T22:06:47.984 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T22:06:47.984 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T22:06:47.984 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T22:06:49.343 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2024-05-20T22:06:49.345 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2024-05-20T22:06:49.345 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-20T22:06:49.384 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-20T22:06:49.457 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-20T22:06:49.525 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-20T22:06:49.525 INFO:tasks.ceph:role: osd.2 2024-05-20T22:06:49.525 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:06:49.525 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-20T22:06:49.622 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T22:06:49.622 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T22:06:49.622 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T22:06:49.622 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T22:06:49.622 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T22:06:49.622 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2024-05-20T22:06:49.622 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T22:06:49.623 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T22:06:49.623 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T22:06:49.623 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T22:06:50.936 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2024-05-20T22:06:50.939 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2024-05-20T22:06:50.939 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-20T22:06:50.975 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-20T22:06:51.048 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-20T22:06:51.118 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-20T22:06:51.118 INFO:tasks.ceph:role: osd.3 2024-05-20T22:06:51.118 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:06:51.118 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-20T22:06:51.214 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T22:06:51.214 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T22:06:51.214 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T22:06:51.214 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T22:06:51.214 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T22:06:51.215 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2024-05-20T22:06:51.215 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T22:06:51.215 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T22:06:51.215 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T22:06:51.215 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T22:06:52.538 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2024-05-20T22:06:52.540 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2024-05-20T22:06:52.540 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-20T22:06:52.577 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-20T22:06:52.650 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:06:52.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:52.776+0000 7f0b4dfc4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-20T22:06:52.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:52.777+0000 7f0b4dfc4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-20T22:06:52.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:52.777+0000 7f0b4dfc4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-20T22:06:52.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:52.777+0000 7f0b4dfc4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-20T22:06:52.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:52.777+0000 7f0b4dfc4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-20T22:06:52.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:52.778+0000 7f0b4dfc4640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-20T22:06:55.646 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-20T22:06:55.676 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:06:55.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:55.782+0000 7f13ace1d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-20T22:06:55.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:55.782+0000 7f13ace1d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-20T22:06:55.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:55.782+0000 7f13ace1d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-20T22:06:55.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:55.782+0000 7f13ace1d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-20T22:06:55.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:55.782+0000 7f13ace1d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-20T22:06:55.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:55.783+0000 7f13ace1d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-20T22:06:58.666 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-20T22:06:58.697 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:06:58.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:58.801+0000 7f2852c0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-20T22:06:58.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:58.802+0000 7f2852c0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-20T22:06:58.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:58.802+0000 7f2852c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-20T22:06:58.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:58.802+0000 7f2852c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-20T22:06:58.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:58.802+0000 7f2852c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-20T22:06:58.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:06:58.802+0000 7f2852c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-20T22:07:01.679 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-20T22:07:01.710 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:07:01.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:01.814+0000 7f7493019640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-20T22:07:01.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:01.814+0000 7f7493019640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-20T22:07:01.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:01.814+0000 7f7493019640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-20T22:07:01.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:01.814+0000 7f7493019640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-20T22:07:01.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:01.814+0000 7f7493019640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-20T22:07:01.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:01.815+0000 7f7493019640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-20T22:07:04.688 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-20T22:07:04.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-20T22:07:04.750 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-20T22:07:04.750 INFO:tasks.ceph:role: osd.4 2024-05-20T22:07:04.750 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:07:04.751 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-20T22:07:04.826 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T22:07:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T22:07:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T22:07:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T22:07:04.827 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T22:07:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-05-20T22:07:04.827 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T22:07:04.827 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T22:07:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T22:07:04.827 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T22:07:05.319 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-05-20T22:07:05.322 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-05-20T22:07:05.323 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-20T22:07:05.431 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-20T22:07:05.467 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-20T22:07:05.534 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-20T22:07:05.535 INFO:tasks.ceph:role: osd.5 2024-05-20T22:07:05.535 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:07:05.535 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-20T22:07:05.608 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T22:07:05.608 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T22:07:05.608 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T22:07:05.608 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T22:07:05.608 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T22:07:05.608 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-05-20T22:07:05.609 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T22:07:05.609 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T22:07:05.609 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T22:07:05.609 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-05-20T22:07:06.068 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-05-20T22:07:06.068 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-20T22:07:06.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-20T22:07:06.192 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-20T22:07:06.257 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-20T22:07:06.257 INFO:tasks.ceph:role: osd.6 2024-05-20T22:07:06.258 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:07:06.258 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-20T22:07:06.330 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T22:07:06.330 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T22:07:06.330 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T22:07:06.330 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T22:07:06.330 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T22:07:06.331 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-05-20T22:07:06.331 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T22:07:06.331 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T22:07:06.331 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T22:07:06.331 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T22:07:06.812 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-05-20T22:07:06.814 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-05-20T22:07:06.814 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-20T22:07:06.933 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-20T22:07:06.962 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-20T22:07:07.027 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-20T22:07:07.027 INFO:tasks.ceph:role: osd.7 2024-05-20T22:07:07.027 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:07:07.028 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-20T22:07:07.100 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T22:07:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T22:07:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T22:07:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T22:07:07.101 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T22:07:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-05-20T22:07:07.101 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T22:07:07.101 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T22:07:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T22:07:07.101 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T22:07:07.611 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-05-20T22:07:07.613 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-05-20T22:07:07.613 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-20T22:07:07.744 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-20T22:07:07.775 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:07:07.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:07.892+0000 7ff79381b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-20T22:07:07.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:07.893+0000 7ff79381b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-20T22:07:07.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:07.893+0000 7ff79381b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-20T22:07:07.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:07.893+0000 7ff79381b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-20T22:07:07.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:07.893+0000 7ff79381b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-20T22:07:07.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:07.893+0000 7ff79381b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-20T22:07:10.745 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-20T22:07:10.775 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:07:10.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:10.876+0000 7f739de0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-20T22:07:10.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:10.876+0000 7f739de0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-20T22:07:10.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:10.876+0000 7f739de0d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-20T22:07:10.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:10.876+0000 7f739de0d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-20T22:07:10.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:10.876+0000 7f739de0d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-20T22:07:10.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:10.877+0000 7f739de0d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-20T22:07:12.990 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:07:12.993 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:07:13.726 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-20T22:07:13.752 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:07:13.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:13.848+0000 7f4c973d6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-20T22:07:13.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:13.848+0000 7f4c973d6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-20T22:07:13.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:13.848+0000 7f4c973d6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-20T22:07:13.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:13.848+0000 7f4c973d6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-20T22:07:13.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:13.848+0000 7f4c973d6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-20T22:07:13.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:13.849+0000 7f4c973d6640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-20T22:07:16.697 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-20T22:07:16.726 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:07:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:16.826+0000 7f35b480f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-20T22:07:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:16.826+0000 7f35b480f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-20T22:07:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:16.826+0000 7f35b480f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-20T22:07:16.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:16.826+0000 7f35b480f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-20T22:07:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:16.826+0000 7f35b480f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-20T22:07:16.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:16.827+0000 7f35b480f640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-20T22:07:19.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-20T22:07:19.704 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-20T22:07:19.705 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:19.705 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-20T22:07:19.737 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:19.737 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-20T22:07:19.806 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:19.806 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-20T22:07:19.875 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:19.875 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-20T22:07:19.943 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:19.944 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-20T22:07:20.011 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:20.011 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-20T22:07:20.079 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:20.079 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-20T22:07:20.146 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:20.147 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-20T22:07:20.176 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:20.176 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-20T22:07:20.244 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:20.245 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-20T22:07:20.312 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:20.312 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-20T22:07:20.379 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:20.379 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-20T22:07:20.445 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:20.445 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-20T22:07:20.511 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:20.511 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-20T22:07:20.577 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:20.577 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-20T22:07:20.597 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:20.597 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-20T22:07:20.656 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:20.656 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-20T22:07:20.672 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:20.672 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-20T22:07:20.728 INFO:tasks.ceph:Adding keys to all mons... 2024-05-20T22:07:20.728 DEBUG:teuthology.orchestra.run.smithi055:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-20T22:07:20.730 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-20T22:07:20.771 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr.y] 2024-05-20T22:07:20.771 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB1yUtmXhIUHRAAzFxrH0wPo95yWm4KoEFLvw== 2024-05-20T22:07:20.771 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2024-05-20T22:07:20.771 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB1yUtmnF7NJRAAIHaudq8A1NdLVw4llE1GKA== 2024-05-20T22:07:20.771 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2024-05-20T22:07:20.771 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB1yUtmKe+YLRAAagJbIVjyJ+R1oZEBhZyGiQ== 2024-05-20T22:07:20.771 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.0] 2024-05-20T22:07:20.771 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB8yUtm5SdYLhAAleTxIMKS9IsQjv4FusrryA== 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.1] 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB/yUtm7zGqLhAAU1uR214d/bSdRlcl5frlYA== 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.2] 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQCCyUtmwBfXLxAAwRbdPpoNxTIoCSDsNdZOKQ== 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.3] 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQCFyUtmTcaaMBAAQCJ6epS4s2FPsSvRBFdcLg== 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr.x] 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB1yUtmwQ/lIRAAuSUOiqQA/GU0trg5dUB8Sg== 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB1yUtmNDXWMhAAQU8KHijh4cVysa71xLvMjg== 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB1yUtm6mtqOhAA6mTeQoWYP+yUvHt/szLnlw== 2024-05-20T22:07:20.772 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.4] 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQCLyUtmNsNGNRAAQA8pn/2GQ4FPNxu3zsS9JQ== 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.5] 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQCOyUtmdmFLNBAAUH+yBiv2QSbfBqob0d6X8Q== 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.6] 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQCRyUtmVZGkMhAAmI118TqDPXtUvsH8We4hyg== 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.7] 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQCUyUtmNmlRMRAAUL0mlbzjAXGdx7oOZeKQ/w== 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout:[client.0] 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB2yUtm592RAxAAiaDYBHgkpVfsQHm+jLOy3g== 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout:[client.2] 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB2yUtmDwN6BxAAuS4b7g8QxESuhrW9d39wMQ== 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout:[client.1] 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB2yUtmNXZWCxAArgL/Kf6oP/vHiIhC/4f/kg== 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout:[client.3] 2024-05-20T22:07:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB2yUtmIsoxDxAAdHDRYPZ5HeYfqye4wIR2wQ== 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB1yUtmXhIUHRAAzFxrH0wPo95yWm4KoEFLvw== 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB1yUtmnF7NJRAAIHaudq8A1NdLVw4llE1GKA== 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB1yUtmKe+YLRAAagJbIVjyJ+R1oZEBhZyGiQ== 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB8yUtm5SdYLhAAleTxIMKS9IsQjv4FusrryA== 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB/yUtm7zGqLhAAU1uR214d/bSdRlcl5frlYA== 2024-05-20T22:07:20.791 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCCyUtmwBfXLxAAwRbdPpoNxTIoCSDsNdZOKQ== 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCFyUtmTcaaMBAAQCJ6epS4s2FPsSvRBFdcLg== 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB1yUtmwQ/lIRAAuSUOiqQA/GU0trg5dUB8Sg== 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB1yUtmNDXWMhAAQU8KHijh4cVysa71xLvMjg== 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB1yUtm6mtqOhAA6mTeQoWYP+yUvHt/szLnlw== 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.4] 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCLyUtmNsNGNRAAQA8pn/2GQ4FPNxu3zsS9JQ== 2024-05-20T22:07:20.792 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.5] 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCOyUtmdmFLNBAAUH+yBiv2QSbfBqob0d6X8Q== 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.6] 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCRyUtmVZGkMhAAmI118TqDPXtUvsH8We4hyg== 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.7] 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCUyUtmNmlRMRAAUL0mlbzjAXGdx7oOZeKQ/w== 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB2yUtm592RAxAAiaDYBHgkpVfsQHm+jLOy3g== 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout:[client.2] 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB2yUtmDwN6BxAAuS4b7g8QxESuhrW9d39wMQ== 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB2yUtmNXZWCxAArgL/Kf6oP/vHiIhC/4f/kg== 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout:[client.3] 2024-05-20T22:07:20.793 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB2yUtmIsoxDxAAdHDRYPZ5HeYfqye4wIR2wQ== 2024-05-20T22:07:20.794 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-20T22:07:20.817 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-20T22:07:20.895 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T22:07:20.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T22:07:20.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T22:07:20.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T22:07:21.030 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.032 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.093 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.155 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.157 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.218 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.220 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.281 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-20T22:07:21.283 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-20T22:07:21.344 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T22:07:21.346 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T22:07:21.409 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T22:07:21.411 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T22:07:21.476 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.478 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.539 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.541 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.601 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.604 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.665 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.667 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T22:07:21.728 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T22:07:21.730 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T22:07:21.794 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T22:07:21.796 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T22:07:21.860 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T22:07:21.862 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T22:07:21.926 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T22:07:21.928 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T22:07:21.992 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-20T22:07:21.992 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-20T22:07:22.017 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-20T22:07:22.134 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-20T22:07:22.159 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-20T22:07:22.185 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-20T22:07:22.292 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-20T22:07:22.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-20T22:07:22.383 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-20T22:07:22.491 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-20T22:07:22.517 DEBUG:teuthology.orchestra.run.smithi055:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:07:22.519 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-20T22:07:22.573 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-20T22:07:22.574 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-20T22:07:22.574 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-20T22:07:22.577 INFO:tasks.ceph.mon.a:Started 2024-05-20T22:07:22.577 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-20T22:07:22.577 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-20T22:07:22.616 INFO:tasks.ceph.mon.b:Started 2024-05-20T22:07:22.616 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-20T22:07:22.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-20T22:07:22.618 INFO:tasks.ceph.mon.c:Started 2024-05-20T22:07:22.618 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-20T22:07:22.619 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-20T22:07:22.619 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-20T22:07:22.621 INFO:tasks.ceph.mgr.y:Started 2024-05-20T22:07:22.621 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-20T22:07:22.621 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-20T22:07:22.623 INFO:tasks.ceph.mgr.x:Started 2024-05-20T22:07:22.623 INFO:tasks.ceph:Setting crush tunables to default 2024-05-20T22:07:22.623 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd crush tunables default 2024-05-20T22:07:22.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.738+0000 7f51bad2c640 1 Processor -- start 2024-05-20T22:07:22.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.738+0000 7f51bad2c640 1 -- start start 2024-05-20T22:07:22.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51bad2c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:22.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51bad2c640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b40b31a0 con 0x7f51b40d1e80 2024-05-20T22:07:22.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51bad2c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b406e1f0 con 0x7f51b40b2d90 2024-05-20T22:07:22.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51bad2c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51b409a8c0 con 0x7f51b40cd140 2024-05-20T22:07:22.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51b3fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f51b40d1e80 0x7f51b415b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38134/0 (socket says 172.21.15.55:38134) 2024-05-20T22:07:22.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 learned_addr learned my addr 172.21.15.55:0/1037249630 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:22.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51b92c9640 1 -- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 msgr2=0x7f51b409a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-20T22:07:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51b92c9640 1 -- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 msgr2=0x7f51b409a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.740+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.941+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.941+0000 7f51b92c9640 1 -- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 msgr2=0x7f51b409a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-20T22:07:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.941+0000 7f51b92c9640 1 -- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 msgr2=0x7f51b409a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.941+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:22.941+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T22:07:23.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:23.342+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:23.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:23.342+0000 7f51b92c9640 1 -- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 msgr2=0x7f51b409a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-20T22:07:23.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:23.342+0000 7f51b92c9640 1 -- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 msgr2=0x7f51b409a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:23.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:23.342+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:23.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:23.342+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-20T22:07:24.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:24.143+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:24.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:24.143+0000 7f51b92c9640 1 -- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 msgr2=0x7f51b409a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-20T22:07:24.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:24.143+0000 7f51b92c9640 1 -- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 msgr2=0x7f51b409a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:24.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:24.143+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:24.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:24.143+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-20T22:07:25.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b27fc640 1 -- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 msgr2=0x7f51b409a380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b27fc640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40cd140 0x7f51b409a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b27fc640 1 -- 172.21.15.55:0/1037249630 >> v1:172.21.15.138:6789/0 conn(0x7f51b40b2d90 legacy=0x7f51b40d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b27fc640 1 -- 172.21.15.55:0/1037249630 >> v1:172.21.15.55:6789/0 conn(0x7f51b40d1e80 legacy=0x7f51b415b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b27fc640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b27fc640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b27fc640 1 -- 172.21.15.55:0/1037249630 --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f51b409a8c0 con 0x7f5194008420 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b27fc640 1 -- 172.21.15.55:0/1037249630 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f519400afa0 con 0x7f5194005920 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 msgr2=0x7f519400a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:25.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 msgr2=0x7f519400a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:25.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:25.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.740+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:25.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.741+0000 7f51b8ac8640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 msgr2=0x7f5194007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T22:07:25.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.741+0000 7f51b8ac8640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 msgr2=0x7f5194007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:25.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.741+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:25.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.741+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:25.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.941+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:25.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.941+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 msgr2=0x7f519400a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T22:07:25.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.941+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 msgr2=0x7f519400a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:25.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.941+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:25.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.941+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T22:07:25.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.942+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:25.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.942+0000 7f51b8ac8640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 msgr2=0x7f5194007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T22:07:25.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.942+0000 7f51b8ac8640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 msgr2=0x7f5194007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:25.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.942+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:25.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:25.942+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T22:07:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:26.342+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:26.342+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 msgr2=0x7f519400a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T22:07:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:26.342+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 msgr2=0x7f519400a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:26.342+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:26.342+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-20T22:07:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:26.343+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:26.343+0000 7f51b8ac8640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 msgr2=0x7f5194007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T22:07:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:26.343+0000 7f51b8ac8640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 msgr2=0x7f5194007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:26.343+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:26.343+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-20T22:07:27.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:27.143+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:27.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:27.143+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 msgr2=0x7f519400a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T22:07:27.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:27.143+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 msgr2=0x7f519400a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:27.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:27.143+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:27.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:27.143+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-20T22:07:27.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:27.144+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:27.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:27.144+0000 7f51b8ac8640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 msgr2=0x7f5194007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T22:07:27.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:27.144+0000 7f51b8ac8640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 msgr2=0x7f5194007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:27.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:27.144+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:27.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:27.144+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-20T22:07:28.048 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:28.048+0000 7fac4021b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T22:07:28.224 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:28.223+0000 7fac4021b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T22:07:28.280 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:28.279+0000 7fac4021b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T22:07:28.543 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:28.542+0000 7fac4021b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T22:07:28.591 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:28.590+0000 7fac4021b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T22:07:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b27fc640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 msgr2=0x7f5194007dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b27fc640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194005920 0x7f5194007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b27fc640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 msgr2=0x7f519400a8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b27fc640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194008420 0x7f519400a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b92c9640 1 -- 172.21.15.55:0/1037249630 reap_dead start 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b27fc640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194005dc0 0x7f51940088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b27fc640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194008e20 0x7f5194009300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b27fc640 1 -- 172.21.15.55:0/1037249630 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b409ab00 con 0x7f51b40b2d90 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b27fc640 1 -- 172.21.15.55:0/1037249630 --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f51b409a8c0 con 0x7f5194005dc0 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b27fc640 1 -- 172.21.15.55:0/1037249630 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f519400afa0 con 0x7f5194008e20 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.740+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.55:3300/0 conn(0x7f5194005dc0 0x7f51940088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.741+0000 7f51b92c9640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194008e20 0x7f5194009300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.741+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194008e20 msgr2=0x7f5194009300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.741+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194008e20 0x7f5194009300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.741+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 >> v1:172.21.15.138:6789/0 conn(0x7f51b40b2d90 legacy=0x7f519400d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:28.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.741+0000 7f51b3fff640 1 -- 172.21.15.55:0/1037249630 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519400ab10 con 0x7f5194005dc0 2024-05-20T22:07:28.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.741+0000 7f51b3fff640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194005dc0 0x7f51940088e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5194008360 tx=0x7f51a402b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954573b13049aa78 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:28.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.742+0000 7f51b37fe640 1 -- 172.21.15.55:0/1037249630 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4031070 con 0x7f5194005dc0 2024-05-20T22:07:28.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.742+0000 7f51b37fe640 1 -- 172.21.15.55:0/1037249630 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a4035400 con 0x7f5194005dc0 2024-05-20T22:07:28.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.742+0000 7f51b37fe640 1 -- 172.21.15.55:0/1037249630 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a40357f0 con 0x7f5194005dc0 2024-05-20T22:07:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.742+0000 7f51bad2c640 1 -- 172.21.15.55:0/1037249630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194005dc0 msgr2=0x7f51940088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.742+0000 7f51bad2c640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194005dc0 0x7f51940088e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5194008360 tx=0x7f51a402b380 comp rx=0 tx=0).stop 2024-05-20T22:07:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.742+0000 7f51bad2c640 1 -- 172.21.15.55:0/1037249630 shutdown_connections 2024-05-20T22:07:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.743+0000 7f51bad2c640 1 --2- 172.21.15.55:0/1037249630 >> v2:172.21.15.138:3300/0 conn(0x7f5194008e20 0x7f5194009300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.743+0000 7f51bad2c640 1 --2- 172.21.15.55:0/1037249630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194005dc0 0x7f51940088e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.743+0000 7f51bad2c640 1 -- 172.21.15.55:0/1037249630 >> 172.21.15.55:0/1037249630 conn(0x7f51b40b6500 msgr2=0x7f51b40b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.743+0000 7f51bad2c640 1 -- 172.21.15.55:0/1037249630 shutdown_connections 2024-05-20T22:07:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.743+0000 7f51bad2c640 1 -- 172.21.15.55:0/1037249630 wait complete. 2024-05-20T22:07:28.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.744+0000 7f51bad2c640 1 Processor -- start 2024-05-20T22:07:28.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.745+0000 7f51bad2c640 1 -- start start 2024-05-20T22:07:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51bad2c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5194005dc0 0x7f51b4156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51bad2c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194008e20 0x7f51b40aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51bad2c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40b2d90 0x7f51b40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51bad2c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f51b406df10 con 0x7f5194008e20 2024-05-20T22:07:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51bad2c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51b406dc30 con 0x7f5194005dc0 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51bad2c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51b40b0ea0 con 0x7f51b40b2d90 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194008e20 0x7f51b40aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194008e20 0x7f51b40aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42946/0 (socket says 172.21.15.55:42946) 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b3fff640 1 -- 172.21.15.55:0/155437245 learned_addr learned my addr 172.21.15.55:0/155437245 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b92c9640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40b2d90 0x7f51b40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b8ac8640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5194005dc0 0x7f51b4156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b92c9640 1 -- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40b2d90 msgr2=0x7f51b40aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b92c9640 1 -- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40b2d90 msgr2=0x7f51b40aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b92c9640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40b2d90 0x7f51b40aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b92c9640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40b2d90 0x7f51b40aefc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b3fff640 1 -- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40b2d90 msgr2=0x7f51b40aefc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b3fff640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40b2d90 0x7f51b40aefc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b3fff640 1 -- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5194005dc0 msgr2=0x7f51b4156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:28.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.746+0000 7f51b3fff640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5194005dc0 0x7f51b4156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.747+0000 7f51b3fff640 1 -- 172.21.15.55:0/155437245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b40c7180 con 0x7f5194008e20 2024-05-20T22:07:28.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.747+0000 7f51b3fff640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194008e20 0x7f51b40aea80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f51a4000f80 tx=0x7f51a40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:28.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.747+0000 7f51b17fa640 1 -- 172.21.15.55:0/155437245 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4044020 con 0x7f5194008e20 2024-05-20T22:07:28.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.747+0000 7f51b17fa640 1 -- 172.21.15.55:0/155437245 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a4040670 con 0x7f5194008e20 2024-05-20T22:07:28.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.747+0000 7f51bad2c640 1 -- 172.21.15.55:0/155437245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b40c7680 con 0x7f5194008e20 2024-05-20T22:07:28.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.747+0000 7f51b17fa640 1 -- 172.21.15.55:0/155437245 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4040a40 con 0x7f5194008e20 2024-05-20T22:07:28.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.747+0000 7f51bad2c640 1 -- 172.21.15.55:0/155437245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b40c8870 con 0x7f5194008e20 2024-05-20T22:07:28.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.747+0000 7f51b17fa640 1 -- 172.21.15.55:0/155437245 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f51a4047020 con 0x7f5194008e20 2024-05-20T22:07:28.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.747+0000 7f51b17fa640 1 -- 172.21.15.55:0/155437245 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f51a403e9b0 con 0x7f5194008e20 2024-05-20T22:07:28.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.748+0000 7f51bad2c640 1 -- 172.21.15.55:0/155437245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b4074710 con 0x7f5194008e20 2024-05-20T22:07:28.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.750+0000 7f51b17fa640 1 -- 172.21.15.55:0/155437245 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f51a404e030 con 0x7f5194008e20 2024-05-20T22:07:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.817+0000 7f51bad2c640 1 -- 172.21.15.55:0/155437245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f51b40c73d0 con 0x7f5194008e20 2024-05-20T22:07:28.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.819+0000 7f51b17fa640 1 -- 172.21.15.55:0/155437245 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f51a403e440 con 0x7f5194008e20 2024-05-20T22:07:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:adjusted tunables profile to default 2024-05-20T22:07:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.820+0000 7f51bad2c640 1 -- 172.21.15.55:0/155437245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194008e20 msgr2=0x7f51b40aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.820+0000 7f51bad2c640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194008e20 0x7f51b40aea80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f51a4000f80 tx=0x7f51a40014a0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.820+0000 7f51bad2c640 1 -- 172.21.15.55:0/155437245 shutdown_connections 2024-05-20T22:07:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.820+0000 7f51bad2c640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40b2d90 0x7f51b40aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.820+0000 7f51bad2c640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194008e20 0x7f51b40aea80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.820+0000 7f51bad2c640 1 --2- 172.21.15.55:0/155437245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5194005dc0 0x7f51b4156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.820+0000 7f51bad2c640 1 -- 172.21.15.55:0/155437245 >> 172.21.15.55:0/155437245 conn(0x7f51b40b6500 msgr2=0x7f51b40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.820+0000 7f51bad2c640 1 -- 172.21.15.55:0/155437245 shutdown_connections 2024-05-20T22:07:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:28.820+0000 7f51bad2c640 1 -- 172.21.15.55:0/155437245 wait complete. 2024-05-20T22:07:28.827 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:28.826+0000 7f52cb22e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T22:07:28.832 INFO:tasks.ceph:check_enable_crimson: False 2024-05-20T22:07:28.832 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-20T22:07:28.832 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:28.833 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-20T22:07:28.859 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:28.859 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-20T22:07:28.927 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:28.927 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-20T22:07:28.991 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:28.991+0000 7f52cb22e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T22:07:28.995 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:07:28.995 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-20T22:07:29.044 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:29.043+0000 7f52cb22e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T22:07:29.060 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:29.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-20T22:07:29.086 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:29.086 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-20T22:07:29.151 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:29.151 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-20T22:07:29.216 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:07:29.216 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-20T22:07:29.281 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 2a65a30c-09c8-4932-b55c-3be3667d8acc 0 2024-05-20T22:07:29.292 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:29.291+0000 7f52cb22e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T22:07:29.340 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:29.340+0000 7f52cb22e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T22:07:29.389 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:29.388+0000 7fac4021b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T22:07:29.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.401+0000 7f2e5b614640 1 Processor -- start 2024-05-20T22:07:29.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.402+0000 7f2e5b614640 1 -- start start 2024-05-20T22:07:29.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e5b614640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 0x7f2e540d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e5b614640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e540af840 con 0x7f2e5406bd90 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e5b614640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e540afb20 con 0x7f2e5406e470 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e5b614640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e540afe00 con 0x7f2e540d08b0 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e58baf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 0x7f2e540d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e593b0640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2e5406e470 0x7f2e5409aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40312/0 (socket says 172.21.15.138:40312) 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e593b0640 1 -- 172.21.15.138:0/1645282838 learned_addr learned my addr 172.21.15.138:0/1645282838 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e58baf640 1 -- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 msgr2=0x7f2e540d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e58baf640 1 -- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 msgr2=0x7f2e540d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e58baf640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 0x7f2e540d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e58baf640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 0x7f2e540d0cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2288381790 0 0) 0x7f2e540afb20 con 0x7f2e5406e470 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.403+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e40003520 con 0x7f2e5406e470 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214822290 0 0) 0x7f2e540af840 con 0x7f2e5406bd90 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e540afb20 con 0x7f2e5406bd90 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 873570877 0 0) 0x7f2e40003520 con 0x7f2e5406e470 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 msgr2=0x7f2e540d0cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 0x7f2e540d0cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 >> v1:172.21.15.55:6789/0 conn(0x7f2e5406bd90 legacy=0x7f2e5406c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e540b00e0 con 0x7f2e5406e470 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f2e50001670 con 0x7f2e5406e470 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 >> v1:172.21.15.138:6789/0 conn(0x7f2e5406e470 legacy=0x7f2e5409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 0x7f2e40003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e400040a0 0x7f2e40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e40004d90 0x7f2e40005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2e540afe00 con 0x7f2e40004d90 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4000bcf0 con 0x7f2e400040a0 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4000bef0 con 0x7f2e540d08b0 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e58baf640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e400040a0 0x7f2e40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e593b0640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 0x7f2e40003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e59bb1640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e40004d90 0x7f2e40005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e58baf640 1 -- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 msgr2=0x7f2e40003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e58baf640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540d08b0 0x7f2e40003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e58baf640 1 -- 172.21.15.138:0/1645282838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e40004d90 msgr2=0x7f2e40005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e58baf640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e40004d90 0x7f2e40005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.404+0000 7f2e58baf640 1 -- 172.21.15.138:0/1645282838 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e40003520 con 0x7f2e400040a0 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.405+0000 7f2e58baf640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e400040a0 0x7f2e40004800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2e48006fd0 tx=0x7f2e40005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896ef174ffa30356 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.405+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48030070 con 0x7f2e400040a0 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.405+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4802ad70 con 0x7f2e400040a0 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.405+0000 7f2e47fff640 1 -- 172.21.15.138:0/1645282838 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4800b5e0 con 0x7f2e400040a0 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.406+0000 7f2e5b614640 1 -- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e400040a0 msgr2=0x7f2e40004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.406+0000 7f2e5b614640 1 --2- 172.21.15.138:0/1645282838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e400040a0 0x7f2e40004800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2e48006fd0 tx=0x7f2e40005830 comp rx=0 tx=0).stop 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.406+0000 7f2e59bb1640 1 -- 172.21.15.138:0/1645282838 reap_dead start 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.406+0000 7f2e5b614640 1 -- 172.21.15.138:0/1645282838 shutdown_connections 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.406+0000 7f2e5b614640 1 -- 172.21.15.138:0/1645282838 >> 172.21.15.138:0/1645282838 conn(0x7f2e540b6500 msgr2=0x7f2e540b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.406+0000 7f2e5b614640 1 -- 172.21.15.138:0/1645282838 shutdown_connections 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.406+0000 7f2e5b614640 1 -- 172.21.15.138:0/1645282838 wait complete. 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.408+0000 7f2e5b614640 1 Processor -- start 2024-05-20T22:07:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.408+0000 7f2e5b614640 1 -- start start 2024-05-20T22:07:29.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e5b614640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e400040a0 0x7f2e541f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e5b614640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e540cbe30 0x7f2e5414e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e5b614640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540cc240 0x7f2e5414ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e5b614640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2e540d0e40 con 0x7f2e400040a0 2024-05-20T22:07:29.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e5b614640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e540d0cc0 con 0x7f2e540cbe30 2024-05-20T22:07:29.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e5b614640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e540d0fc0 con 0x7f2e540cc240 2024-05-20T22:07:29.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e59bb1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540cc240 0x7f2e5414ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e58baf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e540cbe30 0x7f2e5414e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e59bb1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540cc240 0x7f2e5414ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:51342/0 (socket says 172.21.15.138:51342) 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e58baf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e540cbe30 0x7f2e5414e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39898/0 (socket says 172.21.15.138:39898) 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e59bb1640 1 -- 172.21.15.138:0/2953470186 learned_addr learned my addr 172.21.15.138:0/2953470186 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e593b0640 1 --2- 172.21.15.138:0/2953470186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e400040a0 0x7f2e541f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e59bb1640 1 -- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540cc240 msgr2=0x7f2e5414ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e59bb1640 1 -- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540cc240 msgr2=0x7f2e5414ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e59bb1640 1 --2- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540cc240 0x7f2e5414ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.409+0000 7f2e59bb1640 1 --2- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540cc240 0x7f2e5414ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e58baf640 1 -- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540cc240 msgr2=0x7f2e5414ea10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e58baf640 1 --2- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540cc240 0x7f2e5414ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e58baf640 1 -- 172.21.15.138:0/2953470186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e400040a0 msgr2=0x7f2e541f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e58baf640 1 --2- 172.21.15.138:0/2953470186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e400040a0 0x7f2e541f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e58baf640 1 -- 172.21.15.138:0/2953470186 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e540ca240 con 0x7f2e540cbe30 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e58baf640 1 --2- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e540cbe30 0x7f2e5414e4d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000bec0 tx=0x7f2e48004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e45ffb640 1 -- 172.21.15.138:0/2953470186 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48043020 con 0x7f2e540cbe30 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e45ffb640 1 -- 172.21.15.138:0/2953470186 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4803f680 con 0x7f2e540cbe30 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e5b614640 1 -- 172.21.15.138:0/2953470186 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e540ca520 con 0x7f2e540cbe30 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e45ffb640 1 -- 172.21.15.138:0/2953470186 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4800b5e0 con 0x7f2e540cbe30 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.410+0000 7f2e5b614640 1 -- 172.21.15.138:0/2953470186 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e540caae0 con 0x7f2e540cbe30 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.411+0000 7f2e45ffb640 1 -- 172.21.15.138:0/2953470186 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e48046020 con 0x7f2e540cbe30 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.411+0000 7f2e45ffb640 1 -- 172.21.15.138:0/2953470186 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f2e4803dc00 con 0x7f2e540cbe30 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.411+0000 7f2e5b614640 1 -- 172.21.15.138:0/2953470186 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e54074710 con 0x7f2e540cbe30 2024-05-20T22:07:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.414+0000 7f2e45ffb640 1 -- 172.21.15.138:0/2953470186 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2e4804d030 con 0x7f2e540cbe30 2024-05-20T22:07:29.481 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:29.481+0000 7fac4021b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T22:07:29.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.486+0000 7f2e5b614640 1 -- 172.21.15.138:0/2953470186 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2a65a30c-09c8-4932-b55c-3be3667d8acc", "id": 0} v 0) -- 0x7f2e54069090 con 0x7f2e540cbe30 2024-05-20T22:07:29.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.491+0000 7f2e45ffb640 1 -- 172.21.15.138:0/2953470186 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a65a30c-09c8-4932-b55c-3be3667d8acc", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f2e4803de50 con 0x7f2e540cbe30 2024-05-20T22:07:29.493 INFO:teuthology.orchestra.run.smithi138.stdout:0 2024-05-20T22:07:29.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.492+0000 7f2e5b614640 1 -- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e540cbe30 msgr2=0x7f2e5414e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.492+0000 7f2e5b614640 1 --2- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e540cbe30 0x7f2e5414e4d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000bec0 tx=0x7f2e48004730 comp rx=0 tx=0).stop 2024-05-20T22:07:29.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.492+0000 7f2e5b614640 1 -- 172.21.15.138:0/2953470186 shutdown_connections 2024-05-20T22:07:29.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.492+0000 7f2e5b614640 1 --2- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e540cc240 0x7f2e5414ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.492+0000 7f2e5b614640 1 --2- 172.21.15.138:0/2953470186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e540cbe30 0x7f2e5414e4d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.492+0000 7f2e5b614640 1 --2- 172.21.15.138:0/2953470186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e400040a0 0x7f2e541f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.492+0000 7f2e5b614640 1 -- 172.21.15.138:0/2953470186 >> 172.21.15.138:0/2953470186 conn(0x7f2e540b6500 msgr2=0x7f2e5406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:29.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.492+0000 7f2e5b614640 1 -- 172.21.15.138:0/2953470186 shutdown_connections 2024-05-20T22:07:29.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.492+0000 7f2e5b614640 1 -- 172.21.15.138:0/2953470186 wait complete. 2024-05-20T22:07:29.503 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 9087b53a-b367-45fe-bb5f-4d05def6d27c 1 2024-05-20T22:07:29.527 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:29.526+0000 7fac4021b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T22:07:29.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.613+0000 7fb417ed7640 1 Processor -- start 2024-05-20T22:07:29.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.613+0000 7fb417ed7640 1 -- start start 2024-05-20T22:07:29.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.614+0000 7fb417ed7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb41015f440 0x7fb4100b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.614+0000 7fb417ed7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41006bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.614+0000 7fb417ed7640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4100cee70 con 0x7fb4100cbf80 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.614+0000 7fb417ed7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb4100cc650 con 0x7fb41015f440 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.614+0000 7fb417ed7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4100cc8f0 con 0x7fb4100d09e0 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb415c73640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb41015f440 0x7fb4100b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.614+0000 7fb416474640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41006bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb415c73640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb41015f440 0x7fb4100b6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39904/0 (socket says 172.21.15.138:39904) 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb415c73640 1 -- 172.21.15.138:0/115421415 learned_addr learned my addr 172.21.15.138:0/115421415 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb416474640 1 -- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 msgr2=0x7fb41006bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb416474640 1 -- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 msgr2=0x7fb41006bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb416474640 1 --2- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41006bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb416474640 1 --2- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41006bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb415c73640 1 -- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 msgr2=0x7fb41006bba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:29.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb415c73640 1 --2- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41006bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb415c73640 1 -- 172.21.15.138:0/115421415 >> v1:172.21.15.55:6789/0 conn(0x7fb4100cbf80 legacy=0x7fb4100d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.615+0000 7fb415c73640 1 -- 172.21.15.138:0/115421415 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4100cc390 con 0x7fb41015f440 2024-05-20T22:07:29.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.616+0000 7fb415c73640 1 --2- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb41015f440 0x7fb4100b6470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009870 tx=0x7fb40c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=922d66e57cc99053 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:29.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.616+0000 7fb414c71640 1 -- 172.21.15.138:0/115421415 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c037070 con 0x7fb41015f440 2024-05-20T22:07:29.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.616+0000 7fb414c71640 1 -- 172.21.15.138:0/115421415 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c032440 con 0x7fb41015f440 2024-05-20T22:07:29.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.616+0000 7fb414c71640 1 -- 172.21.15.138:0/115421415 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c032810 con 0x7fb41015f440 2024-05-20T22:07:29.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.617+0000 7fb417ed7640 1 -- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb41015f440 msgr2=0x7fb4100b6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.617+0000 7fb417ed7640 1 --2- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb41015f440 0x7fb4100b6470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009870 tx=0x7fb40c02a270 comp rx=0 tx=0).stop 2024-05-20T22:07:29.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.617+0000 7fb417ed7640 1 -- 172.21.15.138:0/115421415 shutdown_connections 2024-05-20T22:07:29.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.617+0000 7fb417ed7640 1 --2- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41006bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.617+0000 7fb417ed7640 1 --2- 172.21.15.138:0/115421415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb41015f440 0x7fb4100b6470 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.617+0000 7fb417ed7640 1 -- 172.21.15.138:0/115421415 >> 172.21.15.138:0/115421415 conn(0x7fb41009a260 msgr2=0x7fb4100b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:29.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.617+0000 7fb417ed7640 1 -- 172.21.15.138:0/115421415 shutdown_connections 2024-05-20T22:07:29.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.617+0000 7fb417ed7640 1 -- 172.21.15.138:0/115421415 wait complete. 2024-05-20T22:07:29.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.618+0000 7fb417ed7640 1 Processor -- start 2024-05-20T22:07:29.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.619+0000 7fb417ed7640 1 -- start start 2024-05-20T22:07:29.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb417ed7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4100cbf80 0x7fb41015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb417ed7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41014e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb417ed7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb41015f440 0x7fb4100ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb417ed7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb41006c0e0 con 0x7fb41015f440 2024-05-20T22:07:29.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb417ed7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb41006b680 con 0x7fb4100cbf80 2024-05-20T22:07:29.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb417ed7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb41006c260 con 0x7fb4100d09e0 2024-05-20T22:07:29.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415c73640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4100cbf80 0x7fb41015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415c73640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4100cbf80 0x7fb41015db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39920/0 (socket says 172.21.15.138:39920) 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415472640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41014e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415c73640 1 -- 172.21.15.138:0/3583793177 learned_addr learned my addr 172.21.15.138:0/3583793177 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415472640 1 -- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 msgr2=0x7fb41014e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415472640 1 -- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 msgr2=0x7fb41014e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415472640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41014e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb416474640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb41015f440 0x7fb4100ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415472640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41014e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415c73640 1 -- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 msgr2=0x7fb41014e2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415c73640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41014e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415c73640 1 -- 172.21.15.138:0/3583793177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb41015f440 msgr2=0x7fb4100ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415c73640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb41015f440 0x7fb4100ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415c73640 1 -- 172.21.15.138:0/3583793177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4101f87b0 con 0x7fb4100cbf80 2024-05-20T22:07:29.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb416474640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb41015f440 0x7fb4100ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:07:29.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb415c73640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4100cbf80 0x7fb41015db50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009870 tx=0x7fb40c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:29.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.620+0000 7fb4067fc640 1 -- 172.21.15.138:0/3583793177 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c041020 con 0x7fb4100cbf80 2024-05-20T22:07:29.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.621+0000 7fb4067fc640 1 -- 172.21.15.138:0/3583793177 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c0306c0 con 0x7fb4100cbf80 2024-05-20T22:07:29.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.621+0000 7fb4067fc640 1 -- 172.21.15.138:0/3583793177 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c037040 con 0x7fb4100cbf80 2024-05-20T22:07:29.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.621+0000 7fb417ed7640 1 -- 172.21.15.138:0/3583793177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4101514a0 con 0x7fb4100cbf80 2024-05-20T22:07:29.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.621+0000 7fb417ed7640 1 -- 172.21.15.138:0/3583793177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb410151a60 con 0x7fb4100cbf80 2024-05-20T22:07:29.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.621+0000 7fb4067fc640 1 -- 172.21.15.138:0/3583793177 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb40c044020 con 0x7fb4100cbf80 2024-05-20T22:07:29.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.621+0000 7fb4067fc640 1 -- 172.21.15.138:0/3583793177 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fb40c03dd60 con 0x7fb4100cbf80 2024-05-20T22:07:29.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.622+0000 7fb417ed7640 1 -- 172.21.15.138:0/3583793177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb410074710 con 0x7fb4100cbf80 2024-05-20T22:07:29.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.624+0000 7fb4067fc640 1 -- 172.21.15.138:0/3583793177 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb40c04b030 con 0x7fb4100cbf80 2024-05-20T22:07:29.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.695+0000 7fb417ed7640 1 -- 172.21.15.138:0/3583793177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9087b53a-b367-45fe-bb5f-4d05def6d27c", "id": 1} v 0) -- 0x7fb410069090 con 0x7fb4100cbf80 2024-05-20T22:07:29.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.701+0000 7fb4067fc640 1 -- 172.21.15.138:0/3583793177 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9087b53a-b367-45fe-bb5f-4d05def6d27c", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fb40c03d420 con 0x7fb4100cbf80 2024-05-20T22:07:29.701 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-20T22:07:29.702 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:29.701+0000 7fac4021b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T22:07:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.702+0000 7fb417ed7640 1 -- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4100cbf80 msgr2=0x7fb41015db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.702+0000 7fb417ed7640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4100cbf80 0x7fb41015db50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009870 tx=0x7fb40c0046c0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.702+0000 7fb417ed7640 1 -- 172.21.15.138:0/3583793177 shutdown_connections 2024-05-20T22:07:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.702+0000 7fb417ed7640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb41015f440 0x7fb4100ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.702+0000 7fb417ed7640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4100d09e0 0x7fb41014e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.702+0000 7fb417ed7640 1 --2- 172.21.15.138:0/3583793177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4100cbf80 0x7fb41015db50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.702+0000 7fb417ed7640 1 -- 172.21.15.138:0/3583793177 >> 172.21.15.138:0/3583793177 conn(0x7fb41009a260 msgr2=0x7fb41006f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.702+0000 7fb417ed7640 1 -- 172.21.15.138:0/3583793177 shutdown_connections 2024-05-20T22:07:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.702+0000 7fb417ed7640 1 -- 172.21.15.138:0/3583793177 wait complete. 2024-05-20T22:07:29.714 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new f32d4168-f8e0-4ef1-b75e-604198b040ed 2 2024-05-20T22:07:29.752 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:29.752+0000 7fac4021b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T22:07:29.809 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:29.807+0000 7fac4021b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T22:07:29.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.826+0000 7f2d03377640 1 Processor -- start 2024-05-20T22:07:29.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.827+0000 7f2d03377640 1 -- start start 2024-05-20T22:07:29.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d03377640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2cfc09be80 0x7f2cfc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d03377640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc0b66b0 0x7f2cfc167630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d03377640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cfc0ced80 con 0x7f2cfc09ba70 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d03377640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f2cfc0d38a0 con 0x7f2cfc09be80 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d03377640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2cfc0d0cf0 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d01914640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc0b66b0 0x7f2cfc167630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d01914640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc0b66b0 0x7f2cfc167630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:51370/0 (socket says 172.21.15.138:51370) 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d01914640 1 -- 172.21.15.138:0/742269010 learned_addr learned my addr 172.21.15.138:0/742269010 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d00912640 1 --2- 172.21.15.138:0/742269010 >> v2:172.21.15.55:3300/0 conn(0x7f2cfc09be80 0x7f2cfc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d01914640 1 -- 172.21.15.138:0/742269010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc0b66b0 msgr2=0x7f2cfc167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d01914640 1 -- 172.21.15.138:0/742269010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc0b66b0 msgr2=0x7f2cfc167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d01914640 1 --2- 172.21.15.138:0/742269010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc0b66b0 0x7f2cfc167630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.828+0000 7f2d01914640 1 --2- 172.21.15.138:0/742269010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc0b66b0 0x7f2cfc167630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.829+0000 7f2d00912640 1 -- 172.21.15.138:0/742269010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc0b66b0 msgr2=0x7f2cfc167630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:29.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.829+0000 7f2d00912640 1 --2- 172.21.15.138:0/742269010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc0b66b0 0x7f2cfc167630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.829+0000 7f2d00912640 1 -- 172.21.15.138:0/742269010 >> v1:172.21.15.55:6789/0 conn(0x7f2cfc09ba70 legacy=0x7f2cfc0d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.829+0000 7f2cf3fff640 1 -- 172.21.15.138:0/742269010 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785343484 0 0) 0x7f2cfc0ced80 con 0x7f2cfc09ba70 2024-05-20T22:07:29.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.829+0000 7f2d00912640 1 -- 172.21.15.138:0/742269010 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc0b80c0 con 0x7f2cfc09be80 2024-05-20T22:07:29.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.829+0000 7f2d00912640 1 --2- 172.21.15.138:0/742269010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc09be80 0x7f2cfc0b6170 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0097a0 tx=0x7f2cec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2dc4ae7ae2e4c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:29.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2cf3fff640 1 -- 172.21.15.138:0/742269010 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec037070 con 0x7f2cfc09be80 2024-05-20T22:07:29.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2cf3fff640 1 -- 172.21.15.138:0/742269010 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cec032440 con 0x7f2cfc09be80 2024-05-20T22:07:29.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2cf3fff640 1 -- 172.21.15.138:0/742269010 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec032810 con 0x7f2cfc09be80 2024-05-20T22:07:29.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2d03377640 1 -- 172.21.15.138:0/742269010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc09be80 msgr2=0x7f2cfc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2d03377640 1 --2- 172.21.15.138:0/742269010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc09be80 0x7f2cfc0b6170 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0097a0 tx=0x7f2cec02a180 comp rx=0 tx=0).stop 2024-05-20T22:07:29.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2d03377640 1 -- 172.21.15.138:0/742269010 shutdown_connections 2024-05-20T22:07:29.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2d03377640 1 --2- 172.21.15.138:0/742269010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc0b66b0 0x7f2cfc167630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2d03377640 1 --2- 172.21.15.138:0/742269010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc09be80 0x7f2cfc0b6170 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2d03377640 1 -- 172.21.15.138:0/742269010 >> 172.21.15.138:0/742269010 conn(0x7f2cfc0cf3e0 msgr2=0x7f2cfc0cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:29.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2d03377640 1 -- 172.21.15.138:0/742269010 shutdown_connections 2024-05-20T22:07:29.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.830+0000 7f2d03377640 1 -- 172.21.15.138:0/742269010 wait complete. 2024-05-20T22:07:29.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.832+0000 7f2d03377640 1 Processor -- start 2024-05-20T22:07:29.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.832+0000 7f2d03377640 1 -- start start 2024-05-20T22:07:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.833+0000 7f2d03377640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc09ba70 0x7f2cfc09feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.833+0000 7f2d03377640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc09be80 0x7f2cfc09e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.833+0000 7f2d03377640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc0b66b0 0x7f2cfc09eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.833+0000 7f2d03377640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc0d09f0 con 0x7f2cfc09ba70 2024-05-20T22:07:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.833+0000 7f2d03377640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc0d1850 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.833+0000 7f2d03377640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2cfc0ce760 con 0x7f2cfc09be80 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d00912640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc09be80 0x7f2cfc09e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d00912640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc09be80 0x7f2cfc09e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:51374/0 (socket says 172.21.15.138:51374) 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d01914640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc0b66b0 0x7f2cfc09eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d00912640 1 -- 172.21.15.138:0/2458395902 learned_addr learned my addr 172.21.15.138:0/2458395902 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d01113640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc09ba70 0x7f2cfc09feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d00912640 1 -- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc09be80 msgr2=0x7f2cfc09e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d00912640 1 -- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc09be80 msgr2=0x7f2cfc09e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d00912640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc09be80 0x7f2cfc09e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d00912640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc09be80 0x7f2cfc09e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d01914640 1 -- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc09be80 msgr2=0x7f2cfc09e640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d01914640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc09be80 0x7f2cfc09e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d01914640 1 -- 172.21.15.138:0/2458395902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc09ba70 msgr2=0x7f2cfc09feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d01914640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc09ba70 0x7f2cfc09feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d01914640 1 -- 172.21.15.138:0/2458395902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc09d610 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d01113640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc09ba70 0x7f2cfc09feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d01914640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc0b66b0 0x7f2cfc09eb80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800d5a0 tx=0x7f2cf800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2cf1ffb640 1 -- 172.21.15.138:0/2458395902 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8004030 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2cf1ffb640 1 -- 172.21.15.138:0/2458395902 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf80041d0 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2d03377640 1 -- 172.21.15.138:0/2458395902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc1d8a20 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.834+0000 7f2cf1ffb640 1 -- 172.21.15.138:0/2458395902 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8014620 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.835+0000 7f2d03377640 1 -- 172.21.15.138:0/2458395902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc0a03f0 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.835+0000 7f2cf1ffb640 1 -- 172.21.15.138:0/2458395902 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2cf8014b50 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.835+0000 7f2cf1ffb640 1 -- 172.21.15.138:0/2458395902 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f2cf801daf0 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.836+0000 7f2d03377640 1 -- 172.21.15.138:0/2458395902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc0a05c0 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.838+0000 7f2cf1ffb640 1 -- 172.21.15.138:0/2458395902 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2cf801dd90 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.911+0000 7f2d03377640 1 -- 172.21.15.138:0/2458395902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f32d4168-f8e0-4ef1-b75e-604198b040ed", "id": 2} v 0) -- 0x7f2cfc0b3cb0 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.914+0000 7f2cf1ffb640 1 -- 172.21.15.138:0/2458395902 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f32d4168-f8e0-4ef1-b75e-604198b040ed", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f2cf801d7a0 con 0x7f2cfc0b66b0 2024-05-20T22:07:29.915 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-05-20T22:07:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.915+0000 7f2d03377640 1 -- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc0b66b0 msgr2=0x7f2cfc09eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.915+0000 7f2d03377640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc0b66b0 0x7f2cfc09eb80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800d5a0 tx=0x7f2cf800da60 comp rx=0 tx=0).stop 2024-05-20T22:07:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.915+0000 7f2d03377640 1 -- 172.21.15.138:0/2458395902 shutdown_connections 2024-05-20T22:07:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.915+0000 7f2d03377640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc0b66b0 0x7f2cfc09eb80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.915+0000 7f2d03377640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2cfc09be80 0x7f2cfc09e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.915+0000 7f2d03377640 1 --2- 172.21.15.138:0/2458395902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc09ba70 0x7f2cfc09feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.915+0000 7f2d03377640 1 -- 172.21.15.138:0/2458395902 >> 172.21.15.138:0/2458395902 conn(0x7f2cfc0cf3e0 msgr2=0x7f2cfc0d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:29.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.915+0000 7f2d03377640 1 -- 172.21.15.138:0/2458395902 shutdown_connections 2024-05-20T22:07:29.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:29.915+0000 7f2d03377640 1 -- 172.21.15.138:0/2458395902 wait complete. 2024-05-20T22:07:29.926 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:29.925+0000 7fac4021b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T22:07:29.927 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 3f99a86a-2fe8-407c-90fa-9cec443ba0dc 3 2024-05-20T22:07:30.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.037+0000 7f888cdb5640 1 Processor -- start 2024-05-20T22:07:30.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.037+0000 7f888cdb5640 1 -- start start 2024-05-20T22:07:30.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888cdb5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f888806e470 0x7f888809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888cdb5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880d08b0 0x7f88880d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888cdb5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f88880ae690 0x7f88880aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888cdb5640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f88880af840 con 0x7f88880ae690 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888cdb5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f88880afb20 con 0x7f888806e470 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888cdb5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f88880afe00 con 0x7f88880d08b0 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f8885d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880d08b0 0x7f88880d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888659c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f888806e470 0x7f888809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888659c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f888806e470 0x7f888809aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39936/0 (socket says 172.21.15.138:39936) 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f8885d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880d08b0 0x7f88880d0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:51384/0 (socket says 172.21.15.138:51384) 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888659c640 1 -- 172.21.15.138:0/2778809505 learned_addr learned my addr 172.21.15.138:0/2778809505 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f8886d9d640 1 --2- 172.21.15.138:0/2778809505 >> v2:172.21.15.55:3300/0 conn(0x7f88880ae690 0x7f88880aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f8885d9b640 1 -- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880d08b0 msgr2=0x7f88880d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f8885d9b640 1 -- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880d08b0 msgr2=0x7f88880d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f8885d9b640 1 --2- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880d08b0 0x7f88880d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f8885d9b640 1 --2- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880d08b0 0x7f88880d0cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888659c640 1 -- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880d08b0 msgr2=0x7f88880d0cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888659c640 1 --2- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880d08b0 0x7f88880d0cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888659c640 1 -- 172.21.15.138:0/2778809505 >> v2:172.21.15.55:3300/0 conn(0x7f88880ae690 msgr2=0x7f88880aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888659c640 1 --2- 172.21.15.138:0/2778809505 >> v2:172.21.15.55:3300/0 conn(0x7f88880ae690 0x7f88880aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.039+0000 7f888659c640 1 -- 172.21.15.138:0/2778809505 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88880b00e0 con 0x7f888806e470 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.040+0000 7f888659c640 1 --2- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888806e470 0x7f888809aa70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f88880b7510 tx=0x7f887802a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6fb58be43148f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.040+0000 7f888559a640 1 -- 172.21.15.138:0/2778809505 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878037070 con 0x7f888806e470 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.040+0000 7f888559a640 1 -- 172.21.15.138:0/2778809505 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887802ae00 con 0x7f888806e470 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.040+0000 7f888559a640 1 -- 172.21.15.138:0/2778809505 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878032660 con 0x7f888806e470 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.041+0000 7f888cdb5640 1 -- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888806e470 msgr2=0x7f888809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.041+0000 7f888cdb5640 1 --2- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888806e470 0x7f888809aa70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f88880b7510 tx=0x7f887802a0b0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.041+0000 7f888cdb5640 1 -- 172.21.15.138:0/2778809505 shutdown_connections 2024-05-20T22:07:30.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.041+0000 7f888cdb5640 1 --2- 172.21.15.138:0/2778809505 >> v2:172.21.15.55:3300/0 conn(0x7f88880ae690 0x7f88880aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.041+0000 7f888cdb5640 1 --2- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880d08b0 0x7f88880d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.041+0000 7f888cdb5640 1 --2- 172.21.15.138:0/2778809505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888806e470 0x7f888809aa70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.041+0000 7f888cdb5640 1 -- 172.21.15.138:0/2778809505 >> 172.21.15.138:0/2778809505 conn(0x7f88880b6500 msgr2=0x7f88880b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:30.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.041+0000 7f888cdb5640 1 -- 172.21.15.138:0/2778809505 shutdown_connections 2024-05-20T22:07:30.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.041+0000 7f888cdb5640 1 -- 172.21.15.138:0/2778809505 wait complete. 2024-05-20T22:07:30.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.043+0000 7f888cdb5640 1 Processor -- start 2024-05-20T22:07:30.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.043+0000 7f888cdb5640 1 -- start start 2024-05-20T22:07:30.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.044+0000 7f888cdb5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f888806e470 0x7f88881625b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.044+0000 7f888cdb5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880ae690 0x7f88880c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.044+0000 7f888cdb5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88880d08b0 0x7f8888156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.044+0000 7f888cdb5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f888815f330 con 0x7f888806e470 2024-05-20T22:07:30.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.044+0000 7f888cdb5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88880d1200 con 0x7f88880d08b0 2024-05-20T22:07:30.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.044+0000 7f888cdb5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f888815f4b0 con 0x7f88880ae690 2024-05-20T22:07:30.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8886d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88880d08b0 0x7f8888156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8885d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880ae690 0x7f88880c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f888659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f888806e470 0x7f88881625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8886d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88880d08b0 0x7f8888156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39942/0 (socket says 172.21.15.138:39942) 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8886d9d640 1 -- 172.21.15.138:0/546450065 learned_addr learned my addr 172.21.15.138:0/546450065 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8885d9b640 1 -- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880ae690 msgr2=0x7f88880c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8885d9b640 1 -- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880ae690 msgr2=0x7f88880c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8885d9b640 1 --2- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880ae690 0x7f88880c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8885d9b640 1 --2- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880ae690 0x7f88880c9190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f888659c640 1 -- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880ae690 msgr2=0x7f88880c9190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f888659c640 1 --2- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880ae690 0x7f88880c9190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f888659c640 1 -- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88880d08b0 msgr2=0x7f8888156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f888659c640 1 --2- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88880d08b0 0x7f8888156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f888659c640 1 -- 172.21.15.138:0/546450065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88880c7500 con 0x7f888806e470 2024-05-20T22:07:30.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8886d9d640 1 --2- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88880d08b0 0x7f8888156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f888659c640 1 --2- 172.21.15.138:0/546450065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f888806e470 0x7f88881625b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f88880afca0 tx=0x7f88780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8876ffd640 1 -- 172.21.15.138:0/546450065 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878041030 con 0x7f888806e470 2024-05-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f8876ffd640 1 -- 172.21.15.138:0/546450065 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887802ac00 con 0x7f888806e470 2024-05-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.045+0000 7f888cdb5640 1 -- 172.21.15.138:0/546450065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88880c86b0 con 0x7f888806e470 2024-05-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.046+0000 7f8876ffd640 1 -- 172.21.15.138:0/546450065 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878037040 con 0x7f888806e470 2024-05-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.046+0000 7f888cdb5640 1 -- 172.21.15.138:0/546450065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88881620e0 con 0x7f888806e470 2024-05-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.046+0000 7f8876ffd640 1 -- 172.21.15.138:0/546450065 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8878044070 con 0x7f888806e470 2024-05-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.046+0000 7f8876ffd640 1 -- 172.21.15.138:0/546450065 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f8878046a30 con 0x7f888806e470 2024-05-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.047+0000 7f888cdb5640 1 -- 172.21.15.138:0/546450065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8888074710 con 0x7f888806e470 2024-05-20T22:07:30.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.051+0000 7f8876ffd640 1 -- 172.21.15.138:0/546450065 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f887802add0 con 0x7f888806e470 2024-05-20T22:07:30.122 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:30.121+0000 7f52cb22e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T22:07:30.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.136+0000 7f888cdb5640 1 -- 172.21.15.138:0/546450065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3f99a86a-2fe8-407c-90fa-9cec443ba0dc", "id": 3} v 0) -- 0x7f88880c96d0 con 0x7f888806e470 2024-05-20T22:07:30.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.139+0000 7f8876ffd640 1 -- 172.21.15.138:0/546450065 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3f99a86a-2fe8-407c-90fa-9cec443ba0dc", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f8878046d50 con 0x7f888806e470 2024-05-20T22:07:30.139 INFO:teuthology.orchestra.run.smithi138.stdout:3 2024-05-20T22:07:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.140+0000 7f888cdb5640 1 -- 172.21.15.138:0/546450065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f888806e470 msgr2=0x7f88881625b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.140+0000 7f888cdb5640 1 --2- 172.21.15.138:0/546450065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f888806e470 0x7f88881625b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f88880afca0 tx=0x7f88780046c0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.140+0000 7f888cdb5640 1 -- 172.21.15.138:0/546450065 shutdown_connections 2024-05-20T22:07:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.140+0000 7f888cdb5640 1 --2- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88880d08b0 0x7f8888156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.140+0000 7f888cdb5640 1 --2- 172.21.15.138:0/546450065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88880ae690 0x7f88880c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.140+0000 7f888cdb5640 1 --2- 172.21.15.138:0/546450065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f888806e470 0x7f88881625b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.140+0000 7f888cdb5640 1 -- 172.21.15.138:0/546450065 >> 172.21.15.138:0/546450065 conn(0x7f88880b6500 msgr2=0x7f888806f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.140+0000 7f888cdb5640 1 -- 172.21.15.138:0/546450065 shutdown_connections 2024-05-20T22:07:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.140+0000 7f888cdb5640 1 -- 172.21.15.138:0/546450065 wait complete. 2024-05-20T22:07:30.151 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new d8a41f58-5245-4f29-98d5-316fb50a126a 4 2024-05-20T22:07:30.215 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:30.214+0000 7f52cb22e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T22:07:30.260 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:30.259+0000 7f52cb22e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T22:07:30.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.261+0000 7f1511b4b640 1 Processor -- start 2024-05-20T22:07:30.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.261+0000 7f1511b4b640 1 -- start start 2024-05-20T22:07:30.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f1511b4b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 0x7f150c0d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f1511b4b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150c0ced80 con 0x7f150c09be80 2024-05-20T22:07:30.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f1511b4b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150c0d0cf0 con 0x7f150c0d1680 2024-05-20T22:07:30.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f1511b4b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f150c0d38a0 con 0x7f150c09ba70 2024-05-20T22:07:30.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f150b7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 0x7f150c0d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f150affd640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f150c0d1680 0x7f150c0b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40320/0 (socket says 172.21.15.138:40320) 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f150affd640 1 -- 172.21.15.138:0/1966062012 learned_addr learned my addr 172.21.15.138:0/1966062012 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f150b7fe640 1 -- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 msgr2=0x7f150c0d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f150b7fe640 1 -- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 msgr2=0x7f150c0d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f150b7fe640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 0x7f150c0d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.262+0000 7f150b7fe640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 0x7f150c0d1140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2940214606 0 0) 0x7f150c0d0cf0 con 0x7f150c0d1680 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14f4003520 con 0x7f150c0d1680 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2111335753 0 0) 0x7f150c0ced80 con 0x7f150c09be80 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f150c0d0cf0 con 0x7f150c09be80 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3318286743 0 0) 0x7f14f4003520 con 0x7f150c0d1680 2024-05-20T22:07:30.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 msgr2=0x7f150c0d1140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 0x7f150c0d1140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 >> v1:172.21.15.55:6789/0 conn(0x7f150c09be80 legacy=0x7f150c167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c0b80c0 con 0x7f150c0d1680 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f1500001670 con 0x7f150c0d1680 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 >> v1:172.21.15.138:6789/0 conn(0x7f150c0d1680 legacy=0x7f150c0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14f4003b40 0x7f14f4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 0x7f14f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14f4004d40 0x7f14f40051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f150c0d38a0 con 0x7f14f4003b40 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14f400bb50 con 0x7f14f4004d40 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f14f400bd50 con 0x7f150c09ba70 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150affd640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 0x7f14f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150bfff640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14f4004d40 0x7f14f40051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150b7fe640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14f4003b40 0x7f14f4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150affd640 1 -- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14f4004d40 msgr2=0x7f14f40051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:30.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150affd640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14f4004d40 0x7f14f40051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150affd640 1 -- 172.21.15.138:0/1966062012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14f4003b40 msgr2=0x7f14f4003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:30.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150affd640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14f4003b40 0x7f14f4003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.263+0000 7f150affd640 1 -- 172.21.15.138:0/1966062012 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f4003520 con 0x7f150c09ba70 2024-05-20T22:07:30.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.264+0000 7f150affd640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 0x7f14f4004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f150000c190 tx=0x7f14f4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d08ab298417f0ffc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:30.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.264+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500007cc0 con 0x7f150c09ba70 2024-05-20T22:07:30.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.264+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500005ad0 con 0x7f150c09ba70 2024-05-20T22:07:30.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.264+0000 7f150a7fc640 1 -- 172.21.15.138:0/1966062012 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500007070 con 0x7f150c09ba70 2024-05-20T22:07:30.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.265+0000 7f1511b4b640 1 -- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 msgr2=0x7f14f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.265+0000 7f1511b4b640 1 --2- 172.21.15.138:0/1966062012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c09ba70 0x7f14f4004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f150000c190 tx=0x7f14f4005830 comp rx=0 tx=0).stop 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.265+0000 7f150bfff640 1 -- 172.21.15.138:0/1966062012 reap_dead start 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.265+0000 7f1511b4b640 1 -- 172.21.15.138:0/1966062012 shutdown_connections 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.265+0000 7f1511b4b640 1 -- 172.21.15.138:0/1966062012 >> 172.21.15.138:0/1966062012 conn(0x7f150c0cf3e0 msgr2=0x7f150c0cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.265+0000 7f1511b4b640 1 -- 172.21.15.138:0/1966062012 shutdown_connections 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.265+0000 7f1511b4b640 1 -- 172.21.15.138:0/1966062012 wait complete. 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.267+0000 7f1511b4b640 1 Processor -- start 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.267+0000 7f1511b4b640 1 -- start start 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.268+0000 7f1511b4b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f150c09ba70 0x7f150c1dab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.268+0000 7f1511b4b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c1e8310 0x7f150c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.268+0000 7f1511b4b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f150c0cb4d0 0x7f150c0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.268+0000 7f1511b4b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f150c0ce760 con 0x7f150c09ba70 2024-05-20T22:07:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.268+0000 7f1511b4b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f150c0d09f0 con 0x7f150c0cb4d0 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.268+0000 7f1511b4b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f150c0ce8e0 con 0x7f150c1e8310 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150affd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c1e8310 0x7f150c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150affd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c1e8310 0x7f150c06e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:51434/0 (socket says 172.21.15.138:51434) 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f150c0cb4d0 0x7f150c0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150affd640 1 -- 172.21.15.138:0/3699561419 learned_addr learned my addr 172.21.15.138:0/3699561419 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150b7fe640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f150c09ba70 0x7f150c1dab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150affd640 1 -- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c1e8310 msgr2=0x7f150c06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150affd640 1 -- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c1e8310 msgr2=0x7f150c06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150affd640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c1e8310 0x7f150c06e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150affd640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c1e8310 0x7f150c06e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150b7fe640 1 -- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c1e8310 msgr2=0x7f150c06e930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150b7fe640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c1e8310 0x7f150c06e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150b7fe640 1 -- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f150c0cb4d0 msgr2=0x7f150c0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150b7fe640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f150c0cb4d0 0x7f150c0cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150b7fe640 1 -- 172.21.15.138:0/3699561419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c0cbe20 con 0x7f150c09ba70 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150bfff640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f150c0cb4d0 0x7f150c0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f150b7fe640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f150c09ba70 0x7f150c1dab20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006fd0 tx=0x7f14fc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f14f3fff640 1 -- 172.21.15.138:0/3699561419 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc004240 con 0x7f150c09ba70 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f14f3fff640 1 -- 172.21.15.138:0/3699561419 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14fc01b070 con 0x7f150c09ba70 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f1511b4b640 1 -- 172.21.15.138:0/3699561419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c1db060 con 0x7f150c09ba70 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f14f3fff640 1 -- 172.21.15.138:0/3699561419 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc00e780 con 0x7f150c09ba70 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.269+0000 7f1511b4b640 1 -- 172.21.15.138:0/3699561419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c1e8b50 con 0x7f150c09ba70 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.270+0000 7f14f3fff640 1 -- 172.21.15.138:0/3699561419 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f14fc012040 con 0x7f150c09ba70 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.270+0000 7f14f3fff640 1 -- 172.21.15.138:0/3699561419 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f14fc016480 con 0x7f150c09ba70 2024-05-20T22:07:30.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.271+0000 7f1511b4b640 1 -- 172.21.15.138:0/3699561419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150c074710 con 0x7f150c09ba70 2024-05-20T22:07:30.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.275+0000 7f14f3fff640 1 -- 172.21.15.138:0/3699561419 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f150c0cbe20 con 0x7f150c09ba70 2024-05-20T22:07:30.334 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:30.333+0000 7fac4021b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T22:07:30.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.347+0000 7f1511b4b640 1 -- 172.21.15.138:0/3699561419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d8a41f58-5245-4f29-98d5-316fb50a126a", "id": 4} v 0) -- 0x7f150c06ee70 con 0x7f150c09ba70 2024-05-20T22:07:30.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.350+0000 7f14f3fff640 1 -- 172.21.15.138:0/3699561419 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d8a41f58-5245-4f29-98d5-316fb50a126a", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f14fc00ee20 con 0x7f150c09ba70 2024-05-20T22:07:30.351 INFO:teuthology.orchestra.run.smithi138.stdout:4 2024-05-20T22:07:30.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.351+0000 7f1511b4b640 1 -- 172.21.15.138:0/3699561419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f150c09ba70 msgr2=0x7f150c1dab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.351+0000 7f1511b4b640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f150c09ba70 0x7f150c1dab20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006fd0 tx=0x7f14fc010d50 comp rx=0 tx=0).stop 2024-05-20T22:07:30.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.352+0000 7f1511b4b640 1 -- 172.21.15.138:0/3699561419 shutdown_connections 2024-05-20T22:07:30.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.352+0000 7f1511b4b640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f150c0cb4d0 0x7f150c0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.352+0000 7f1511b4b640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150c1e8310 0x7f150c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.352+0000 7f1511b4b640 1 --2- 172.21.15.138:0/3699561419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f150c09ba70 0x7f150c1dab20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.352+0000 7f1511b4b640 1 -- 172.21.15.138:0/3699561419 >> 172.21.15.138:0/3699561419 conn(0x7f150c0cf3e0 msgr2=0x7f150c0716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:30.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.352+0000 7f1511b4b640 1 -- 172.21.15.138:0/3699561419 shutdown_connections 2024-05-20T22:07:30.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.352+0000 7f1511b4b640 1 -- 172.21.15.138:0/3699561419 wait complete. 2024-05-20T22:07:30.363 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new b6744a73-c861-4ce7-a303-5281e03b687d 5 2024-05-20T22:07:30.437 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:30.436+0000 7f52cb22e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T22:07:30.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.473+0000 7fa57e2e9640 1 Processor -- start 2024-05-20T22:07:30.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.473+0000 7fa57e2e9640 1 -- start start 2024-05-20T22:07:30.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57e2e9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5780d1730 0x7fa5780d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57e2e9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa57815d400 0x7fa57815d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57e2e9640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5780ce120 con 0x7fa5780cdd10 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57e2e9640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa5780b3220 con 0x7fa5780d1730 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57e2e9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa5780d2080 con 0x7fa57815d400 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57c886640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa57815d400 0x7fa57815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa56ffff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5780d1730 0x7fa5780d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa577fff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa5780cdd10 0x7fa5780b2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40332/0 (socket says 172.21.15.138:40332) 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa56ffff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5780d1730 0x7fa5780d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39978/0 (socket says 172.21.15.138:39978) 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa577fff640 1 -- 172.21.15.138:0/530398562 learned_addr learned my addr 172.21.15.138:0/530398562 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57c886640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa57815d400 0x7fa57815d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:51444/0 (socket says 172.21.15.138:51444) 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57c886640 1 -- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa57815d400 msgr2=0x7fa57815d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57c886640 1 -- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa57815d400 msgr2=0x7fa57815d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:30.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57c886640 1 --2- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa57815d400 0x7fa57815d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:30.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa57c886640 1 --2- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa57815d400 0x7fa57815d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:30.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa56ffff640 1 -- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa57815d400 msgr2=0x7fa57815d810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:30.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa56ffff640 1 --2- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa57815d400 0x7fa57815d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa56ffff640 1 -- 172.21.15.138:0/530398562 >> v1:172.21.15.138:6789/0 conn(0x7fa5780cdd10 legacy=0x7fa5780b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa5777fe640 1 -- 172.21.15.138:0/530398562 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1376327574 0 0) 0x7fa5780ce120 con 0x7fa5780cdd10 2024-05-20T22:07:30.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.475+0000 7fa56ffff640 1 -- 172.21.15.138:0/530398562 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5780ce3b0 con 0x7fa5780d1730 2024-05-20T22:07:30.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.476+0000 7fa56ffff640 1 --2- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780d1730 0x7fa5780d1b40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa5600097a0 tx=0x7fa56002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27aa68602aabf7bc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:30.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.476+0000 7fa5777fe640 1 -- 172.21.15.138:0/530398562 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560037070 con 0x7fa5780d1730 2024-05-20T22:07:30.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.476+0000 7fa5777fe640 1 -- 172.21.15.138:0/530398562 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56002ae50 con 0x7fa5780d1730 2024-05-20T22:07:30.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.476+0000 7fa5777fe640 1 -- 172.21.15.138:0/530398562 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560032660 con 0x7fa5780d1730 2024-05-20T22:07:30.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.477+0000 7fa57e2e9640 1 -- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780d1730 msgr2=0x7fa5780d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.477+0000 7fa57e2e9640 1 --2- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780d1730 0x7fa5780d1b40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa5600097a0 tx=0x7fa56002a100 comp rx=0 tx=0).stop 2024-05-20T22:07:30.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.477+0000 7fa57e2e9640 1 -- 172.21.15.138:0/530398562 shutdown_connections 2024-05-20T22:07:30.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.477+0000 7fa57e2e9640 1 --2- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa57815d400 0x7fa57815d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.477+0000 7fa57e2e9640 1 --2- 172.21.15.138:0/530398562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780d1730 0x7fa5780d1b40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.477+0000 7fa57e2e9640 1 -- 172.21.15.138:0/530398562 >> 172.21.15.138:0/530398562 conn(0x7fa5780b6500 msgr2=0x7fa5780b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:30.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.477+0000 7fa57e2e9640 1 -- 172.21.15.138:0/530398562 shutdown_connections 2024-05-20T22:07:30.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.477+0000 7fa57e2e9640 1 -- 172.21.15.138:0/530398562 wait complete. 2024-05-20T22:07:30.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.479+0000 7fa57e2e9640 1 Processor -- start 2024-05-20T22:07:30.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.479+0000 7fa57e2e9640 1 -- start start 2024-05-20T22:07:30.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.480+0000 7fa57e2e9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5780cdd10 0x7fa5780c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.480+0000 7fa57e2e9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780d1730 0x7fa5780b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.480+0000 7fa57e2e9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57815d400 0x7fa5781f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.480+0000 7fa57e2e9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa5780cec50 con 0x7fa57815d400 2024-05-20T22:07:30.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.480+0000 7fa57e2e9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5780ce970 con 0x7fa5780d1730 2024-05-20T22:07:30.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.480+0000 7fa57e2e9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa5780b1220 con 0x7fa5780cdd10 2024-05-20T22:07:30.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa577fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5780cdd10 0x7fa5780c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa56ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780d1730 0x7fa5780b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa577fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5780cdd10 0x7fa5780c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:51458/0 (socket says 172.21.15.138:51458) 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa577fff640 1 -- 172.21.15.138:0/3240387736 learned_addr learned my addr 172.21.15.138:0/3240387736 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa577fff640 1 -- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5780cdd10 msgr2=0x7fa5780c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa577fff640 1 -- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5780cdd10 msgr2=0x7fa5780c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa577fff640 1 --2- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5780cdd10 0x7fa5780c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa577fff640 1 --2- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5780cdd10 0x7fa5780c8890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa57c886640 1 --2- 172.21.15.138:0/3240387736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57815d400 0x7fa5781f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa56ffff640 1 -- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5780cdd10 msgr2=0x7fa5780c8890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa56ffff640 1 --2- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5780cdd10 0x7fa5780c8890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa56ffff640 1 -- 172.21.15.138:0/3240387736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57815d400 msgr2=0x7fa5781f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa56ffff640 1 --2- 172.21.15.138:0/3240387736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57815d400 0x7fa5781f2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa56ffff640 1 -- 172.21.15.138:0/3240387736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5780c9050 con 0x7fa5780d1730 2024-05-20T22:07:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa56ffff640 1 --2- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780d1730 0x7fa5780b0400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa5780743e0 tx=0x7fa5600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa5757fa640 1 -- 172.21.15.138:0/3240387736 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560041020 con 0x7fa5780d1730 2024-05-20T22:07:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa5757fa640 1 -- 172.21.15.138:0/3240387736 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56003c670 con 0x7fa5780d1730 2024-05-20T22:07:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa57e2e9640 1 -- 172.21.15.138:0/3240387736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5780c9330 con 0x7fa5780d1730 2024-05-20T22:07:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa5757fa640 1 -- 172.21.15.138:0/3240387736 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560037040 con 0x7fa5780d1730 2024-05-20T22:07:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.481+0000 7fa57e2e9640 1 -- 172.21.15.138:0/3240387736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5780c98f0 con 0x7fa5780d1730 2024-05-20T22:07:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.482+0000 7fa5757fa640 1 -- 172.21.15.138:0/3240387736 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa560044020 con 0x7fa5780d1730 2024-05-20T22:07:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.482+0000 7fa5757fa640 1 -- 172.21.15.138:0/3240387736 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fa560046400 con 0x7fa5780d1730 2024-05-20T22:07:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.483+0000 7fa57e2e9640 1 -- 172.21.15.138:0/3240387736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa578074710 con 0x7fa5780d1730 2024-05-20T22:07:30.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.485+0000 7fa5757fa640 1 -- 172.21.15.138:0/3240387736 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa56004b030 con 0x7fa5780d1730 2024-05-20T22:07:30.495 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:30.494+0000 7f52cb22e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T22:07:30.548 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:30.547+0000 7f52cb22e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T22:07:30.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.555+0000 7fa57e2e9640 1 -- 172.21.15.138:0/3240387736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b6744a73-c861-4ce7-a303-5281e03b687d", "id": 5} v 0) -- 0x7fa578069090 con 0x7fa5780d1730 2024-05-20T22:07:30.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.563+0000 7fa5757fa640 1 -- 172.21.15.138:0/3240387736 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b6744a73-c861-4ce7-a303-5281e03b687d", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fa56003d410 con 0x7fa5780d1730 2024-05-20T22:07:30.564 INFO:teuthology.orchestra.run.smithi138.stdout:5 2024-05-20T22:07:30.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.564+0000 7fa57e2e9640 1 -- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780d1730 msgr2=0x7fa5780b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.564+0000 7fa57e2e9640 1 --2- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780d1730 0x7fa5780b0400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa5780743e0 tx=0x7fa5600046c0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.564+0000 7fa57e2e9640 1 -- 172.21.15.138:0/3240387736 shutdown_connections 2024-05-20T22:07:30.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.564+0000 7fa57e2e9640 1 --2- 172.21.15.138:0/3240387736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa57815d400 0x7fa5781f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.564+0000 7fa57e2e9640 1 --2- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5780d1730 0x7fa5780b0400 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.564+0000 7fa57e2e9640 1 --2- 172.21.15.138:0/3240387736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5780cdd10 0x7fa5780c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.564+0000 7fa57e2e9640 1 -- 172.21.15.138:0/3240387736 >> 172.21.15.138:0/3240387736 conn(0x7fa5780b6500 msgr2=0x7fa5780b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:30.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.564+0000 7fa57e2e9640 1 -- 172.21.15.138:0/3240387736 shutdown_connections 2024-05-20T22:07:30.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.564+0000 7fa57e2e9640 1 -- 172.21.15.138:0/3240387736 wait complete. 2024-05-20T22:07:30.576 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 6541c8c6-e672-44e2-9604-253abe988813 6 2024-05-20T22:07:30.607 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:30.607+0000 7fac4021b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T22:07:30.662 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:30.662+0000 7fac4021b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T22:07:30.667 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:30.666+0000 7f52cb22e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T22:07:30.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.685+0000 7f6ffae5a640 1 Processor -- start 2024-05-20T22:07:30.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.685+0000 7f6ffae5a640 1 -- start start 2024-05-20T22:07:30.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.686+0000 7f6ffae5a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6ff40cf5f0 0x7f6ff4162b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.686+0000 7f6ffae5a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff4163080 0x7f6ff4163490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.686+0000 7f6ffae5a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ff40cfa00 con 0x7f6ff406b500 2024-05-20T22:07:30.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.686+0000 7f6ffae5a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f6ff41639d0 con 0x7f6ff40cf5f0 2024-05-20T22:07:30.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.686+0000 7f6ffae5a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff40d31e0 con 0x7f6ff4163080 2024-05-20T22:07:30.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.686+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff4163080 0x7f6ff4163490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.686+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff4163080 0x7f6ff4163490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:51474/0 (socket says 172.21.15.138:51474) 2024-05-20T22:07:30.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.686+0000 7f6ff3fff640 1 -- 172.21.15.138:0/1964668228 learned_addr learned my addr 172.21.15.138:0/1964668228 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:30.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.687+0000 7f6ff3fff640 1 -- 172.21.15.138:0/1964668228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff4163080 msgr2=0x7f6ff4163490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-20T22:07:30.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.687+0000 7f6ff3fff640 1 -- 172.21.15.138:0/1964668228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff4163080 msgr2=0x7f6ff4163490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:30.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.687+0000 7f6ff3fff640 1 --2- 172.21.15.138:0/1964668228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff4163080 0x7f6ff4163490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:30.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.687+0000 7f6ff8bf6640 1 --2- 172.21.15.138:0/1964668228 >> v2:172.21.15.55:3300/0 conn(0x7f6ff40cf5f0 0x7f6ff4162b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.687+0000 7f6ff3fff640 1 --2- 172.21.15.138:0/1964668228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff4163080 0x7f6ff4163490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:30.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.687+0000 7f6ff8bf6640 1 -- 172.21.15.138:0/1964668228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff4163080 msgr2=0x7f6ff4163490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:30.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.687+0000 7f6ff8bf6640 1 --2- 172.21.15.138:0/1964668228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff4163080 0x7f6ff4163490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.687+0000 7f6ff8bf6640 1 -- 172.21.15.138:0/1964668228 >> v1:172.21.15.55:6789/0 conn(0x7f6ff406b500 legacy=0x7f6ff406b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.687+0000 7f6ff8bf6640 1 -- 172.21.15.138:0/1964668228 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff40d34c0 con 0x7f6ff40cf5f0 2024-05-20T22:07:30.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.688+0000 7f6ff8bf6640 1 --2- 172.21.15.138:0/1964668228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ff40cf5f0 0x7f6ff4162b40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80097a0 tx=0x7f6fe802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa64391f0a361bee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:30.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.688+0000 7f6ff37fe640 1 -- 172.21.15.138:0/1964668228 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8037070 con 0x7f6ff40cf5f0 2024-05-20T22:07:30.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.688+0000 7f6ff37fe640 1 -- 172.21.15.138:0/1964668228 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe802ae50 con 0x7f6ff40cf5f0 2024-05-20T22:07:30.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.688+0000 7f6ff37fe640 1 -- 172.21.15.138:0/1964668228 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8032660 con 0x7f6ff40cf5f0 2024-05-20T22:07:30.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.689+0000 7f6ffae5a640 1 -- 172.21.15.138:0/1964668228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ff40cf5f0 msgr2=0x7f6ff4162b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.689+0000 7f6ffae5a640 1 --2- 172.21.15.138:0/1964668228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ff40cf5f0 0x7f6ff4162b40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80097a0 tx=0x7f6fe802a100 comp rx=0 tx=0).stop 2024-05-20T22:07:30.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.689+0000 7f6ffae5a640 1 -- 172.21.15.138:0/1964668228 shutdown_connections 2024-05-20T22:07:30.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.689+0000 7f6ffae5a640 1 --2- 172.21.15.138:0/1964668228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff4163080 0x7f6ff4163490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.689+0000 7f6ffae5a640 1 --2- 172.21.15.138:0/1964668228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ff40cf5f0 0x7f6ff4162b40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.689+0000 7f6ffae5a640 1 -- 172.21.15.138:0/1964668228 >> 172.21.15.138:0/1964668228 conn(0x7f6ff40d5200 msgr2=0x7f6ff40b6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.689+0000 7f6ffae5a640 1 -- 172.21.15.138:0/1964668228 shutdown_connections 2024-05-20T22:07:30.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.689+0000 7f6ffae5a640 1 -- 172.21.15.138:0/1964668228 wait complete. 2024-05-20T22:07:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.690+0000 7f6ffae5a640 1 Processor -- start 2024-05-20T22:07:30.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.691+0000 7f6ffae5a640 1 -- start start 2024-05-20T22:07:30.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ffae5a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff406b500 0x7f6ff41f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ffae5a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ff40cf5f0 0x7f6ff415b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ffae5a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ff4163080 0x7f6ff4161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ffae5a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff40d3d60 con 0x7f6ff4163080 2024-05-20T22:07:30.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ffae5a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff40d3a80 con 0x7f6ff40cf5f0 2024-05-20T22:07:30.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ffae5a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff40d4280 con 0x7f6ff406b500 2024-05-20T22:07:30.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff8bf6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff406b500 0x7f6ff41f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff8bf6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff406b500 0x7f6ff41f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:51482/0 (socket says 172.21.15.138:51482) 2024-05-20T22:07:30.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff8bf6640 1 -- 172.21.15.138:0/2437496478 learned_addr learned my addr 172.21.15.138:0/2437496478 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:30.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff3fff640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ff40cf5f0 0x7f6ff415b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff93f7640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ff4163080 0x7f6ff4161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff8bf6640 1 -- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff406b500 msgr2=0x7f6ff41f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-20T22:07:30.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff8bf6640 1 -- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff406b500 msgr2=0x7f6ff41f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:30.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff8bf6640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff406b500 0x7f6ff41f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:30.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff8bf6640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff406b500 0x7f6ff41f61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:30.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff93f7640 1 -- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff406b500 msgr2=0x7f6ff41f61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:30.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff93f7640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff406b500 0x7f6ff41f61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff93f7640 1 -- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ff40cf5f0 msgr2=0x7f6ff415b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff93f7640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ff40cf5f0 0x7f6ff415b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.692+0000 7f6ff93f7640 1 -- 172.21.15.138:0/2437496478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff415a0a0 con 0x7f6ff4163080 2024-05-20T22:07:30.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.693+0000 7f6ff93f7640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ff4163080 0x7f6ff4161700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00c730 tx=0x7f6fec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:30.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.693+0000 7f6ff17fa640 1 -- 172.21.15.138:0/2437496478 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec003b70 con 0x7f6ff4163080 2024-05-20T22:07:30.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.693+0000 7f6ff17fa640 1 -- 172.21.15.138:0/2437496478 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fec003d10 con 0x7f6ff4163080 2024-05-20T22:07:30.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.693+0000 7f6ffae5a640 1 -- 172.21.15.138:0/2437496478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff415a5a0 con 0x7f6ff4163080 2024-05-20T22:07:30.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.693+0000 7f6ff17fa640 1 -- 172.21.15.138:0/2437496478 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec005310 con 0x7f6ff4163080 2024-05-20T22:07:30.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.693+0000 7f6ffae5a640 1 -- 172.21.15.138:0/2437496478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff415ab60 con 0x7f6ff4163080 2024-05-20T22:07:30.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.693+0000 7f6ff17fa640 1 -- 172.21.15.138:0/2437496478 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6fec0054b0 con 0x7f6ff4163080 2024-05-20T22:07:30.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.693+0000 7f6ff17fa640 1 -- 172.21.15.138:0/2437496478 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f6fec012a10 con 0x7f6ff4163080 2024-05-20T22:07:30.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.694+0000 7f6ffae5a640 1 -- 172.21.15.138:0/2437496478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff4069090 con 0x7f6ff4163080 2024-05-20T22:07:30.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.698+0000 7f6ff17fa640 1 -- 172.21.15.138:0/2437496478 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6fec012420 con 0x7f6ff4163080 2024-05-20T22:07:30.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.769+0000 7f6ffae5a640 1 -- 172.21.15.138:0/2437496478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6541c8c6-e672-44e2-9604-253abe988813", "id": 6} v 0) -- 0x7f6ff40d2e90 con 0x7f6ff4163080 2024-05-20T22:07:30.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.771+0000 7f6ff17fa640 1 -- 172.21.15.138:0/2437496478 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6541c8c6-e672-44e2-9604-253abe988813", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f6fec012600 con 0x7f6ff4163080 2024-05-20T22:07:30.773 INFO:teuthology.orchestra.run.smithi138.stdout:6 2024-05-20T22:07:30.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.773+0000 7f6ffae5a640 1 -- 172.21.15.138:0/2437496478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ff4163080 msgr2=0x7f6ff4161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.773+0000 7f6ffae5a640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ff4163080 0x7f6ff4161700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00c730 tx=0x7f6fec00cbf0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.773+0000 7f6ffae5a640 1 -- 172.21.15.138:0/2437496478 shutdown_connections 2024-05-20T22:07:30.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.773+0000 7f6ffae5a640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ff4163080 0x7f6ff4161700 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.773+0000 7f6ffae5a640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ff40cf5f0 0x7f6ff415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.773+0000 7f6ffae5a640 1 --2- 172.21.15.138:0/2437496478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ff406b500 0x7f6ff41f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.773+0000 7f6ffae5a640 1 -- 172.21.15.138:0/2437496478 >> 172.21.15.138:0/2437496478 conn(0x7f6ff40d5200 msgr2=0x7f6ff40ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:30.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.773+0000 7f6ffae5a640 1 -- 172.21.15.138:0/2437496478 shutdown_connections 2024-05-20T22:07:30.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.773+0000 7f6ffae5a640 1 -- 172.21.15.138:0/2437496478 wait complete. 2024-05-20T22:07:30.785 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 986dd45e-e14c-4f4a-9320-5e2adc4bd06e 7 2024-05-20T22:07:30.787 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:30.787+0000 7fac4021b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T22:07:30.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.896+0000 7f4276c21640 1 Processor -- start 2024-05-20T22:07:30.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.896+0000 7f4276c21640 1 -- start start 2024-05-20T22:07:30.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f4276c21640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 0x7f427009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f4276c21640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42700af840 con 0x7f427006bd90 2024-05-20T22:07:30.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f4276c21640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42700afb20 con 0x7f42700d08b0 2024-05-20T22:07:30.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f4276c21640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f42700afe00 con 0x7f427006e470 2024-05-20T22:07:30.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f42749bd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 0x7f427009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f426ffff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f42700d08b0 0x7f42700d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40340/0 (socket says 172.21.15.138:40340) 2024-05-20T22:07:30.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f426ffff640 1 -- 172.21.15.138:0/3353815110 learned_addr learned my addr 172.21.15.138:0/3353815110 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f42749bd640 1 -- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 msgr2=0x7f427009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f42749bd640 1 -- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 msgr2=0x7f427009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f42749bd640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 0x7f427009aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f42749bd640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 0x7f427009aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 920659786 0 0) 0x7f42700afb20 con 0x7f42700d08b0 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4260003520 con 0x7f42700d08b0 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3102370216 0 0) 0x7f42700af840 con 0x7f427006bd90 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.898+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42700afb20 con 0x7f427006bd90 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2593865663 0 0) 0x7f4260003520 con 0x7f42700d08b0 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 msgr2=0x7f427009aa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 0x7f427009aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 >> v1:172.21.15.55:6789/0 conn(0x7f427006bd90 legacy=0x7f427006c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42700b00e0 con 0x7f42700d08b0 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f425c001670 con 0x7f42700d08b0 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 >> v1:172.21.15.138:6789/0 conn(0x7f42700d08b0 legacy=0x7f42700d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 0x7f4260003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42600040a0 0x7f4260004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4260004d90 0x7f4260005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f42700afe00 con 0x7f42600040a0 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f426000bcf0 con 0x7f4260004d90 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f426000bef0 con 0x7f427006e470 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f42749bd640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 0x7f4260003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f42751be640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4260004d90 0x7f4260005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f426ffff640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42600040a0 0x7f4260004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f42749bd640 1 -- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4260004d90 msgr2=0x7f4260005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f42749bd640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4260004d90 0x7f4260005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f42749bd640 1 -- 172.21.15.138:0/3353815110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42600040a0 msgr2=0x7f4260004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f42749bd640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f42600040a0 0x7f4260004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.899+0000 7f42749bd640 1 -- 172.21.15.138:0/3353815110 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4260003520 con 0x7f427006e470 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.900+0000 7f42749bd640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 0x7f4260003b40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f42700730b0 tx=0x7f4260005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac25e41b26b07e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.900+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264030070 con 0x7f427006e470 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.900+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426402ad70 con 0x7f427006e470 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.900+0000 7f426f7fe640 1 -- 172.21.15.138:0/3353815110 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426400b5e0 con 0x7f427006e470 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.901+0000 7f4276c21640 1 -- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 msgr2=0x7f4260003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.901+0000 7f4276c21640 1 --2- 172.21.15.138:0/3353815110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f427006e470 0x7f4260003b40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f42700730b0 tx=0x7f4260005830 comp rx=0 tx=0).stop 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.901+0000 7f42751be640 1 -- 172.21.15.138:0/3353815110 reap_dead start 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.901+0000 7f4276c21640 1 -- 172.21.15.138:0/3353815110 shutdown_connections 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.901+0000 7f4276c21640 1 -- 172.21.15.138:0/3353815110 >> 172.21.15.138:0/3353815110 conn(0x7f42700b6500 msgr2=0x7f42700b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.901+0000 7f4276c21640 1 -- 172.21.15.138:0/3353815110 shutdown_connections 2024-05-20T22:07:30.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.901+0000 7f4276c21640 1 -- 172.21.15.138:0/3353815110 wait complete. 2024-05-20T22:07:30.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.903+0000 7f4276c21640 1 Processor -- start 2024-05-20T22:07:30.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.903+0000 7f4276c21640 1 -- start start 2024-05-20T22:07:30.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f4276c21640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f427006e470 0x7f42700cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f4276c21640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42700cc5a0 0x7f4270156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f4276c21640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42700cdf00 0x7f42700ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:30.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f4276c21640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f427009aa70 con 0x7f427006e470 2024-05-20T22:07:30.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f4276c21640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f427015f540 con 0x7f42700cdf00 2024-05-20T22:07:30.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f4276c21640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f427009abf0 con 0x7f42700cc5a0 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f426ffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42700cc5a0 0x7f4270156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f426ffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42700cc5a0 0x7f4270156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:51494/0 (socket says 172.21.15.138:51494) 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f426ffff640 1 -- 172.21.15.138:0/3188763142 learned_addr learned my addr 172.21.15.138:0/3188763142 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f42751be640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42700cdf00 0x7f42700ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f42749bd640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f427006e470 0x7f42700cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f426ffff640 1 -- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42700cc5a0 msgr2=0x7f4270156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f426ffff640 1 -- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42700cc5a0 msgr2=0x7f4270156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f426ffff640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42700cc5a0 0x7f4270156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.904+0000 7f426ffff640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42700cc5a0 0x7f4270156cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f42751be640 1 -- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42700cc5a0 msgr2=0x7f4270156cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f42751be640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42700cc5a0 0x7f4270156cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f42751be640 1 -- 172.21.15.138:0/3188763142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f427006e470 msgr2=0x7f42700cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f42751be640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f427006e470 0x7f42700cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f42751be640 1 -- 172.21.15.138:0/3188763142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4270157230 con 0x7f42700cdf00 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f42751be640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42700cdf00 0x7f42700ce310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f427006e3b0 tx=0x7f426800ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f426d7fa640 1 -- 172.21.15.138:0/3188763142 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268019070 con 0x7f42700cdf00 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f426d7fa640 1 -- 172.21.15.138:0/3188763142 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4268006e20 con 0x7f42700cdf00 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f426d7fa640 1 -- 172.21.15.138:0/3188763142 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268003bb0 con 0x7f42700cdf00 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f4276c21640 1 -- 172.21.15.138:0/3188763142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42700ce850 con 0x7f42700cdf00 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f4276c21640 1 -- 172.21.15.138:0/3188763142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42700c6280 con 0x7f42700cdf00 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f426d7fa640 1 -- 172.21.15.138:0/3188763142 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4268005030 con 0x7f42700cdf00 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.905+0000 7f426d7fa640 1 -- 172.21.15.138:0/3188763142 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f4268007690 con 0x7f42700cdf00 2024-05-20T22:07:30.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.906+0000 7f4276c21640 1 -- 172.21.15.138:0/3188763142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4270074710 con 0x7f42700cdf00 2024-05-20T22:07:30.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.908+0000 7f426d7fa640 1 -- 172.21.15.138:0/3188763142 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4268034dd0 con 0x7f42700cdf00 2024-05-20T22:07:30.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.982+0000 7f4276c21640 1 -- 172.21.15.138:0/3188763142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "986dd45e-e14c-4f4a-9320-5e2adc4bd06e", "id": 7} v 0) -- 0x7f42701f5270 con 0x7f42700cdf00 2024-05-20T22:07:30.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.985+0000 7f426d7fa640 1 -- 172.21.15.138:0/3188763142 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "986dd45e-e14c-4f4a-9320-5e2adc4bd06e", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f4268023030 con 0x7f42700cdf00 2024-05-20T22:07:30.986 INFO:teuthology.orchestra.run.smithi138.stdout:7 2024-05-20T22:07:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.986+0000 7f4276c21640 1 -- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42700cdf00 msgr2=0x7f42700ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.986+0000 7f4276c21640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42700cdf00 0x7f42700ce310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f427006e3b0 tx=0x7f426800ff70 comp rx=0 tx=0).stop 2024-05-20T22:07:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.986+0000 7f4276c21640 1 -- 172.21.15.138:0/3188763142 shutdown_connections 2024-05-20T22:07:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.986+0000 7f4276c21640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42700cdf00 0x7f42700ce310 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.986+0000 7f4276c21640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42700cc5a0 0x7f4270156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.986+0000 7f4276c21640 1 --2- 172.21.15.138:0/3188763142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f427006e470 0x7f42700cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:30.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.986+0000 7f4276c21640 1 -- 172.21.15.138:0/3188763142 >> 172.21.15.138:0/3188763142 conn(0x7f42700b6500 msgr2=0x7f427015e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:30.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.986+0000 7f4276c21640 1 -- 172.21.15.138:0/3188763142 shutdown_connections 2024-05-20T22:07:30.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:07:30.986+0000 7f4276c21640 1 -- 172.21.15.138:0/3188763142 wait complete. 2024-05-20T22:07:30.998 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-20T22:07:30.998 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-20T22:07:31.000 INFO:tasks.ceph.osd.0:Started 2024-05-20T22:07:31.000 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-20T22:07:31.001 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-20T22:07:31.002 INFO:tasks.ceph.osd.1:Started 2024-05-20T22:07:31.003 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-20T22:07:31.003 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-20T22:07:31.005 INFO:tasks.ceph.osd.2:Started 2024-05-20T22:07:31.005 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-20T22:07:31.005 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-20T22:07:31.007 INFO:tasks.ceph.osd.3:Started 2024-05-20T22:07:31.007 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-20T22:07:31.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-20T22:07:31.010 INFO:tasks.ceph.osd.4:Started 2024-05-20T22:07:31.010 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-20T22:07:31.010 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-20T22:07:31.013 INFO:tasks.ceph.osd.5:Started 2024-05-20T22:07:31.013 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-20T22:07:31.013 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-20T22:07:31.016 INFO:tasks.ceph.osd.6:Started 2024-05-20T22:07:31.016 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-20T22:07:31.016 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-20T22:07:31.018 INFO:tasks.ceph.osd.7:Started 2024-05-20T22:07:31.019 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:07:31.104 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:31.104+0000 7f52cb22e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T22:07:31.138 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:31.138+0000 7fac4021b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T22:07:31.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.160+0000 7fa4feca2640 1 Processor -- start 2024-05-20T22:07:31.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.160+0000 7fa4feca2640 1 -- start start 2024-05-20T22:07:31.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.161+0000 7fa4feca2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa4f80cd840 0x7fa4f80b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:31.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.161+0000 7fa4feca2640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4f80d2e10 0x7fa4f80d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:31.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4feca2640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f815da90 con 0x7fa4f80b5830 2024-05-20T22:07:31.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4fd23f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4f80d2e10 0x7fa4f80d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:31.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4f7fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa4f80b5830 0x7fa4f80d2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59486/0 (socket says 172.21.15.55:59486) 2024-05-20T22:07:31.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4f7fff640 1 -- 172.21.15.55:0/645487070 learned_addr learned my addr 172.21.15.55:0/645487070 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:31.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4feca2640 1 -- 172.21.15.55:0/645487070 --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f80b6990 con 0x7fa4f80d2e10 2024-05-20T22:07:31.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4feca2640 1 -- 172.21.15.55:0/645487070 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f80b5d80 con 0x7fa4f80cd840 2024-05-20T22:07:31.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4fca3e640 1 --2- 172.21.15.55:0/645487070 >> v2:172.21.15.138:3300/0 conn(0x7fa4f80cd840 0x7fa4f80b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:31.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4f77fe640 1 -- 172.21.15.55:0/645487070 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 683430484 0 0) 0x7fa4f815da90 con 0x7fa4f80b5830 2024-05-20T22:07:31.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4f77fe640 1 -- 172.21.15.55:0/645487070 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4e8003520 con 0x7fa4f80b5830 2024-05-20T22:07:31.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4fca3e640 1 -- 172.21.15.55:0/645487070 >> v2:172.21.15.55:3300/0 conn(0x7fa4f80d2e10 msgr2=0x7fa4f80d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:31.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4fca3e640 1 --2- 172.21.15.55:0/645487070 >> v2:172.21.15.55:3300/0 conn(0x7fa4f80d2e10 0x7fa4f80d1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:31.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4fca3e640 1 -- 172.21.15.55:0/645487070 >> v1:172.21.15.55:6789/0 conn(0x7fa4f80b5830 legacy=0x7fa4f80d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:31.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.162+0000 7fa4fca3e640 1 -- 172.21.15.55:0/645487070 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f80b6060 con 0x7fa4f80cd840 2024-05-20T22:07:31.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.163+0000 7fa4fca3e640 1 --2- 172.21.15.55:0/645487070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80cd840 0x7fa4f80b1fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80d3ae0 tx=0x7fa4ec02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46c5e943bbbe98 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:31.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.163+0000 7fa4f77fe640 1 -- 172.21.15.55:0/645487070 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec037070 con 0x7fa4f80cd840 2024-05-20T22:07:31.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.163+0000 7fa4f77fe640 1 -- 172.21.15.55:0/645487070 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec032440 con 0x7fa4f80cd840 2024-05-20T22:07:31.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.163+0000 7fa4f77fe640 1 -- 172.21.15.55:0/645487070 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec02ae60 con 0x7fa4f80cd840 2024-05-20T22:07:31.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.164+0000 7fa4feca2640 1 -- 172.21.15.55:0/645487070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80cd840 msgr2=0x7fa4f80b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:31.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.164+0000 7fa4feca2640 1 --2- 172.21.15.55:0/645487070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80cd840 0x7fa4f80b1fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80d3ae0 tx=0x7fa4ec02a110 comp rx=0 tx=0).stop 2024-05-20T22:07:31.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.164+0000 7fa4feca2640 1 -- 172.21.15.55:0/645487070 shutdown_connections 2024-05-20T22:07:31.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.164+0000 7fa4feca2640 1 --2- 172.21.15.55:0/645487070 >> v2:172.21.15.55:3300/0 conn(0x7fa4f80d2e10 0x7fa4f80d1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:31.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.164+0000 7fa4feca2640 1 --2- 172.21.15.55:0/645487070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80cd840 0x7fa4f80b1fa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:31.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.164+0000 7fa4feca2640 1 -- 172.21.15.55:0/645487070 >> 172.21.15.55:0/645487070 conn(0x7fa4f808bff0 msgr2=0x7fa4f808c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:31.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.164+0000 7fa4feca2640 1 -- 172.21.15.55:0/645487070 shutdown_connections 2024-05-20T22:07:31.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.164+0000 7fa4feca2640 1 -- 172.21.15.55:0/645487070 wait complete. 2024-05-20T22:07:31.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.166+0000 7fa4feca2640 1 Processor -- start 2024-05-20T22:07:31.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.166+0000 7fa4feca2640 1 -- start start 2024-05-20T22:07:31.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4feca2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80b5830 0x7fa4f8160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4feca2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4f80cd840 0x7fa4f80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4feca2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4f80d2e10 0x7fa4f80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4feca2640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f80b8160 con 0x7fa4f80d2e10 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4feca2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f80b6620 con 0x7fa4f80b5830 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4feca2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f80b6e20 con 0x7fa4f80cd840 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4fca3e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80b5830 0x7fa4f8160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4f7fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4f80cd840 0x7fa4f80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4fca3e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80b5830 0x7fa4f8160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:47016/0 (socket says 172.21.15.55:47016) 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4fca3e640 1 -- 172.21.15.55:0/1592365835 learned_addr learned my addr 172.21.15.55:0/1592365835 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4fd23f640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4f80d2e10 0x7fa4f80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4f7fff640 1 -- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4f80cd840 msgr2=0x7fa4f80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4f7fff640 1 -- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4f80cd840 msgr2=0x7fa4f80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4f7fff640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4f80cd840 0x7fa4f80ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.168+0000 7fa4f7fff640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4f80cd840 0x7fa4f80ce940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4fca3e640 1 -- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4f80cd840 msgr2=0x7fa4f80ce940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4fca3e640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4f80cd840 0x7fa4f80ce940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4fca3e640 1 -- 172.21.15.55:0/1592365835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4f80d2e10 msgr2=0x7fa4f80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4fca3e640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4f80d2e10 0x7fa4f80cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4fca3e640 1 -- 172.21.15.55:0/1592365835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f80cf700 con 0x7fa4f80b5830 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4fd23f640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4f80d2e10 0x7fa4f80cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4fca3e640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80b5830 0x7fa4f8160440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec035040 tx=0x7fa4ec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4f57fa640 1 -- 172.21.15.55:0/1592365835 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec037040 con 0x7fa4f80b5830 2024-05-20T22:07:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4f57fa640 1 -- 172.21.15.55:0/1592365835 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec042070 con 0x7fa4f80b5830 2024-05-20T22:07:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4f57fa640 1 -- 172.21.15.55:0/1592365835 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec006db0 con 0x7fa4f80b5830 2024-05-20T22:07:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4feca2640 1 -- 172.21.15.55:0/1592365835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f815dc80 con 0x7fa4f80b5830 2024-05-20T22:07:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4feca2640 1 -- 172.21.15.55:0/1592365835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f815e240 con 0x7fa4f80b5830 2024-05-20T22:07:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4f57fa640 1 -- 172.21.15.55:0/1592365835 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa4ec030bb0 con 0x7fa4f80b5830 2024-05-20T22:07:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.169+0000 7fa4f57fa640 1 -- 172.21.15.55:0/1592365835 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fa4ec03d440 con 0x7fa4f80b5830 2024-05-20T22:07:31.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.170+0000 7fa4feca2640 1 -- 172.21.15.55:0/1592365835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f8069090 con 0x7fa4f80b5830 2024-05-20T22:07:31.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.173+0000 7fa4f57fa640 1 -- 172.21.15.55:0/1592365835 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa4ec047460 con 0x7fa4f80b5830 2024-05-20T22:07:31.190 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:31.190+0000 7fac4021b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T22:07:31.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.235+0000 7fa4feca2640 1 -- 172.21.15.55:0/1592365835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4f80b16c0 con 0x7fa4f80b5830 2024-05-20T22:07:31.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.236+0000 7fa4f57fa640 1 -- 172.21.15.55:0/1592365835 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fa4ec047640 con 0x7fa4f80b5830 2024-05-20T22:07:31.236 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:07:31.237 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":10,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:30.984077+0000","last_up_change":"0.000000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:07:31.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.236+0000 7fa4feca2640 1 -- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80b5830 msgr2=0x7fa4f8160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:31.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.236+0000 7fa4feca2640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80b5830 0x7fa4f8160440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec035040 tx=0x7fa4ec0046c0 comp rx=0 tx=0).stop 2024-05-20T22:07:31.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.237+0000 7fa4feca2640 1 -- 172.21.15.55:0/1592365835 shutdown_connections 2024-05-20T22:07:31.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.237+0000 7fa4feca2640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4f80d2e10 0x7fa4f80cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:31.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.237+0000 7fa4feca2640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4f80cd840 0x7fa4f80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:31.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.237+0000 7fa4feca2640 1 --2- 172.21.15.55:0/1592365835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4f80b5830 0x7fa4f8160440 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:31.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.237+0000 7fa4feca2640 1 -- 172.21.15.55:0/1592365835 >> 172.21.15.55:0/1592365835 conn(0x7fa4f808bff0 msgr2=0x7fa4f80b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:31.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.237+0000 7fa4feca2640 1 -- 172.21.15.55:0/1592365835 shutdown_connections 2024-05-20T22:07:31.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:31.237+0000 7fa4feca2640 1 -- 172.21.15.55:0/1592365835 wait complete. 2024-05-20T22:07:31.249 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-20T22:07:31.249 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-20T22:07:31.357 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:31.357+0000 7fac4021b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T22:07:31.369 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:31.368+0000 7f52cb22e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T22:07:31.422 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:31.422+0000 7f52cb22e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T22:07:31.502 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:31.501+0000 7fac4021b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T22:07:31.538 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:31.538+0000 7f52cb22e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T22:07:31.554 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:31.553+0000 7fac4021b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T22:07:31.628 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:31.627+0000 7fac4021b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T22:07:31.628 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-05-20T22:07:31.627+0000 7fe7cbcdd640 -1 Falling back to public interface 2024-05-20T22:07:31.630 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-05-20T22:07:31.629+0000 7f2bedf33640 -1 Falling back to public interface 2024-05-20T22:07:31.636 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-05-20T22:07:31.635+0000 7f028b22c640 -1 Falling back to public interface 2024-05-20T22:07:31.640 INFO:tasks.ceph.osd.2.smithi055.stderr:2024-05-20T22:07:31.639+0000 7f32cd7cb640 -1 Falling back to public interface 2024-05-20T22:07:31.643 INFO:tasks.ceph.osd.3.smithi055.stderr:2024-05-20T22:07:31.643+0000 7f82d4e18640 -1 Falling back to public interface 2024-05-20T22:07:31.647 INFO:tasks.ceph.osd.1.smithi055.stderr:2024-05-20T22:07:31.647+0000 7fcdb505f640 -1 Falling back to public interface 2024-05-20T22:07:31.653 INFO:tasks.ceph.osd.0.smithi055.stderr:2024-05-20T22:07:31.652+0000 7f6e0960f640 -1 Falling back to public interface 2024-05-20T22:07:31.657 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-05-20T22:07:31.657+0000 7f8d25c14640 -1 Falling back to public interface 2024-05-20T22:07:31.682 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:31.682+0000 7fac4021b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T22:07:31.838 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:31.837+0000 7f52cb22e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T22:07:31.885 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:31.884+0000 7f52cb22e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T22:07:32.044 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:32.043+0000 7f52cb22e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T22:07:32.055 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-20T22:07:32.161 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.160+0000 7f03eb4e9640 1 Processor -- start 2024-05-20T22:07:32.161 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.161+0000 7f03eb4e9640 1 -- start start 2024-05-20T22:07:32.164 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03eb4e9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f03e41628c0 0x7f03e409a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:32.164 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03eb4e9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e4162de0 0x7f03e41631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:32.164 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03eb4e9640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f03e40cdd40 0x7f03e40ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:32.164 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03eb4e9640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f03e409a8c0 con 0x7f03e40cdd40 2024-05-20T22:07:32.164 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03eb4e9640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f03e4163730 con 0x7f03e41628c0 2024-05-20T22:07:32.164 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03eb4e9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03e40d13a0 con 0x7f03e4162de0 2024-05-20T22:07:32.164 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e9a86640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f03e40cdd40 0x7f03e40ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e9a86640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f03e40cdd40 0x7f03e40ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43002/0 (socket says 172.21.15.55:43002) 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e9a86640 1 -- 172.21.15.55:0/2365966258 learned_addr learned my addr 172.21.15.55:0/2365966258 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e8a84640 1 --2- 172.21.15.55:0/2365966258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e4162de0 0x7f03e41631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e9285640 1 --2- 172.21.15.55:0/2365966258 >> v2:172.21.15.138:3300/0 conn(0x7f03e41628c0 0x7f03e409a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e8a84640 1 -- 172.21.15.55:0/2365966258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e4162de0 msgr2=0x7f03e41631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e8a84640 1 -- 172.21.15.55:0/2365966258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e4162de0 msgr2=0x7f03e41631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e8a84640 1 --2- 172.21.15.55:0/2365966258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e4162de0 0x7f03e41631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e8a84640 1 --2- 172.21.15.55:0/2365966258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e4162de0 0x7f03e41631f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e9a86640 1 -- 172.21.15.55:0/2365966258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e4162de0 msgr2=0x7f03e41631f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e9a86640 1 --2- 172.21.15.55:0/2365966258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e4162de0 0x7f03e41631f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e9a86640 1 -- 172.21.15.55:0/2365966258 >> v2:172.21.15.138:3300/0 conn(0x7f03e41628c0 msgr2=0x7f03e409a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e9a86640 1 --2- 172.21.15.55:0/2365966258 >> v2:172.21.15.138:3300/0 conn(0x7f03e41628c0 0x7f03e409a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.162+0000 7f03e9a86640 1 -- 172.21.15.55:0/2365966258 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e40d1680 con 0x7f03e40cdd40 2024-05-20T22:07:32.166 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.163+0000 7f03e9a86640 1 --2- 172.21.15.55:0/2365966258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e40cdd40 0x7f03e40ce150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f03e406f520 tx=0x7f03dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16dd124fbc82d7a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:32.166 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.163+0000 7f03dbfff640 1 -- 172.21.15.55:0/2365966258 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc036070 con 0x7f03e40cdd40 2024-05-20T22:07:32.166 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.163+0000 7f03dbfff640 1 -- 172.21.15.55:0/2365966258 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03dc031600 con 0x7f03e40cdd40 2024-05-20T22:07:32.166 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.163+0000 7f03dbfff640 1 -- 172.21.15.55:0/2365966258 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0319d0 con 0x7f03e40cdd40 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.164+0000 7f03eb4e9640 1 -- 172.21.15.55:0/2365966258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e40cdd40 msgr2=0x7f03e40ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.164+0000 7f03eb4e9640 1 --2- 172.21.15.55:0/2365966258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e40cdd40 0x7f03e40ce150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f03e406f520 tx=0x7f03dc02a6a0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.164+0000 7f03eb4e9640 1 -- 172.21.15.55:0/2365966258 shutdown_connections 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.164+0000 7f03eb4e9640 1 --2- 172.21.15.55:0/2365966258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e40cdd40 0x7f03e40ce150 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.164+0000 7f03eb4e9640 1 --2- 172.21.15.55:0/2365966258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e4162de0 0x7f03e41631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.164+0000 7f03eb4e9640 1 --2- 172.21.15.55:0/2365966258 >> v2:172.21.15.138:3300/0 conn(0x7f03e41628c0 0x7f03e409a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.164+0000 7f03eb4e9640 1 -- 172.21.15.55:0/2365966258 >> 172.21.15.55:0/2365966258 conn(0x7f03e40b64e0 msgr2=0x7f03e40b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.164+0000 7f03eb4e9640 1 -- 172.21.15.55:0/2365966258 shutdown_connections 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.164+0000 7f03eb4e9640 1 -- 172.21.15.55:0/2365966258 wait complete. 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.166+0000 7f03eb4e9640 1 Processor -- start 2024-05-20T22:07:32.167 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.166+0000 7f03eb4e9640 1 -- start start 2024-05-20T22:07:32.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.167+0000 7f03eb4e9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e40cdd40 0x7f03e415ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:32.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.167+0000 7f03eb4e9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e41628c0 0x7f03e40ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:32.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03eb4e9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4162de0 0x7f03e40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:32.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03eb4e9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f03e40d1f20 con 0x7f03e4162de0 2024-05-20T22:07:32.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03eb4e9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03e40d1c40 con 0x7f03e40cdd40 2024-05-20T22:07:32.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03eb4e9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03e40ceea0 con 0x7f03e41628c0 2024-05-20T22:07:32.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e9a86640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4162de0 0x7f03e40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:32.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e9a86640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4162de0 0x7f03e40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43012/0 (socket says 172.21.15.55:43012) 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e9a86640 1 -- 172.21.15.55:0/359610025 learned_addr learned my addr 172.21.15.55:0/359610025 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e9285640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e40cdd40 0x7f03e415ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e8a84640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e41628c0 0x7f03e40ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e8a84640 1 -- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e41628c0 msgr2=0x7f03e40ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e8a84640 1 -- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e41628c0 msgr2=0x7f03e40ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e8a84640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e41628c0 0x7f03e40ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e8a84640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e41628c0 0x7f03e40ca930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e9a86640 1 -- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e41628c0 msgr2=0x7f03e40ca930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e9a86640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e41628c0 0x7f03e40ca930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e9a86640 1 -- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e40cdd40 msgr2=0x7f03e415ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e9a86640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e40cdd40 0x7f03e415ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e9a86640 1 -- 172.21.15.55:0/359610025 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e414a870 con 0x7f03e4162de0 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03e9a86640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4162de0 0x7f03e40cae70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009f40 tx=0x7f03dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:32.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03d9ffb640 1 -- 172.21.15.55:0/359610025 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc03f020 con 0x7f03e4162de0 2024-05-20T22:07:32.172 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03d9ffb640 1 -- 172.21.15.55:0/359610025 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03dc037040 con 0x7f03e4162de0 2024-05-20T22:07:32.172 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03eb4e9640 1 -- 172.21.15.55:0/359610025 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e415d5c0 con 0x7f03e4162de0 2024-05-20T22:07:32.172 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.168+0000 7f03d9ffb640 1 -- 172.21.15.55:0/359610025 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc036070 con 0x7f03e4162de0 2024-05-20T22:07:32.172 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.169+0000 7f03eb4e9640 1 -- 172.21.15.55:0/359610025 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e415db80 con 0x7f03e4162de0 2024-05-20T22:07:32.172 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.169+0000 7f03d9ffb640 1 -- 172.21.15.55:0/359610025 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f03dc042020 con 0x7f03e4162de0 2024-05-20T22:07:32.172 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.169+0000 7f03d9ffb640 1 -- 172.21.15.55:0/359610025 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f03dc044cb0 con 0x7f03e4162de0 2024-05-20T22:07:32.172 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.170+0000 7f03eb4e9640 1 -- 172.21.15.55:0/359610025 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e409a380 con 0x7f03e4162de0 2024-05-20T22:07:32.173 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.172+0000 7f03d9ffb640 1 -- 172.21.15.55:0/359610025 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f03e409a380 con 0x7f03e4162de0 2024-05-20T22:07:32.180 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:32.179+0000 7f52cb22e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T22:07:32.193 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:32.192+0000 7fac4021b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T22:07:32.232 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:32.231+0000 7f52cb22e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T22:07:32.235 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.235+0000 7f03eb4e9640 1 -- 172.21.15.55:0/359610025 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03e409af90 con 0x7f03e4162de0 2024-05-20T22:07:32.236 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.236+0000 7f03d9ffb640 1 -- 172.21.15.55:0/359610025 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f03e409af90 con 0x7f03e4162de0 2024-05-20T22:07:32.237 INFO:teuthology.misc.health.smithi055.stdout: 2024-05-20T22:07:32.237 INFO:teuthology.misc.health.smithi055.stdout:{"epoch":10,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:30.984077+0000","last_up_change":"0.000000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:07:32.238 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.236+0000 7f03eb4e9640 1 -- 172.21.15.55:0/359610025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4162de0 msgr2=0x7f03e40cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:32.238 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.236+0000 7f03eb4e9640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4162de0 0x7f03e40cae70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009f40 tx=0x7f03dc004a40 comp rx=0 tx=0).stop 2024-05-20T22:07:32.238 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.236+0000 7f03eb4e9640 1 -- 172.21.15.55:0/359610025 shutdown_connections 2024-05-20T22:07:32.238 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.236+0000 7f03eb4e9640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4162de0 0x7f03e40cae70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.238 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.237+0000 7f03eb4e9640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03e41628c0 0x7f03e40ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.238 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.237+0000 7f03eb4e9640 1 --2- 172.21.15.55:0/359610025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e40cdd40 0x7f03e415ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:32.238 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.237+0000 7f03eb4e9640 1 -- 172.21.15.55:0/359610025 >> 172.21.15.55:0/359610025 conn(0x7f03e40b64e0 msgr2=0x7f03e406cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:32.238 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.237+0000 7f03eb4e9640 1 -- 172.21.15.55:0/359610025 shutdown_connections 2024-05-20T22:07:32.238 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:32.237+0000 7f03eb4e9640 1 -- 172.21.15.55:0/359610025 wait complete. 2024-05-20T22:07:32.247 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-20T22:07:32.301 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:32.300+0000 7f52cb22e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T22:07:32.316 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:32.316+0000 7fac4021b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T22:07:32.351 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:32.351+0000 7f52cb22e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T22:07:32.832 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:32.831+0000 7f52cb22e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T22:07:32.949 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-05-20T22:07:32.948+0000 7f52cb22e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T22:07:33.334 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:33.332+0000 7fabe0946640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-20T22:07:33.334 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-20T22:07:33.333+0000 7fabe0946640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-20T22:07:35.085 INFO:tasks.ceph.osd.2.smithi055.stderr:2024-05-20T22:07:35.085+0000 7f32cd7cb640 -1 osd.2 0 log_to_monitors true 2024-05-20T22:07:35.099 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-05-20T22:07:35.098+0000 7fe7cbcdd640 -1 osd.5 0 log_to_monitors true 2024-05-20T22:07:35.133 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-05-20T22:07:35.132+0000 7f2bedf33640 -1 osd.4 0 log_to_monitors true 2024-05-20T22:07:35.168 INFO:tasks.ceph.osd.3.smithi055.stderr:2024-05-20T22:07:35.167+0000 7f82d4e18640 -1 osd.3 0 log_to_monitors true 2024-05-20T22:07:35.173 INFO:tasks.ceph.osd.0.smithi055.stderr:2024-05-20T22:07:35.172+0000 7f6e0960f640 -1 osd.0 0 log_to_monitors true 2024-05-20T22:07:35.178 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-05-20T22:07:35.177+0000 7f028b22c640 -1 osd.7 0 log_to_monitors true 2024-05-20T22:07:35.179 INFO:tasks.ceph.osd.1.smithi055.stderr:2024-05-20T22:07:35.179+0000 7fcdb505f640 -1 osd.1 0 log_to_monitors true 2024-05-20T22:07:35.183 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-05-20T22:07:35.183+0000 7f8d25c14640 -1 osd.6 0 log_to_monitors true 2024-05-20T22:07:38.981 INFO:tasks.ceph.osd.0.smithi055.stderr:2024-05-20T22:07:38.980+0000 7f6e04fff640 -1 osd.0 0 waiting for initial osdmap 2024-05-20T22:07:38.988 INFO:tasks.ceph.osd.2.smithi055.stderr:2024-05-20T22:07:38.987+0000 7f32c9761640 -1 osd.2 0 waiting for initial osdmap 2024-05-20T22:07:39.000 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-05-20T22:07:38.999+0000 7f02871c2640 -1 osd.7 0 waiting for initial osdmap 2024-05-20T22:07:39.002 INFO:tasks.ceph.osd.0.smithi055.stderr:2024-05-20T22:07:39.002+0000 7f6dfcdfa640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T22:07:39.003 INFO:tasks.ceph.osd.2.smithi055.stderr:2024-05-20T22:07:39.002+0000 7f32c155c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T22:07:39.004 INFO:tasks.ceph.osd.1.smithi055.stderr:2024-05-20T22:07:39.003+0000 7fcdb0ff5640 -1 osd.1 0 waiting for initial osdmap 2024-05-20T22:07:39.013 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-05-20T22:07:39.012+0000 7fe7c7c73640 -1 osd.5 0 waiting for initial osdmap 2024-05-20T22:07:39.016 INFO:tasks.ceph.osd.1.smithi055.stderr:2024-05-20T22:07:39.016+0000 7fcda8df0640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T22:07:39.017 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-05-20T22:07:39.017+0000 7f027efbd640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T22:07:39.027 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-05-20T22:07:39.027+0000 7fe7bfa6e640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T22:07:39.030 INFO:tasks.ceph.osd.3.smithi055.stderr:2024-05-20T22:07:39.029+0000 7f82d082a640 -1 osd.3 0 waiting for initial osdmap 2024-05-20T22:07:39.037 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-05-20T22:07:39.036+0000 7f2be9ec9640 -1 osd.4 0 waiting for initial osdmap 2024-05-20T22:07:39.040 INFO:tasks.ceph.osd.3.smithi055.stderr:2024-05-20T22:07:39.040+0000 7f82c8625640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T22:07:39.045 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-05-20T22:07:39.045+0000 7f8d21659640 -1 osd.6 0 waiting for initial osdmap 2024-05-20T22:07:39.053 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-05-20T22:07:39.053+0000 7f2be1cc4640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T22:07:39.056 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-20T22:07:39.065 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-05-20T22:07:39.065+0000 7f8d19454640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T22:07:39.160 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.160+0000 7f307f091640 1 Processor -- start 2024-05-20T22:07:39.161 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.160+0000 7f307f091640 1 -- start start 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.161+0000 7f307f091640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f307815f0e0 0x7f30780cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.161+0000 7f307f091640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307815f4f0 con 0x7f307806bbc0 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.161+0000 7f307f091640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30780b3e70 con 0x7f30780b3640 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.161+0000 7f307f091640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f30780d44f0 con 0x7f307815f0e0 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.161+0000 7f307d62e640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f307806bbc0 0x7f307806bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43962/0 (socket says 172.21.15.55:43962) 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.161+0000 7f307d62e640 1 -- 172.21.15.55:0/1484368418 learned_addr learned my addr 172.21.15.55:0/1484368418 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.161+0000 7f3077fff640 1 --2- 172.21.15.55:0/1484368418 >> v2:172.21.15.138:3300/0 conn(0x7f307815f0e0 0x7f30780cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.162+0000 7f30777fe640 1 -- 172.21.15.55:0/1484368418 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1144517938 0 0) 0x7f3070001110 con 0x7f307806bbc0 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.162+0000 7f30777fe640 1 -- 172.21.15.55:0/1484368418 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3064003520 con 0x7f307806bbc0 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.162+0000 7f3077fff640 1 -- 172.21.15.55:0/1484368418 >> v1:172.21.15.138:6789/0 conn(0x7f30780b3640 legacy=0x7f30780cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.162+0000 7f30777fe640 1 -- 172.21.15.55:0/1484368418 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1207285454 0 0) 0x7f3064003520 con 0x7f307806bbc0 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.162+0000 7f3077fff640 1 -- 172.21.15.55:0/1484368418 >> v1:172.21.15.55:6789/0 conn(0x7f307806bbc0 legacy=0x7f307806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:39.163 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.162+0000 7f3077fff640 1 -- 172.21.15.55:0/1484368418 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30780b3b90 con 0x7f307815f0e0 2024-05-20T22:07:39.164 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.163+0000 7f3077fff640 1 --2- 172.21.15.55:0/1484368418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f307815f0e0 0x7f30780cfef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3068009d90 tx=0x7f306802a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c998b6a16c40ac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:39.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.163+0000 7f30777fe640 1 -- 172.21.15.55:0/1484368418 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068035020 con 0x7f307815f0e0 2024-05-20T22:07:39.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.163+0000 7f30777fe640 1 -- 172.21.15.55:0/1484368418 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3068007540 con 0x7f307815f0e0 2024-05-20T22:07:39.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.163+0000 7f30777fe640 1 -- 172.21.15.55:0/1484368418 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070001840 con 0x7f307815f0e0 2024-05-20T22:07:39.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.163+0000 7f307f091640 1 -- 172.21.15.55:0/1484368418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f307815f0e0 msgr2=0x7f30780cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:39.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.163+0000 7f307f091640 1 --2- 172.21.15.55:0/1484368418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f307815f0e0 0x7f30780cfef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3068009d90 tx=0x7f306802a5b0 comp rx=0 tx=0).stop 2024-05-20T22:07:39.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.163+0000 7f307f091640 1 -- 172.21.15.55:0/1484368418 shutdown_connections 2024-05-20T22:07:39.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.163+0000 7f307f091640 1 --2- 172.21.15.55:0/1484368418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f307815f0e0 0x7f30780cfef0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:39.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.163+0000 7f307f091640 1 -- 172.21.15.55:0/1484368418 >> 172.21.15.55:0/1484368418 conn(0x7f30780d4aa0 msgr2=0x7f30780d4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:39.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.164+0000 7f307f091640 1 -- 172.21.15.55:0/1484368418 shutdown_connections 2024-05-20T22:07:39.165 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.164+0000 7f307f091640 1 -- 172.21.15.55:0/1484368418 wait complete. 2024-05-20T22:07:39.166 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.165+0000 7f307f091640 1 Processor -- start 2024-05-20T22:07:39.166 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.165+0000 7f307f091640 1 -- start start 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307f091640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f307806bbc0 0x7f3078151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307f091640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30780b3640 0x7f3078156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307f091640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f307815f0e0 0x7f3078152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307f091640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30780d4210 con 0x7f307815f0e0 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307f091640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f307806b730 con 0x7f307806bbc0 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307f091640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f30780d0430 con 0x7f30780b3640 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307d62e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f307815f0e0 0x7f3078152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307ce2d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f307806bbc0 0x7f3078151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307d62e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f307815f0e0 0x7f3078152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40684/0 (socket says 172.21.15.55:40684) 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307d62e640 1 -- 172.21.15.55:0/1014054231 learned_addr learned my addr 172.21.15.55:0/1014054231 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f3077fff640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30780b3640 0x7f3078156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:39.169 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307d62e640 1 -- 172.21.15.55:0/1014054231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30780b3640 msgr2=0x7f3078156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307d62e640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30780b3640 0x7f3078156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307d62e640 1 -- 172.21.15.55:0/1014054231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f307806bbc0 msgr2=0x7f3078151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307d62e640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f307806bbc0 0x7f3078151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307d62e640 1 -- 172.21.15.55:0/1014054231 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3078151540 con 0x7f307815f0e0 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307ce2d640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f307806bbc0 0x7f3078151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.167+0000 7f307d62e640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f307815f0e0 0x7f3078152920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f307000c4d0 tx=0x7f307000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.168+0000 7f30757fa640 1 -- 172.21.15.55:0/1014054231 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070018070 con 0x7f307815f0e0 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.168+0000 7f30757fa640 1 -- 172.21.15.55:0/1014054231 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30700074e0 con 0x7f307815f0e0 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.168+0000 7f307f091640 1 -- 172.21.15.55:0/1014054231 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3078161410 con 0x7f307815f0e0 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.168+0000 7f30757fa640 1 -- 172.21.15.55:0/1014054231 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070004a00 con 0x7f307815f0e0 2024-05-20T22:07:39.170 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.168+0000 7f307f091640 1 -- 172.21.15.55:0/1014054231 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30781619d0 con 0x7f307815f0e0 2024-05-20T22:07:39.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.169+0000 7f307f091640 1 -- 172.21.15.55:0/1014054231 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30780759e0 con 0x7f307815f0e0 2024-05-20T22:07:39.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.169+0000 7f30757fa640 1 -- 172.21.15.55:0/1014054231 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f3070004ba0 con 0x7f307815f0e0 2024-05-20T22:07:39.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.169+0000 7f30757fa640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f305806bda0 0x7f305806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:39.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.170+0000 7f30757fa640 1 -- 172.21.15.55:0/1014054231 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f3070089300 con 0x7f307815f0e0 2024-05-20T22:07:39.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.170+0000 7f307ce2d640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f305806bda0 0x7f305806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:39.171 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.170+0000 7f307ce2d640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f305806bda0 0x7f305806e2b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3060000f60 tx=0x7f3060009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:39.175 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.175+0000 7f30757fa640 1 -- 172.21.15.55:0/1014054231 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f307815f4f0 con 0x7f307815f0e0 2024-05-20T22:07:39.360 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.359+0000 7f307f091640 1 -- 172.21.15.55:0/1014054231 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30780d2f10 con 0x7f307815f0e0 2024-05-20T22:07:39.361 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.360+0000 7f30757fa640 1 -- 172.21.15.55:0/1014054231 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f30700583c0 con 0x7f307815f0e0 2024-05-20T22:07:39.361 INFO:teuthology.misc.health.smithi055.stdout: 2024-05-20T22:07:39.361 INFO:teuthology.misc.health.smithi055.stdout:{"epoch":12,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:36.843138+0000","last_up_change":"0.000000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:07:39.363 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 -- 172.21.15.55:0/1014054231 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f305806bda0 msgr2=0x7f305806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:39.363 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f305806bda0 0x7f305806e2b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3060000f60 tx=0x7f3060009040 comp rx=0 tx=0).stop 2024-05-20T22:07:39.364 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 -- 172.21.15.55:0/1014054231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f307815f0e0 msgr2=0x7f3078152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:39.364 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f307815f0e0 0x7f3078152920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f307000c4d0 tx=0x7f307000c990 comp rx=0 tx=0).stop 2024-05-20T22:07:39.364 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 -- 172.21.15.55:0/1014054231 shutdown_connections 2024-05-20T22:07:39.364 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f305806bda0 0x7f305806e2b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:39.364 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f307815f0e0 0x7f3078152920 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:39.364 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30780b3640 0x7f3078156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:39.364 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 --2- 172.21.15.55:0/1014054231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f307806bbc0 0x7f3078151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:39.364 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 -- 172.21.15.55:0/1014054231 >> 172.21.15.55:0/1014054231 conn(0x7f30780d4aa0 msgr2=0x7f307815d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:39.364 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.362+0000 7f307f091640 1 -- 172.21.15.55:0/1014054231 shutdown_connections 2024-05-20T22:07:39.364 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:39.363+0000 7f307f091640 1 -- 172.21.15.55:0/1014054231 wait complete. 2024-05-20T22:07:39.373 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-20T22:07:43.095 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:07:43.098 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:07:46.180 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-20T22:07:46.298 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.297+0000 7f4307a1b640 1 Processor -- start 2024-05-20T22:07:46.298 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.297+0000 7f4307a1b640 1 -- start start 2024-05-20T22:07:46.300 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.298+0000 7f4307a1b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43000b8060 0x7f43000cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.300 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4307a1b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43000b3ac0 0x7f43000cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.300 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4307a1b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43000d3fd0 con 0x7f430015f170 2024-05-20T22:07:46.300 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4307a1b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f430006ceb0 con 0x7f43000b3ac0 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4307a1b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43000d42b0 con 0x7f43000b8060 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4305fb8640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43000b3ac0 0x7f43000cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4304fb6640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f430015f170 0x7f4300162b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43964/0 (socket says 172.21.15.55:43964) 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4304fb6640 1 -- 172.21.15.55:0/1258855641 learned_addr learned my addr 172.21.15.55:0/1258855641 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f43057b7640 1 --2- 172.21.15.55:0/1258855641 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43000b8060 0x7f43000cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f42f7fff640 1 -- 172.21.15.55:0/1258855641 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1727832759 0 0) 0x7f43000d3fd0 con 0x7f430015f170 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f42f7fff640 1 -- 172.21.15.55:0/1258855641 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42ec003520 con 0x7f430015f170 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4305fb8640 1 -- 172.21.15.55:0/1258855641 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43000b8060 msgr2=0x7f43000cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4305fb8640 1 --2- 172.21.15.55:0/1258855641 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43000b8060 0x7f43000cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4305fb8640 1 -- 172.21.15.55:0/1258855641 >> v1:172.21.15.55:6789/0 conn(0x7f430015f170 legacy=0x7f4300162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.299+0000 7f4305fb8640 1 -- 172.21.15.55:0/1258855641 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430006b6a0 con 0x7f43000b3ac0 2024-05-20T22:07:46.301 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.300+0000 7f4305fb8640 1 --2- 172.21.15.55:0/1258855641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 0x7f43000cfd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009d90 tx=0x7f42f802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195ef91a6eececd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:46.302 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.300+0000 7f42f7fff640 1 -- 172.21.15.55:0/1258855641 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8031600 con 0x7f43000b3ac0 2024-05-20T22:07:46.303 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.300+0000 7f42f7fff640 1 -- 172.21.15.55:0/1258855641 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f80317a0 con 0x7f43000b3ac0 2024-05-20T22:07:46.303 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.300+0000 7f42f7fff640 1 -- 172.21.15.55:0/1258855641 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8031b70 con 0x7f43000b3ac0 2024-05-20T22:07:46.303 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.301+0000 7f4307a1b640 1 -- 172.21.15.55:0/1258855641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 msgr2=0x7f43000cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.303 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.301+0000 7f4307a1b640 1 --2- 172.21.15.55:0/1258855641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 0x7f43000cfd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009d90 tx=0x7f42f802a5d0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.303 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.301+0000 7f4307a1b640 1 -- 172.21.15.55:0/1258855641 shutdown_connections 2024-05-20T22:07:46.303 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.301+0000 7f4307a1b640 1 --2- 172.21.15.55:0/1258855641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 0x7f43000cfd00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.303 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.301+0000 7f4307a1b640 1 --2- 172.21.15.55:0/1258855641 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43000b8060 0x7f43000cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.303 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.301+0000 7f4307a1b640 1 -- 172.21.15.55:0/1258855641 >> 172.21.15.55:0/1258855641 conn(0x7f43000d1c30 msgr2=0x7f43000d4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:46.303 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.301+0000 7f4307a1b640 1 -- 172.21.15.55:0/1258855641 shutdown_connections 2024-05-20T22:07:46.303 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.301+0000 7f4307a1b640 1 -- 172.21.15.55:0/1258855641 wait complete. 2024-05-20T22:07:46.304 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.303+0000 7f4307a1b640 1 Processor -- start 2024-05-20T22:07:46.304 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.303+0000 7f4307a1b640 1 -- start start 2024-05-20T22:07:46.306 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f4307a1b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 0x7f4300151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.306 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f4307a1b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43000b8060 0x7f4300156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.306 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f4307a1b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f430015f170 0x7f43000b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.306 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f4307a1b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f43000d2ce0 con 0x7f43000b3ac0 2024-05-20T22:07:46.306 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f4307a1b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43000d0610 con 0x7f43000b8060 2024-05-20T22:07:46.306 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f4307a1b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43000d2e60 con 0x7f430015f170 2024-05-20T22:07:46.306 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f43057b7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 0x7f4300151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f43057b7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 0x7f4300151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40714/0 (socket says 172.21.15.55:40714) 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f43057b7640 1 -- 172.21.15.55:0/841751091 learned_addr learned my addr 172.21.15.55:0/841751091 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f4305fb8640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f430015f170 0x7f43000b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f43057b7640 1 -- 172.21.15.55:0/841751091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f430015f170 msgr2=0x7f43000b6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f43057b7640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f430015f170 0x7f43000b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f43057b7640 1 -- 172.21.15.55:0/841751091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43000b8060 msgr2=0x7f4300156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f43057b7640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43000b8060 0x7f4300156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.304+0000 7f43057b7640 1 -- 172.21.15.55:0/841751091 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43001f9600 con 0x7f43000b3ac0 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.305+0000 7f43057b7640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 0x7f4300151f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00d5f0 tx=0x7f42fc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.305+0000 7f42f5ffb640 1 -- 172.21.15.55:0/841751091 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc0125f0 con 0x7f43000b3ac0 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.305+0000 7f42f5ffb640 1 -- 172.21.15.55:0/841751091 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42fc019070 con 0x7f43000b3ac0 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.305+0000 7f42f5ffb640 1 -- 172.21.15.55:0/841751091 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc0129d0 con 0x7f43000b3ac0 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.305+0000 7f4307a1b640 1 -- 172.21.15.55:0/841751091 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43001f8840 con 0x7f43000b3ac0 2024-05-20T22:07:46.307 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.305+0000 7f4307a1b640 1 -- 172.21.15.55:0/841751091 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43001f8e00 con 0x7f43000b3ac0 2024-05-20T22:07:46.308 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.306+0000 7f4307a1b640 1 -- 172.21.15.55:0/841751091 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43000759e0 con 0x7f43000b3ac0 2024-05-20T22:07:46.308 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.307+0000 7f42f5ffb640 1 -- 172.21.15.55:0/841751091 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f42fc012b70 con 0x7f43000b3ac0 2024-05-20T22:07:46.308 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.307+0000 7f42f5ffb640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f42e406be30 0x7f42e406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.309 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.307+0000 7f42f5ffb640 1 -- 172.21.15.55:0/841751091 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f42fc08ae90 con 0x7f43000b3ac0 2024-05-20T22:07:46.309 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.307+0000 7f4304fb6640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f42e406be30 0x7f42e406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.309 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.308+0000 7f4304fb6640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f42e406be30 0x7f42e406e340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000f60 tx=0x7f42f00078f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:46.313 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.312+0000 7f42f5ffb640 1 -- 172.21.15.55:0/841751091 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f42fc059c20 con 0x7f43000b3ac0 2024-05-20T22:07:46.501 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.500+0000 7f4307a1b640 1 -- 172.21.15.55:0/841751091 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43000b7980 con 0x7f43000b3ac0 2024-05-20T22:07:46.501 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.501+0000 7f42f5ffb640 1 -- 172.21.15.55:0/841751091 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f42fc059520 con 0x7f43000b3ac0 2024-05-20T22:07:46.502 INFO:teuthology.misc.health.smithi055.stdout: 2024-05-20T22:07:46.502 INFO:teuthology.misc.health.smithi055.stdout:{"epoch":16,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:43.010992+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:07:46.504 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 -- 172.21.15.55:0/841751091 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f42e406be30 msgr2=0x7f42e406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.504 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f42e406be30 0x7f42e406e340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000f60 tx=0x7f42f00078f0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.505 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 -- 172.21.15.55:0/841751091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 msgr2=0x7f4300151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.505 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 0x7f4300151f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00d5f0 tx=0x7f42fc00dab0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.505 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 -- 172.21.15.55:0/841751091 shutdown_connections 2024-05-20T22:07:46.505 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f42e406be30 0x7f42e406e340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.505 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f430015f170 0x7f43000b6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.505 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43000b8060 0x7f4300156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.505 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 --2- 172.21.15.55:0/841751091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43000b3ac0 0x7f4300151f80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.505 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 -- 172.21.15.55:0/841751091 >> 172.21.15.55:0/841751091 conn(0x7f43000d1c30 msgr2=0x7f43000b8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:46.505 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 -- 172.21.15.55:0/841751091 shutdown_connections 2024-05-20T22:07:46.505 INFO:teuthology.misc.health.smithi055.stderr:2024-05-20T22:07:46.503+0000 7f4307a1b640 1 -- 172.21.15.55:0/841751091 wait complete. 2024-05-20T22:07:46.513 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-20T22:07:46.514 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-20T22:07:46.514 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T22:07:46.514 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T22:07:46.516 INFO:tasks.ceph.mds.a:Started 2024-05-20T22:07:46.516 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T22:07:46.517 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T22:07:46.518 INFO:tasks.ceph.mds.c:Started 2024-05-20T22:07:46.519 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T22:07:46.519 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T22:07:46.521 INFO:tasks.ceph.mds.b:Started 2024-05-20T22:07:46.522 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T22:07:46.522 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T22:07:46.524 INFO:tasks.ceph.mds.d:Started 2024-05-20T22:07:46.524 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-20T22:07:46.524 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:07:46.606 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-20T22:07:46.607 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-20T22:07:46.607 INFO:tasks.ceph.mds.a.smithi055.stderr:2024-05-20T22:07:46.606+0000 7f150d39d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-05-20T22:07:46.622 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-05-20T22:07:46.622 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-05-20T22:07:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.644+0000 7f4072138640 1 Processor -- start 2024-05-20T22:07:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.644+0000 7f4072138640 1 -- start start 2024-05-20T22:07:46.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f4072138640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0b0e00 0x7f406c0d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f4072138640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f406c0d0e00 0x7f406c0ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f4072138640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406c09b220 con 0x7f406c0b2d50 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f4072138640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f406c15f610 con 0x7f406c0d0e00 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f4072138640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f406c09ac60 con 0x7f406c0b0e00 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f4062ffd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f406c0b2d50 0x7f406c0cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43996/0 (socket says 172.21.15.55:43996) 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f4062ffd640 1 -- 172.21.15.55:0/1671333084 learned_addr learned my addr 172.21.15.55:0/1671333084 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f406bfff640 1 --2- 172.21.15.55:0/1671333084 >> v2:172.21.15.55:3300/0 conn(0x7f406c0d0e00 0x7f406c0ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f406b7fe640 1 --2- 172.21.15.55:0/1671333084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0b0e00 0x7f406c0d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f406affd640 1 -- 172.21.15.55:0/1671333084 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364974418 0 0) 0x7f406c09b220 con 0x7f406c0b2d50 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f406affd640 1 -- 172.21.15.55:0/1671333084 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4058003520 con 0x7f406c0b2d50 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f406bfff640 1 -- 172.21.15.55:0/1671333084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0b0e00 msgr2=0x7f406c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f406bfff640 1 --2- 172.21.15.55:0/1671333084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0b0e00 0x7f406c0d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f406bfff640 1 -- 172.21.15.55:0/1671333084 >> v1:172.21.15.55:6789/0 conn(0x7f406c0b2d50 legacy=0x7f406c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f406bfff640 1 -- 172.21.15.55:0/1671333084 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c09af40 con 0x7f406c0d0e00 2024-05-20T22:07:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.646+0000 7f406b7fe640 1 --2- 172.21.15.55:0/1671333084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0b0e00 0x7f406c0d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.647+0000 7f406bfff640 1 --2- 172.21.15.55:0/1671333084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0d0e00 0x7f406c0ccdf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4064009d90 tx=0x7f406402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eca4fb26d6725e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.647+0000 7f406affd640 1 -- 172.21.15.55:0/1671333084 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064031600 con 0x7f406c0d0e00 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.647+0000 7f406affd640 1 -- 172.21.15.55:0/1671333084 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40640317a0 con 0x7f406c0d0e00 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.647+0000 7f406affd640 1 -- 172.21.15.55:0/1671333084 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064031b70 con 0x7f406c0d0e00 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.648+0000 7f4072138640 1 -- 172.21.15.55:0/1671333084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0d0e00 msgr2=0x7f406c0ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.648+0000 7f4072138640 1 --2- 172.21.15.55:0/1671333084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0d0e00 0x7f406c0ccdf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4064009d90 tx=0x7f406402a5d0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.648+0000 7f4072138640 1 -- 172.21.15.55:0/1671333084 shutdown_connections 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.648+0000 7f4072138640 1 --2- 172.21.15.55:0/1671333084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0d0e00 0x7f406c0ccdf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.648+0000 7f4072138640 1 --2- 172.21.15.55:0/1671333084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0b0e00 0x7f406c0d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.648+0000 7f4072138640 1 -- 172.21.15.55:0/1671333084 >> 172.21.15.55:0/1671333084 conn(0x7f406c0b6420 msgr2=0x7f406c0b6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.648+0000 7f4072138640 1 -- 172.21.15.55:0/1671333084 shutdown_connections 2024-05-20T22:07:46.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.648+0000 7f4072138640 1 -- 172.21.15.55:0/1671333084 wait complete. 2024-05-20T22:07:46.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.650+0000 7f4072138640 1 Processor -- start 2024-05-20T22:07:46.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.650+0000 7f4072138640 1 -- start start 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.651+0000 7f4072138640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0b0e00 0x7f406c09c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.651+0000 7f4072138640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f406c0b2d50 0x7f406c09ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f4072138640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0d0e00 0x7f406c150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f4072138640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f406c06e800 con 0x7f406c0b0e00 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f4072138640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f406c0cc1c0 con 0x7f406c0b2d50 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f4072138640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f406c0d1210 con 0x7f406c0d0e00 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f406b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0b0e00 0x7f406c09c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f406b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0b0e00 0x7f406c09c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40742/0 (socket says 172.21.15.55:40742) 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f406b7fe640 1 -- 172.21.15.55:0/4223525674 learned_addr learned my addr 172.21.15.55:0/4223525674 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f4062ffd640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f406c0b2d50 0x7f406c09ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f406bfff640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0d0e00 0x7f406c150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f406b7fe640 1 -- 172.21.15.55:0/4223525674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0d0e00 msgr2=0x7f406c150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f406b7fe640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0d0e00 0x7f406c150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f406b7fe640 1 -- 172.21.15.55:0/4223525674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f406c0b2d50 msgr2=0x7f406c09ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f406b7fe640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f406c0b2d50 0x7f406c09ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f406b7fe640 1 -- 172.21.15.55:0/4223525674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c09cdb0 con 0x7f406c0b0e00 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f406b7fe640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0b0e00 0x7f406c09c870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f405c00e960 tx=0x7f405c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f4068ff9640 1 -- 172.21.15.55:0/4223525674 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c017070 con 0x7f406c0b0e00 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f4068ff9640 1 -- 172.21.15.55:0/4223525674 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f405c00ca00 con 0x7f406c0b0e00 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f4072138640 1 -- 172.21.15.55:0/4223525674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c09b7d0 con 0x7f406c0b0e00 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f4068ff9640 1 -- 172.21.15.55:0/4223525674 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c00cdd0 con 0x7f406c0b0e00 2024-05-20T22:07:46.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.652+0000 7f4072138640 1 -- 172.21.15.55:0/4223525674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c09bd90 con 0x7f406c0b0e00 2024-05-20T22:07:46.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.654+0000 7f4072138640 1 -- 172.21.15.55:0/4223525674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c069090 con 0x7f406c0b0e00 2024-05-20T22:07:46.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.654+0000 7f4068ff9640 1 -- 172.21.15.55:0/4223525674 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f405c01a070 con 0x7f406c0b0e00 2024-05-20T22:07:46.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.654+0000 7f4068ff9640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f404c06be30 0x7f404c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.654+0000 7f4068ff9640 1 -- 172.21.15.55:0/4223525674 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f405c08ad30 con 0x7f406c0b0e00 2024-05-20T22:07:46.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.654+0000 7f4062ffd640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f404c06be30 0x7f404c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.655+0000 7f4062ffd640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f404c06be30 0x7f404c06e340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f406c0712c0 tx=0x7f4054009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:46.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.659+0000 7f4068ff9640 1 -- 172.21.15.55:0/4223525674 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f405c059350 con 0x7f406c0b0e00 2024-05-20T22:07:46.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.843+0000 7f4072138640 1 -- 172.21.15.55:0/4223525674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f406c071010 con 0x7f406c0b0e00 2024-05-20T22:07:46.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.844+0000 7f4068ff9640 1 -- 172.21.15.55:0/4223525674 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f405c05d650 con 0x7f406c0b0e00 2024-05-20T22:07:46.845 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:07:46.845 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:43.010992+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:07:46.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 -- 172.21.15.55:0/4223525674 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f404c06be30 msgr2=0x7f404c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f404c06be30 0x7f404c06e340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f406c0712c0 tx=0x7f4054009040 comp rx=0 tx=0).stop 2024-05-20T22:07:46.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 -- 172.21.15.55:0/4223525674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0b0e00 msgr2=0x7f406c09c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0b0e00 0x7f406c09c870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f405c00e960 tx=0x7f405c00ee20 comp rx=0 tx=0).stop 2024-05-20T22:07:46.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 -- 172.21.15.55:0/4223525674 shutdown_connections 2024-05-20T22:07:46.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f404c06be30 0x7f404c06e340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f406c0d0e00 0x7f406c150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f406c0b2d50 0x7f406c09ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 --2- 172.21.15.55:0/4223525674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c0b0e00 0x7f406c09c870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 -- 172.21.15.55:0/4223525674 >> 172.21.15.55:0/4223525674 conn(0x7f406c0b6420 msgr2=0x7f406c0b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:46.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 -- 172.21.15.55:0/4223525674 shutdown_connections 2024-05-20T22:07:46.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.846+0000 7f4072138640 1 -- 172.21.15.55:0/4223525674 wait complete. 2024-05-20T22:07:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T22:07:46.858 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:07:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.975+0000 7f44b359e640 1 Processor -- start 2024-05-20T22:07:46.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.975+0000 7f44b359e640 1 -- start start 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.976+0000 7f44b359e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f44b41597a0 0x7f44b415fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.976+0000 7f44b359e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44b406b590 0x7f44b406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.976+0000 7f44b359e640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44b406d4e0 con 0x7f44b406cf60 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.976+0000 7f44b359e640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f44b40b03d0 con 0x7f44b41597a0 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.976+0000 7f44b359e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f44b40afd80 con 0x7f44b406b590 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.976+0000 7f44b259c640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f44b406cf60 0x7f44b415f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44008/0 (socket says 172.21.15.55:44008) 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.976+0000 7f44b259c640 1 -- 172.21.15.55:0/2365144884 learned_addr learned my addr 172.21.15.55:0/2365144884 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.976+0000 7f44b1d9b640 1 --2- 172.21.15.55:0/2365144884 >> v2:172.21.15.55:3300/0 conn(0x7f44b41597a0 0x7f44b415fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.976+0000 7f44b2d9d640 1 --2- 172.21.15.55:0/2365144884 >> v2:172.21.15.138:3300/0 conn(0x7f44b406b590 0x7f44b406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.977+0000 7f44b1d9b640 1 -- 172.21.15.55:0/2365144884 >> v2:172.21.15.138:3300/0 conn(0x7f44b406b590 msgr2=0x7f44b406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.977+0000 7f44b1d9b640 1 --2- 172.21.15.55:0/2365144884 >> v2:172.21.15.138:3300/0 conn(0x7f44b406b590 0x7f44b406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.977+0000 7f44b1d9b640 1 -- 172.21.15.55:0/2365144884 >> v1:172.21.15.55:6789/0 conn(0x7f44b406cf60 legacy=0x7f44b415f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.977+0000 7f44b1d9b640 1 -- 172.21.15.55:0/2365144884 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b40b0060 con 0x7f44b41597a0 2024-05-20T22:07:46.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.977+0000 7f44b159a640 1 -- 172.21.15.55:0/2365144884 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827961150 0 0) 0x7f44b406d4e0 con 0x7f44b406cf60 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.977+0000 7f44b1d9b640 1 --2- 172.21.15.55:0/2365144884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 0x7f44b415fde0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f44b406d6b0 tx=0x7f449c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a4a0264d8bba6b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b159a640 1 -- 172.21.15.55:0/2365144884 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c037070 con 0x7f44b41597a0 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b159a640 1 -- 172.21.15.55:0/2365144884 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449c032440 con 0x7f44b41597a0 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b159a640 1 -- 172.21.15.55:0/2365144884 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c032810 con 0x7f44b41597a0 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b359e640 1 -- 172.21.15.55:0/2365144884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 msgr2=0x7f44b415fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b359e640 1 --2- 172.21.15.55:0/2365144884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 0x7f44b415fde0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f44b406d6b0 tx=0x7f449c02a1a0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b359e640 1 -- 172.21.15.55:0/2365144884 shutdown_connections 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b359e640 1 --2- 172.21.15.55:0/2365144884 >> v2:172.21.15.138:3300/0 conn(0x7f44b406b590 0x7f44b406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b359e640 1 --2- 172.21.15.55:0/2365144884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 0x7f44b415fde0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b359e640 1 -- 172.21.15.55:0/2365144884 >> 172.21.15.55:0/2365144884 conn(0x7f44b4072e20 msgr2=0x7f44b40b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:46.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b359e640 1 -- 172.21.15.55:0/2365144884 shutdown_connections 2024-05-20T22:07:46.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.978+0000 7f44b359e640 1 -- 172.21.15.55:0/2365144884 wait complete. 2024-05-20T22:07:46.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.980+0000 7f44b359e640 1 Processor -- start 2024-05-20T22:07:46.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.980+0000 7f44b359e640 1 -- start start 2024-05-20T22:07:46.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b359e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b406b590 0x7f44b41f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b359e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b406cf60 0x7f44b41f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b359e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 0x7f44b4153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b359e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41604a0 con 0x7f44b41597a0 2024-05-20T22:07:46.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b359e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4160320 con 0x7f44b406b590 2024-05-20T22:07:46.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b359e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44b406bee0 con 0x7f44b406cf60 2024-05-20T22:07:46.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b2d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 0x7f44b4153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b2d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 0x7f44b4153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40764/0 (socket says 172.21.15.55:40764) 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b2d9d640 1 -- 172.21.15.55:0/1008166509 learned_addr learned my addr 172.21.15.55:0/1008166509 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b259c640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b406b590 0x7f44b41f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b1d9b640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b406cf60 0x7f44b41f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b2d9d640 1 -- 172.21.15.55:0/1008166509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b406cf60 msgr2=0x7f44b41f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b2d9d640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b406cf60 0x7f44b41f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b2d9d640 1 -- 172.21.15.55:0/1008166509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b406b590 msgr2=0x7f44b41f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b2d9d640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b406b590 0x7f44b41f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b2d9d640 1 -- 172.21.15.55:0/1008166509 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b415b430 con 0x7f44b41597a0 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.982+0000 7f44b2d9d640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 0x7f44b4153fd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00e7f0 tx=0x7f44ac00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.983+0000 7f44aaffd640 1 -- 172.21.15.55:0/1008166509 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac017070 con 0x7f44b41597a0 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.983+0000 7f44aaffd640 1 -- 172.21.15.55:0/1008166509 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ac00ee40 con 0x7f44b41597a0 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.983+0000 7f44b359e640 1 -- 172.21.15.55:0/1008166509 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b41f2d10 con 0x7f44b41597a0 2024-05-20T22:07:46.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.983+0000 7f44aaffd640 1 -- 172.21.15.55:0/1008166509 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00cb20 con 0x7f44b41597a0 2024-05-20T22:07:46.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.983+0000 7f44b359e640 1 -- 172.21.15.55:0/1008166509 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b41f32d0 con 0x7f44b41597a0 2024-05-20T22:07:46.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.984+0000 7f44b359e640 1 -- 172.21.15.55:0/1008166509 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b40758b0 con 0x7f44b41597a0 2024-05-20T22:07:46.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.984+0000 7f44aaffd640 1 -- 172.21.15.55:0/1008166509 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44ac01a070 con 0x7f44b41597a0 2024-05-20T22:07:46.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.985+0000 7f44aaffd640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f449406be60 0x7f449406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:46.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.985+0000 7f44aaffd640 1 -- 172.21.15.55:0/1008166509 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f44ac08a8e0 con 0x7f44b41597a0 2024-05-20T22:07:46.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.985+0000 7f44b259c640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f449406be60 0x7f449406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:46.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.985+0000 7f44b259c640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f449406be60 0x7f449406e370 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44a40012e0 tx=0x7f44a4001630 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:46.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:46.989+0000 7f44aaffd640 1 -- 172.21.15.55:0/1008166509 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f44ac058f00 con 0x7f44b41597a0 2024-05-20T22:07:47.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.172+0000 7f44b359e640 1 -- 172.21.15.55:0/1008166509 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44b40b22e0 con 0x7f44b41597a0 2024-05-20T22:07:47.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.173+0000 7f44aaffd640 1 -- 172.21.15.55:0/1008166509 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f44ac05d200 con 0x7f44b41597a0 2024-05-20T22:07:47.174 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:07:47.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 -- 172.21.15.55:0/1008166509 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f449406be60 msgr2=0x7f449406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:47.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f449406be60 0x7f449406e370 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44a40012e0 tx=0x7f44a4001630 comp rx=0 tx=0).stop 2024-05-20T22:07:47.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 -- 172.21.15.55:0/1008166509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 msgr2=0x7f44b4153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:47.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 0x7f44b4153fd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00e7f0 tx=0x7f44ac00ebf0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 -- 172.21.15.55:0/1008166509 shutdown_connections 2024-05-20T22:07:47.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f449406be60 0x7f449406e370 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b41597a0 0x7f44b4153fd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b406cf60 0x7f44b41f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 --2- 172.21.15.55:0/1008166509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b406b590 0x7f44b41f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 -- 172.21.15.55:0/1008166509 >> 172.21.15.55:0/1008166509 conn(0x7f44b4072e20 msgr2=0x7f44b415f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:47.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 -- 172.21.15.55:0/1008166509 shutdown_connections 2024-05-20T22:07:47.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.175+0000 7f44b359e640 1 -- 172.21.15.55:0/1008166509 wait complete. 2024-05-20T22:07:47.187 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:07:47.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.299+0000 7f2f41fd5640 1 Processor -- start 2024-05-20T22:07:47.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.299+0000 7f2f41fd5640 1 -- start start 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f41fd5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c0b3b50 0x7f2f3c163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f41fd5640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f3c0d3ff0 con 0x7f2f3c15f140 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f41fd5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f3c0d42d0 con 0x7f2f3c0b8090 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f41fd5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c0b43a0 con 0x7f2f3c0b3b50 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3bfff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2f3c15f140 0x7f2f3c06be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44012/0 (socket says 172.21.15.55:44012) 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3bfff640 1 -- 172.21.15.55:0/2512882051 learned_addr learned my addr 172.21.15.55:0/2512882051 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3affd640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c0b3b50 0x7f2f3c163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4227474170 0 0) 0x7f2f3c0d3ff0 con 0x7f2f3c15f140 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f28003520 con 0x7f2f3c15f140 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2673034490 0 0) 0x7f2f3c0d42d0 con 0x7f2f3c0b8090 2024-05-20T22:07:47.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f3c0d3ff0 con 0x7f2f3c0b8090 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2788106890 0 0) 0x7f2f28003520 con 0x7f2f3c15f140 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c0b3b50 msgr2=0x7f2f3c163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c0b3b50 0x7f2f3c163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 >> v1:172.21.15.138:6789/0 conn(0x7f2f3c0b8090 legacy=0x7f2f3c162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c06b680 con 0x7f2f3c15f140 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f2f340019f0 con 0x7f2f3c15f140 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3affd640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c0b3b50 0x7f2f3c163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 >> v1:172.21.15.55:6789/0 conn(0x7f2f3c15f140 legacy=0x7f2f3c06be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f28003b70 0x7f2f28003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c0b3b50 0x7f2f28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f28004d40 0x7f2f28005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.301+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2800bac0 con 0x7f2f28004d40 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.302+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2800bc70 con 0x7f2f28003b70 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.302+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f2800be70 con 0x7f2f3c0b3b50 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.302+0000 7f2f3bfff640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f28004d40 0x7f2f28005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.302+0000 7f2f3affd640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c0b3b50 0x7f2f28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.302+0000 7f2f3b7fe640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f28003b70 0x7f2f28003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.302+0000 7f2f3bfff640 1 -- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c0b3b50 msgr2=0x7f2f28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.302+0000 7f2f3bfff640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c0b3b50 0x7f2f28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.302+0000 7f2f3bfff640 1 -- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f28003b70 msgr2=0x7f2f28003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.302+0000 7f2f3bfff640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f28003b70 0x7f2f28003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.302+0000 7f2f3bfff640 1 -- 172.21.15.55:0/2512882051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f28003520 con 0x7f2f28004d40 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.303+0000 7f2f3bfff640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f28004d40 0x7f2f28005150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0cbb50 tx=0x7f2f28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99e362d7d78b36c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.303+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400a330 con 0x7f2f28004d40 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.303+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f3402f070 con 0x7f2f28004d40 2024-05-20T22:07:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.303+0000 7f2f3a7fc640 1 -- 172.21.15.55:0/2512882051 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3402bb30 con 0x7f2f28004d40 2024-05-20T22:07:47.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.303+0000 7f2f41fd5640 1 -- 172.21.15.55:0/2512882051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f28004d40 msgr2=0x7f2f28005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:47.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.303+0000 7f2f41fd5640 1 --2- 172.21.15.55:0/2512882051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f28004d40 0x7f2f28005150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0cbb50 tx=0x7f2f28005830 comp rx=0 tx=0).stop 2024-05-20T22:07:47.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.303+0000 7f2f3bfff640 1 -- 172.21.15.55:0/2512882051 reap_dead start 2024-05-20T22:07:47.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.303+0000 7f2f41fd5640 1 -- 172.21.15.55:0/2512882051 shutdown_connections 2024-05-20T22:07:47.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.303+0000 7f2f41fd5640 1 -- 172.21.15.55:0/2512882051 >> 172.21.15.55:0/2512882051 conn(0x7f2f3c0d1bb0 msgr2=0x7f2f3c0d4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:47.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.304+0000 7f2f41fd5640 1 -- 172.21.15.55:0/2512882051 shutdown_connections 2024-05-20T22:07:47.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.304+0000 7f2f41fd5640 1 -- 172.21.15.55:0/2512882051 wait complete. 2024-05-20T22:07:47.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.305+0000 7f2f41fd5640 1 Processor -- start 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.306+0000 7f2f41fd5640 1 -- start start 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f41fd5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f3c1578b0 0x7f2f3c14fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f41fd5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f3c15a2c0 0x7f2f3c15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f41fd5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c1ea750 0x7f2f3c1eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f41fd5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c163400 con 0x7f2f3c1578b0 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f41fd5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c163280 con 0x7f2f3c15a2c0 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f41fd5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c163580 con 0x7f2f3c1ea750 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f3c1578b0 0x7f2f3c14fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f3c1578b0 0x7f2f3c14fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40780/0 (socket says 172.21.15.55:40780) 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3b7fe640 1 -- 172.21.15.55:0/844940580 learned_addr learned my addr 172.21.15.55:0/844940580 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3affd640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f3c15a2c0 0x7f2f3c15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:47.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3bfff640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c1ea750 0x7f2f3c1eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3b7fe640 1 -- 172.21.15.55:0/844940580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c1ea750 msgr2=0x7f2f3c1eab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3b7fe640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c1ea750 0x7f2f3c1eab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3b7fe640 1 -- 172.21.15.55:0/844940580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f3c15a2c0 msgr2=0x7f2f3c15a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3b7fe640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f3c15a2c0 0x7f2f3c15a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3b7fe640 1 -- 172.21.15.55:0/844940580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c15ac10 con 0x7f2f3c1578b0 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.307+0000 7f2f3b7fe640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f3c1578b0 0x7f2f3c14fe00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00f770 tx=0x7f2f2c00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.308+0000 7f2f33fff640 1 -- 172.21.15.55:0/844940580 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c01a070 con 0x7f2f3c1578b0 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.308+0000 7f2f33fff640 1 -- 172.21.15.55:0/844940580 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2c00fe80 con 0x7f2f3c1578b0 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.308+0000 7f2f41fd5640 1 -- 172.21.15.55:0/844940580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3c0b4120 con 0x7f2f3c1578b0 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.308+0000 7f2f33fff640 1 -- 172.21.15.55:0/844940580 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c007050 con 0x7f2f3c1578b0 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.308+0000 7f2f41fd5640 1 -- 172.21.15.55:0/844940580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3c0b46e0 con 0x7f2f3c1578b0 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.309+0000 7f2f41fd5640 1 -- 172.21.15.55:0/844940580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f3c069090 con 0x7f2f3c1578b0 2024-05-20T22:07:47.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.309+0000 7f2f33fff640 1 -- 172.21.15.55:0/844940580 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f2c01d070 con 0x7f2f3c1578b0 2024-05-20T22:07:47.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.310+0000 7f2f33fff640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f1c06be30 0x7f2f1c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:47.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.310+0000 7f2f33fff640 1 -- 172.21.15.55:0/844940580 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2f2c08a380 con 0x7f2f3c1578b0 2024-05-20T22:07:47.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.310+0000 7f2f3affd640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f1c06be30 0x7f2f1c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:47.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.310+0000 7f2f3affd640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f1c06be30 0x7f2f1c06e340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c15a080 tx=0x7f2f24009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:47.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.314+0000 7f2f33fff640 1 -- 172.21.15.55:0/844940580 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2f3c15ac10 con 0x7f2f3c1578b0 2024-05-20T22:07:47.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.537+0000 7f2f41fd5640 1 -- 172.21.15.55:0/844940580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f3c163960 con 0x7f2f3c1578b0 2024-05-20T22:07:47.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.538+0000 7f2f33fff640 1 -- 172.21.15.55:0/844940580 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f2f2c058a30 con 0x7f2f3c1578b0 2024-05-20T22:07:47.539 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:07:47.539 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-20T22:07:47.539 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 2 2024-05-20T22:07:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 -- 172.21.15.55:0/844940580 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f1c06be30 msgr2=0x7f2f1c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f1c06be30 0x7f2f1c06e340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c15a080 tx=0x7f2f24009190 comp rx=0 tx=0).stop 2024-05-20T22:07:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 -- 172.21.15.55:0/844940580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f3c1578b0 msgr2=0x7f2f3c14fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f3c1578b0 0x7f2f3c14fe00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00f770 tx=0x7f2f2c00fc30 comp rx=0 tx=0).stop 2024-05-20T22:07:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 -- 172.21.15.55:0/844940580 shutdown_connections 2024-05-20T22:07:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f1c06be30 0x7f2f1c06e340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f3c1ea750 0x7f2f3c1eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f3c15a2c0 0x7f2f3c15a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 --2- 172.21.15.55:0/844940580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f3c1578b0 0x7f2f3c14fe00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:47.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 -- 172.21.15.55:0/844940580 >> 172.21.15.55:0/844940580 conn(0x7f2f3c0d1bb0 msgr2=0x7f2f3c15ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:47.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 -- 172.21.15.55:0/844940580 shutdown_connections 2024-05-20T22:07:47.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:47.540+0000 7f2f41fd5640 1 -- 172.21.15.55:0/844940580 wait complete. 2024-05-20T22:07:49.552 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:07:49.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.680+0000 7f7ebe63d640 1 Processor -- start 2024-05-20T22:07:49.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.680+0000 7f7ebe63d640 1 -- start start 2024-05-20T22:07:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.682+0000 7f7ebe63d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7eb80d3390 0x7f7eb806c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.682+0000 7f7ebe63d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb806ce50 con 0x7f7eb80b3c90 2024-05-20T22:07:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.682+0000 7f7ebe63d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb806b870 con 0x7f7eb80b8720 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.682+0000 7f7ebe63d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb806bb50 con 0x7f7eb80d3390 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.682+0000 7f7eb77fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7eb80b3c90 0x7f7eb80b8010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35710/0 (socket says 172.21.15.55:35710) 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.682+0000 7f7eb77fe640 1 -- 172.21.15.55:0/1837196569 learned_addr learned my addr 172.21.15.55:0/1837196569 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.682+0000 7f7eb7fff640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7eb80d3390 0x7f7eb806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.682+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 781226349 0 0) 0x7f7eb806ce50 con 0x7f7eb80b3c90 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.682+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ea0003520 con 0x7f7eb80b3c90 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1957191888 0 0) 0x7f7ea0003520 con 0x7f7eb80b3c90 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7eb80d3390 msgr2=0x7f7eb806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7eb80d3390 0x7f7eb806c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 >> v1:172.21.15.138:6789/0 conn(0x7f7eb80b8720 legacy=0x7f7eb815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb806be30 con 0x7f7eb80b3c90 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f7eac001670 con 0x7f7eb80b3c90 2024-05-20T22:07:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 >> v1:172.21.15.55:6789/0 conn(0x7f7eb80b3c90 legacy=0x7f7eb80b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ea00039e0 0x7f7ea0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ea0004390 0x7f7ea00048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ea0005160 0x7f7ea000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb806bb50 con 0x7f7ea0004390 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea000c0c0 con 0x7f7ea0005160 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea000c2c0 con 0x7f7ea00039e0 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb7fff640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7eb80d3390 0x7f7eb806c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb77fe640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ea0004390 0x7f7ea00048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7ebcbda640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ea0005160 0x7f7ea000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb7fff640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ea00039e0 0x7f7ea0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb77fe640 1 -- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ea00039e0 msgr2=0x7f7ea0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb77fe640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ea00039e0 0x7f7ea0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:49.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb77fe640 1 -- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ea0005160 msgr2=0x7f7ea000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:49.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb77fe640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ea0005160 0x7f7ea000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:49.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.683+0000 7f7eb77fe640 1 -- 172.21.15.55:0/1837196569 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea0003520 con 0x7f7ea0004390 2024-05-20T22:07:49.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.684+0000 7f7eb77fe640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ea0004390 0x7f7ea00048c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00c240 tx=0x7f7ea0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca42e6b6b3697436 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:49.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.684+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac007d30 con 0x7f7ea0004390 2024-05-20T22:07:49.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.684+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eac005af0 con 0x7f7ea0004390 2024-05-20T22:07:49.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.684+0000 7f7eb6ffd640 1 -- 172.21.15.55:0/1837196569 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac007070 con 0x7f7ea0004390 2024-05-20T22:07:49.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.685+0000 7f7ebe63d640 1 -- 172.21.15.55:0/1837196569 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ea0004390 msgr2=0x7f7ea00048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:49.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.685+0000 7f7ebe63d640 1 --2- 172.21.15.55:0/1837196569 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ea0004390 0x7f7ea00048c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00c240 tx=0x7f7ea0005830 comp rx=0 tx=0).stop 2024-05-20T22:07:49.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.685+0000 7f7ebcbda640 1 -- 172.21.15.55:0/1837196569 reap_dead start 2024-05-20T22:07:49.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.685+0000 7f7ebe63d640 1 -- 172.21.15.55:0/1837196569 shutdown_connections 2024-05-20T22:07:49.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.685+0000 7f7ebe63d640 1 -- 172.21.15.55:0/1837196569 >> 172.21.15.55:0/1837196569 conn(0x7f7eb80b65f0 msgr2=0x7f7eb80b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:49.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.685+0000 7f7ebe63d640 1 -- 172.21.15.55:0/1837196569 shutdown_connections 2024-05-20T22:07:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.685+0000 7f7ebe63d640 1 -- 172.21.15.55:0/1837196569 wait complete. 2024-05-20T22:07:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.687+0000 7f7ebe63d640 1 Processor -- start 2024-05-20T22:07:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.687+0000 7f7ebe63d640 1 -- start start 2024-05-20T22:07:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.688+0000 7f7ebe63d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eb80cf640 0x7f7eb80cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.688+0000 7f7ebe63d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eb80b2d30 0x7f7eb80b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.688+0000 7f7ebe63d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7eb80b3140 0x7f7eb80b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.688+0000 7f7ebe63d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb806b500 con 0x7f7eb80b2d30 2024-05-20T22:07:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.688+0000 7f7ebe63d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb806c3f0 con 0x7f7eb80cf640 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.688+0000 7f7ebe63d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb80b8b30 con 0x7f7eb80b3140 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.688+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eb80b2d30 0x7f7eb80b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.688+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eb80b2d30 0x7f7eb80b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38374/0 (socket says 172.21.15.55:38374) 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.688+0000 7f7eb77fe640 1 -- 172.21.15.55:0/4029495559 learned_addr learned my addr 172.21.15.55:0/4029495559 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7ebcbda640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7eb80b3140 0x7f7eb80b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7eb7fff640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eb80cf640 0x7f7eb80cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7eb77fe640 1 -- 172.21.15.55:0/4029495559 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7eb80b3140 msgr2=0x7f7eb80b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7eb77fe640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7eb80b3140 0x7f7eb80b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7eb77fe640 1 -- 172.21.15.55:0/4029495559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eb80cf640 msgr2=0x7f7eb80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7eb77fe640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eb80cf640 0x7f7eb80cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7eb77fe640 1 -- 172.21.15.55:0/4029495559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb80cfa50 con 0x7f7eb80b2d30 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7eb77fe640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eb80b2d30 0x7f7eb80b5680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7eac000fb0 tx=0x7f7eb80d10b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7eb4ff9640 1 -- 172.21.15.55:0/4029495559 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac042020 con 0x7f7eb80b2d30 2024-05-20T22:07:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7eb4ff9640 1 -- 172.21.15.55:0/4029495559 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eac0049e0 con 0x7f7eb80b2d30 2024-05-20T22:07:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7ebe63d640 1 -- 172.21.15.55:0/4029495559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb80cd9f0 con 0x7f7eb80b2d30 2024-05-20T22:07:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7eb4ff9640 1 -- 172.21.15.55:0/4029495559 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac004d90 con 0x7f7eb80b2d30 2024-05-20T22:07:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.689+0000 7f7ebe63d640 1 -- 172.21.15.55:0/4029495559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb80cdfb0 con 0x7f7eb80b2d30 2024-05-20T22:07:49.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.690+0000 7f7ebe63d640 1 -- 172.21.15.55:0/4029495559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb80ce670 con 0x7f7eb80b2d30 2024-05-20T22:07:49.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.691+0000 7f7eb4ff9640 1 -- 172.21.15.55:0/4029495559 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7eac045020 con 0x7f7eb80b2d30 2024-05-20T22:07:49.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.691+0000 7f7eb4ff9640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7ea406be30 0x7f7ea406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:49.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.692+0000 7f7eb4ff9640 1 -- 172.21.15.55:0/4029495559 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7eac0ab200 con 0x7f7eb80b2d30 2024-05-20T22:07:49.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.692+0000 7f7eb7fff640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7ea406be30 0x7f7ea406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:49.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.692+0000 7f7eb7fff640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7ea406be30 0x7f7ea406e340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8154680 tx=0x7f7ea8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:49.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.696+0000 7f7eb4ff9640 1 -- 172.21.15.55:0/4029495559 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7eb80ce670 con 0x7f7eb80b2d30 2024-05-20T22:07:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.917+0000 7f7ebe63d640 1 -- 172.21.15.55:0/4029495559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7eb8069090 con 0x7f7eb80b2d30 2024-05-20T22:07:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.917+0000 7f7eb4ff9640 1 -- 172.21.15.55:0/4029495559 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7f7eac04c070 con 0x7f7eb80b2d30 2024-05-20T22:07:49.918 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:07:49.918 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-20T22:07:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 3 2024-05-20T22:07:49.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 -- 172.21.15.55:0/4029495559 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7ea406be30 msgr2=0x7f7ea406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:49.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7ea406be30 0x7f7ea406e340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8154680 tx=0x7f7ea8008040 comp rx=0 tx=0).stop 2024-05-20T22:07:49.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 -- 172.21.15.55:0/4029495559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eb80b2d30 msgr2=0x7f7eb80b5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:49.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eb80b2d30 0x7f7eb80b5680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7eac000fb0 tx=0x7f7eb80d10b0 comp rx=0 tx=0).stop 2024-05-20T22:07:49.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 -- 172.21.15.55:0/4029495559 shutdown_connections 2024-05-20T22:07:49.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7ea406be30 0x7f7ea406e340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:49.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7eb80b3140 0x7f7eb80b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:49.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eb80b2d30 0x7f7eb80b5680 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:49.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 --2- 172.21.15.55:0/4029495559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eb80cf640 0x7f7eb80cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:49.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 -- 172.21.15.55:0/4029495559 >> 172.21.15.55:0/4029495559 conn(0x7f7eb80b65f0 msgr2=0x7f7eb80c88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:49.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 -- 172.21.15.55:0/4029495559 shutdown_connections 2024-05-20T22:07:49.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:49.919+0000 7f7ebe63d640 1 -- 172.21.15.55:0/4029495559 wait complete. 2024-05-20T22:07:49.931 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:07:50.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.045+0000 7f2f1b8fe640 1 Processor -- start 2024-05-20T22:07:50.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.045+0000 7f2f1b8fe640 1 -- start start 2024-05-20T22:07:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.046+0000 7f2f1b8fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f140b3b30 0x7f2f140b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.046+0000 7f2f1b8fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f1406c790 0x7f2f140cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.046+0000 7f2f1b8fe640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f140d3dd0 con 0x7f2f1415f100 2024-05-20T22:07:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.046+0000 7f2f1b8fe640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f2f140d40b0 con 0x7f2f1406c790 2024-05-20T22:07:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.046+0000 7f2f1b8fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f140d4390 con 0x7f2f140b3b30 2024-05-20T22:07:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.047+0000 7f2f19e9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f1406c790 0x7f2f140cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.047+0000 7f2f19e9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f1406c790 0x7f2f140cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38384/0 (socket says 172.21.15.55:38384) 2024-05-20T22:07:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.047+0000 7f2f19e9b640 1 -- 172.21.15.55:0/796635608 learned_addr learned my addr 172.21.15.55:0/796635608 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.047+0000 7f2f1969a640 1 --2- 172.21.15.55:0/796635608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f140b3b30 0x7f2f140b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.047+0000 7f2f19e9b640 1 -- 172.21.15.55:0/796635608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f140b3b30 msgr2=0x7f2f140b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.047+0000 7f2f19e9b640 1 --2- 172.21.15.55:0/796635608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f140b3b30 0x7f2f140b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.047+0000 7f2f19e9b640 1 -- 172.21.15.55:0/796635608 >> v1:172.21.15.138:6789/0 conn(0x7f2f1415f100 legacy=0x7f2f1406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.047+0000 7f2f19e9b640 1 -- 172.21.15.55:0/796635608 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1406b5a0 con 0x7f2f1406c790 2024-05-20T22:07:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.047+0000 7f2f1969a640 1 --2- 172.21.15.55:0/796635608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f140b3b30 0x7f2f140b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:07:50.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.048+0000 7f2f19e9b640 1 --2- 172.21.15.55:0/796635608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f1406c790 0x7f2f140cf740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009e50 tx=0x7f2f0c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab6fe9d1d30f18e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:50.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.048+0000 7f2f0bfff640 1 -- 172.21.15.55:0/796635608 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c036070 con 0x7f2f1406c790 2024-05-20T22:07:50.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.048+0000 7f2f0bfff640 1 -- 172.21.15.55:0/796635608 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c031600 con 0x7f2f1406c790 2024-05-20T22:07:50.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.048+0000 7f2f0bfff640 1 -- 172.21.15.55:0/796635608 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0319d0 con 0x7f2f1406c790 2024-05-20T22:07:50.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.049+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/796635608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f1406c790 msgr2=0x7f2f140cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.049+0000 7f2f1b8fe640 1 --2- 172.21.15.55:0/796635608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f1406c790 0x7f2f140cf740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009e50 tx=0x7f2f0c02a710 comp rx=0 tx=0).stop 2024-05-20T22:07:50.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.049+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/796635608 shutdown_connections 2024-05-20T22:07:50.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.049+0000 7f2f1b8fe640 1 --2- 172.21.15.55:0/796635608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f1406c790 0x7f2f140cf740 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.049+0000 7f2f1b8fe640 1 --2- 172.21.15.55:0/796635608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f140b3b30 0x7f2f140b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.049+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/796635608 >> 172.21.15.55:0/796635608 conn(0x7f2f140d51d0 msgr2=0x7f2f140b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:50.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.049+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/796635608 shutdown_connections 2024-05-20T22:07:50.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.049+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/796635608 wait complete. 2024-05-20T22:07:50.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.050+0000 7f2f1b8fe640 1 Processor -- start 2024-05-20T22:07:50.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.051+0000 7f2f1b8fe640 1 -- start start 2024-05-20T22:07:50.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f1b8fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406c790 0x7f2f141f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f1b8fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f140b3b30 0x7f2f141568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f1b8fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1415f100 0x7f2f141f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f1b8fe640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1406cba0 con 0x7f2f140b3b30 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f1b8fe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1406bb60 con 0x7f2f1406c790 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f1b8fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1406cd20 con 0x7f2f1415f100 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f18e99640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f140b3b30 0x7f2f141568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f18e99640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f140b3b30 0x7f2f141568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38392/0 (socket says 172.21.15.55:38392) 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f18e99640 1 -- 172.21.15.55:0/2376867571 learned_addr learned my addr 172.21.15.55:0/2376867571 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f1969a640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406c790 0x7f2f141f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f19e9b640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1415f100 0x7f2f141f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f18e99640 1 -- 172.21.15.55:0/2376867571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1415f100 msgr2=0x7f2f141f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f18e99640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1415f100 0x7f2f141f92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f18e99640 1 -- 172.21.15.55:0/2376867571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406c790 msgr2=0x7f2f141f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f18e99640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406c790 0x7f2f141f3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f18e99640 1 -- 172.21.15.55:0/2376867571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f140b2d30 con 0x7f2f140b3b30 2024-05-20T22:07:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.052+0000 7f2f18e99640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f140b3b30 0x7f2f141568e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400c3f0 tx=0x7f2f0400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.053+0000 7f2f09ffb640 1 -- 172.21.15.55:0/2376867571 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f040052d0 con 0x7f2f140b3b30 2024-05-20T22:07:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.053+0000 7f2f09ffb640 1 -- 172.21.15.55:0/2376867571 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f04005490 con 0x7f2f140b3b30 2024-05-20T22:07:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.053+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/2376867571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f14153f60 con 0x7f2f140b3b30 2024-05-20T22:07:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.053+0000 7f2f09ffb640 1 -- 172.21.15.55:0/2376867571 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04007260 con 0x7f2f140b3b30 2024-05-20T22:07:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.053+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/2376867571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f140b3230 con 0x7f2f140b3b30 2024-05-20T22:07:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.054+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/2376867571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f14069090 con 0x7f2f140b3b30 2024-05-20T22:07:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.054+0000 7f2f09ffb640 1 -- 172.21.15.55:0/2376867571 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f04007400 con 0x7f2f140b3b30 2024-05-20T22:07:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.055+0000 7f2f09ffb640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ef806be60 0x7f2ef806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.055+0000 7f2f09ffb640 1 -- 172.21.15.55:0/2376867571 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2f0408a040 con 0x7f2f140b3b30 2024-05-20T22:07:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.055+0000 7f2f1969a640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ef806be60 0x7f2ef806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.055+0000 7f2f1969a640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ef806be60 0x7f2ef806e370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f10006fd0 tx=0x7f2f10008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.059+0000 7f2f09ffb640 1 -- 172.21.15.55:0/2376867571 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2f040585e0 con 0x7f2f140b3b30 2024-05-20T22:07:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.249+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/2376867571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f140d2ee0 con 0x7f2f140b3b30 2024-05-20T22:07:50.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.250+0000 7f2f09ffb640 1 -- 172.21.15.55:0/2376867571 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f2f0405c8e0 con 0x7f2f140b3b30 2024-05-20T22:07:50.251 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:07:50.251 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:43.010992+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.252+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/2376867571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ef806be60 msgr2=0x7f2ef806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.252+0000 7f2f1b8fe640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ef806be60 0x7f2ef806e370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f10006fd0 tx=0x7f2f10008040 comp rx=0 tx=0).stop 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.252+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/2376867571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f140b3b30 msgr2=0x7f2f141568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.252+0000 7f2f1b8fe640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f140b3b30 0x7f2f141568e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400c3f0 tx=0x7f2f0400c8b0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.253+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/2376867571 shutdown_connections 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.253+0000 7f2f1b8fe640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ef806be60 0x7f2ef806e370 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.253+0000 7f2f1b8fe640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f1415f100 0x7f2f141f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.253+0000 7f2f1b8fe640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f140b3b30 0x7f2f141568e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.253+0000 7f2f1b8fe640 1 --2- 172.21.15.55:0/2376867571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406c790 0x7f2f141f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.253+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/2376867571 >> 172.21.15.55:0/2376867571 conn(0x7f2f140d51d0 msgr2=0x7f2f140b83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.253+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/2376867571 shutdown_connections 2024-05-20T22:07:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.253+0000 7f2f1b8fe640 1 -- 172.21.15.55:0/2376867571 wait complete. 2024-05-20T22:07:50.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T22:07:50.265 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:07:50.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.382+0000 7f50fb6de640 1 Processor -- start 2024-05-20T22:07:50.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.382+0000 7f50fb6de640 1 -- start start 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50fb6de640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50f40b35e0 0x7f50f415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50fb6de640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f406b500 con 0x7f50f40b4080 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50fb6de640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f40b3e10 con 0x7f50f40d3290 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50fb6de640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50f4163960 con 0x7f50f40b35e0 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50f947a640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f50f40b4080 0x7f50f415f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35712/0 (socket says 172.21.15.55:35712) 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50f947a640 1 -- 172.21.15.55:0/2706941955 learned_addr learned my addr 172.21.15.55:0/2706941955 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50f8c79640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50f40b35e0 0x7f50f415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3485703046 0 0) 0x7f50f0001110 con 0x7f50f40b4080 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50e0003520 con 0x7f50f40b4080 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1629222178 0 0) 0x7f50e0003520 con 0x7f50f40b4080 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50f40b35e0 msgr2=0x7f50f415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50ebfff640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50f40b35e0 0x7f50f415f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 >> v1:172.21.15.138:6789/0 conn(0x7f50f40d3290 legacy=0x7f50f40d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f40b3b30 con 0x7f50f40b4080 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f50f0001840 con 0x7f50f40b4080 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 >> v1:172.21.15.55:6789/0 conn(0x7f50f40b4080 legacy=0x7f50f415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.384+0000 7f50ebfff640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50f40b35e0 0x7f50e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50ebfff640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e0003f60 0x7f50e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50ebfff640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50e0004d40 0x7f50e0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4163960 con 0x7f50e0004d40 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50e000bb50 con 0x7f50e0003f60 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50e000bd50 con 0x7f50f40b35e0 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50f9c7b640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50e0004d40 0x7f50e0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50f947a640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50f40b35e0 0x7f50e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50f8c79640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e0003f60 0x7f50e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50f9c7b640 1 -- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50f40b35e0 msgr2=0x7f50e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50f9c7b640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50f40b35e0 0x7f50e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50f9c7b640 1 -- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e0003f60 msgr2=0x7f50e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50f9c7b640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50e0003f60 0x7f50e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.385+0000 7f50f9c7b640 1 -- 172.21.15.55:0/2706941955 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e0003520 con 0x7f50e0004d40 2024-05-20T22:07:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.386+0000 7f50f9c7b640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50e0004d40 0x7f50e0005150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50f40b7510 tx=0x7f50e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ed79bfaff53b5b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.386+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec001a70 con 0x7f50e0004d40 2024-05-20T22:07:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.386+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec02ae10 con 0x7f50e0004d40 2024-05-20T22:07:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.386+0000 7f50ebfff640 1 -- 172.21.15.55:0/2706941955 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec003bb0 con 0x7f50e0004d40 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.386+0000 7f50fb6de640 1 -- 172.21.15.55:0/2706941955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50e0004d40 msgr2=0x7f50e0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.386+0000 7f50fb6de640 1 --2- 172.21.15.55:0/2706941955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50e0004d40 0x7f50e0005150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50f40b7510 tx=0x7f50e0005830 comp rx=0 tx=0).stop 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.387+0000 7f50f9c7b640 1 -- 172.21.15.55:0/2706941955 reap_dead start 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.387+0000 7f50fb6de640 1 -- 172.21.15.55:0/2706941955 shutdown_connections 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.387+0000 7f50fb6de640 1 -- 172.21.15.55:0/2706941955 >> 172.21.15.55:0/2706941955 conn(0x7f50f40d51b0 msgr2=0x7f50f40b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.387+0000 7f50fb6de640 1 -- 172.21.15.55:0/2706941955 shutdown_connections 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.387+0000 7f50fb6de640 1 -- 172.21.15.55:0/2706941955 wait complete. 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.388+0000 7f50fb6de640 1 Processor -- start 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.389+0000 7f50fb6de640 1 -- start start 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50fb6de640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50e0004d40 0x7f50f415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50fb6de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f415ba10 0x7f50f415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50fb6de640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50f40ccf30 0x7f50f40cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50fb6de640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41635f0 con 0x7f50f40ccf30 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50fb6de640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41604f0 con 0x7f50f415ba10 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50fb6de640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50f415f9f0 con 0x7f50e0004d40 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f9c7b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50f40ccf30 0x7f50f40cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f9c7b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50f40ccf30 0x7f50f40cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38406/0 (socket says 172.21.15.55:38406) 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f9c7b640 1 -- 172.21.15.55:0/2155567367 learned_addr learned my addr 172.21.15.55:0/2155567367 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f947a640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50e0004d40 0x7f50f415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f8c79640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f415ba10 0x7f50f415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f9c7b640 1 -- 172.21.15.55:0/2155567367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50e0004d40 msgr2=0x7f50f415a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f9c7b640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50e0004d40 0x7f50f415a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f9c7b640 1 -- 172.21.15.55:0/2155567367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f415ba10 msgr2=0x7f50f415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f9c7b640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f415ba10 0x7f50f415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f9c7b640 1 -- 172.21.15.55:0/2155567367 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f40cd880 con 0x7f50f40ccf30 2024-05-20T22:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.390+0000 7f50f9c7b640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50f40ccf30 0x7f50f40cd340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f50ec02a8c0 tx=0x7f50ec0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.391+0000 7f50e9ffb640 1 -- 172.21.15.55:0/2155567367 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec037040 con 0x7f50f40ccf30 2024-05-20T22:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.391+0000 7f50e9ffb640 1 -- 172.21.15.55:0/2155567367 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec02ac10 con 0x7f50f40ccf30 2024-05-20T22:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.391+0000 7f50fb6de640 1 -- 172.21.15.55:0/2155567367 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f415c790 con 0x7f50f40ccf30 2024-05-20T22:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.391+0000 7f50e9ffb640 1 -- 172.21.15.55:0/2155567367 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec007db0 con 0x7f50f40ccf30 2024-05-20T22:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.391+0000 7f50fb6de640 1 -- 172.21.15.55:0/2155567367 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f415cd50 con 0x7f50f40ccf30 2024-05-20T22:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.392+0000 7f50fb6de640 1 -- 172.21.15.55:0/2155567367 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f4069090 con 0x7f50f40ccf30 2024-05-20T22:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.392+0000 7f50e9ffb640 1 -- 172.21.15.55:0/2155567367 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50ec042020 con 0x7f50f40ccf30 2024-05-20T22:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.393+0000 7f50e9ffb640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f50d806be60 0x7f50d806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.393+0000 7f50e9ffb640 1 -- 172.21.15.55:0/2155567367 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f50ec0aa1c0 con 0x7f50f40ccf30 2024-05-20T22:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.393+0000 7f50f947a640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f50d806be60 0x7f50d806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.393+0000 7f50f947a640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f50d806be60 0x7f50d806e370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f50f0001050 tx=0x7f50f0000f80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:50.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.398+0000 7f50e9ffb640 1 -- 172.21.15.55:0/2155567367 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f50f40cd880 con 0x7f50f40ccf30 2024-05-20T22:07:50.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.581+0000 7f50fb6de640 1 -- 172.21.15.55:0/2155567367 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f50f40d2ec0 con 0x7f50f40ccf30 2024-05-20T22:07:50.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.582+0000 7f50e9ffb640 1 -- 172.21.15.55:0/2155567367 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f50ec078e10 con 0x7f50f40ccf30 2024-05-20T22:07:50.583 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 -- 172.21.15.55:0/2155567367 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f50d806be60 msgr2=0x7f50d806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f50d806be60 0x7f50d806e370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f50f0001050 tx=0x7f50f0000f80 comp rx=0 tx=0).stop 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 -- 172.21.15.55:0/2155567367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50f40ccf30 msgr2=0x7f50f40cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50f40ccf30 0x7f50f40cd340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f50ec02a8c0 tx=0x7f50ec0014a0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 -- 172.21.15.55:0/2155567367 shutdown_connections 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f50d806be60 0x7f50d806e370 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50f40ccf30 0x7f50f40cd340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f415ba10 0x7f50f415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 --2- 172.21.15.55:0/2155567367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50e0004d40 0x7f50f415a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 -- 172.21.15.55:0/2155567367 >> 172.21.15.55:0/2155567367 conn(0x7f50f40d51b0 msgr2=0x7f50f415ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:50.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 -- 172.21.15.55:0/2155567367 shutdown_connections 2024-05-20T22:07:50.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.584+0000 7f50fb6de640 1 -- 172.21.15.55:0/2155567367 wait complete. 2024-05-20T22:07:50.596 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T22:07:50.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.709+0000 7f66c3c32640 1 Processor -- start 2024-05-20T22:07:50.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.709+0000 7f66c3c32640 1 -- start start 2024-05-20T22:07:50.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c3c32640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f66bc0d3390 0x7f66bc06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c3c32640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f66bc0b3c90 0x7f66bc0b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c3c32640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66bc0b8550 0x7f66bc15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c3c32640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f66bc06ce50 con 0x7f66bc0d3390 2024-05-20T22:07:50.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c3c32640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f66bc06b870 con 0x7f66bc0b3c90 2024-05-20T22:07:50.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c3c32640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f66bc06bb50 con 0x7f66bc0b8550 2024-05-20T22:07:50.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c19ce640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f66bc0d3390 0x7f66bc06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c19ce640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f66bc0d3390 0x7f66bc06c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38412/0 (socket says 172.21.15.55:38412) 2024-05-20T22:07:50.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c19ce640 1 -- 172.21.15.55:0/1803029304 learned_addr learned my addr 172.21.15.55:0/1803029304 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:50.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c21cf640 1 --2- 172.21.15.55:0/1803029304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66bc0b8550 0x7f66bc15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c11cd640 1 --2- 172.21.15.55:0/1803029304 >> v2:172.21.15.138:3300/0 conn(0x7f66bc0b3c90 0x7f66bc0b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c19ce640 1 -- 172.21.15.55:0/1803029304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66bc0b8550 msgr2=0x7f66bc15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:50.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c19ce640 1 --2- 172.21.15.55:0/1803029304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66bc0b8550 0x7f66bc15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c19ce640 1 -- 172.21.15.55:0/1803029304 >> v2:172.21.15.138:3300/0 conn(0x7f66bc0b3c90 msgr2=0x7f66bc0b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:50.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c19ce640 1 --2- 172.21.15.55:0/1803029304 >> v2:172.21.15.138:3300/0 conn(0x7f66bc0b3c90 0x7f66bc0b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.711+0000 7f66c19ce640 1 -- 172.21.15.55:0/1803029304 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc06be30 con 0x7f66bc0d3390 2024-05-20T22:07:50.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.712+0000 7f66c19ce640 1 --2- 172.21.15.55:0/1803029304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0d3390 0x7f66bc06c910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0cbb20 tx=0x7f66b802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611c05f2e9267b73 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:50.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.712+0000 7f66c09cc640 1 -- 172.21.15.55:0/1803029304 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8037070 con 0x7f66bc0d3390 2024-05-20T22:07:50.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.712+0000 7f66c09cc640 1 -- 172.21.15.55:0/1803029304 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b8032440 con 0x7f66bc0d3390 2024-05-20T22:07:50.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.712+0000 7f66c09cc640 1 -- 172.21.15.55:0/1803029304 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8032810 con 0x7f66bc0d3390 2024-05-20T22:07:50.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.713+0000 7f66c3c32640 1 -- 172.21.15.55:0/1803029304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0d3390 msgr2=0x7f66bc06c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.713+0000 7f66c3c32640 1 --2- 172.21.15.55:0/1803029304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0d3390 0x7f66bc06c910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0cbb20 tx=0x7f66b802a140 comp rx=0 tx=0).stop 2024-05-20T22:07:50.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.713+0000 7f66c3c32640 1 -- 172.21.15.55:0/1803029304 shutdown_connections 2024-05-20T22:07:50.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.713+0000 7f66c3c32640 1 --2- 172.21.15.55:0/1803029304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66bc0b8550 0x7f66bc15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.713+0000 7f66c3c32640 1 --2- 172.21.15.55:0/1803029304 >> v2:172.21.15.138:3300/0 conn(0x7f66bc0b3c90 0x7f66bc0b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.713+0000 7f66c3c32640 1 --2- 172.21.15.55:0/1803029304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0d3390 0x7f66bc06c910 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.713+0000 7f66c3c32640 1 -- 172.21.15.55:0/1803029304 >> 172.21.15.55:0/1803029304 conn(0x7f66bc0b65f0 msgr2=0x7f66bc0b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:50.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.713+0000 7f66c3c32640 1 -- 172.21.15.55:0/1803029304 shutdown_connections 2024-05-20T22:07:50.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.713+0000 7f66c3c32640 1 -- 172.21.15.55:0/1803029304 wait complete. 2024-05-20T22:07:50.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.715+0000 7f66c3c32640 1 Processor -- start 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.715+0000 7f66c3c32640 1 -- start start 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c3c32640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0b3c90 0x7f66bc1f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c3c32640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66bc0b8550 0x7f66bc1e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c3c32640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66bc0d3390 0x7f66bc1e9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c3c32640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc06b500 con 0x7f66bc0b3c90 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c3c32640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc06c3f0 con 0x7f66bc0d3390 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c3c32640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f66bc0b40a0 con 0x7f66bc0b8550 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c19ce640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0b3c90 0x7f66bc1f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c19ce640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0b3c90 0x7f66bc1f95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38420/0 (socket says 172.21.15.55:38420) 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c19ce640 1 -- 172.21.15.55:0/3955507434 learned_addr learned my addr 172.21.15.55:0/3955507434 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:50.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c21cf640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66bc0d3390 0x7f66bc1e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c11cd640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66bc0b8550 0x7f66bc1e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c19ce640 1 -- 172.21.15.55:0/3955507434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66bc0b8550 msgr2=0x7f66bc1e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c19ce640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66bc0b8550 0x7f66bc1e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c19ce640 1 -- 172.21.15.55:0/3955507434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66bc0d3390 msgr2=0x7f66bc1e9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.716+0000 7f66c19ce640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66bc0d3390 0x7f66bc1e9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.717+0000 7f66c19ce640 1 -- 172.21.15.55:0/3955507434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc1ea680 con 0x7f66bc0b3c90 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.717+0000 7f66c19ce640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0b3c90 0x7f66bc1f95e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66b802a180 tx=0x7f66b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.717+0000 7f66b27fc640 1 -- 172.21.15.55:0/3955507434 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8041020 con 0x7f66bc0b3c90 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.717+0000 7f66b27fc640 1 -- 172.21.15.55:0/3955507434 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b80306c0 con 0x7f66bc0b3c90 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.717+0000 7f66b27fc640 1 -- 172.21.15.55:0/3955507434 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8037040 con 0x7f66bc0b3c90 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.717+0000 7f66c3c32640 1 -- 172.21.15.55:0/3955507434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc1ea920 con 0x7f66bc0b3c90 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.717+0000 7f66c3c32640 1 -- 172.21.15.55:0/3955507434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc1eaee0 con 0x7f66bc0b3c90 2024-05-20T22:07:50.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.718+0000 7f66c3c32640 1 -- 172.21.15.55:0/3955507434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc069090 con 0x7f66bc0b3c90 2024-05-20T22:07:50.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.719+0000 7f66b27fc640 1 -- 172.21.15.55:0/3955507434 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66b8044020 con 0x7f66bc0b3c90 2024-05-20T22:07:50.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.719+0000 7f66b27fc640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f66a006be30 0x7f66a006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:50.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.719+0000 7f66b27fc640 1 -- 172.21.15.55:0/3955507434 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f66b80ab2c0 con 0x7f66bc0b3c90 2024-05-20T22:07:50.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.719+0000 7f66c11cd640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f66a006be30 0x7f66a006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:50.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.720+0000 7f66c11cd640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f66a006be30 0x7f66a006e340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0c85d0 tx=0x7f66ac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:50.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.724+0000 7f66b27fc640 1 -- 172.21.15.55:0/3955507434 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f66b804b030 con 0x7f66bc0b3c90 2024-05-20T22:07:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.910+0000 7f66c3c32640 1 -- 172.21.15.55:0/3955507434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f66bc0d2eb0 con 0x7f66bc0b3c90 2024-05-20T22:07:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.910+0000 7f66b27fc640 1 -- 172.21.15.55:0/3955507434 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f66b80798e0 con 0x7f66bc0b3c90 2024-05-20T22:07:50.911 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:07:50.911 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-05-20T22:07:50.911 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:07:50.911 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-05-20T22:07:50.911 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-05-20T22:07:50.912 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:07:50.912 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-05-20T22:07:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 -- 172.21.15.55:0/3955507434 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f66a006be30 msgr2=0x7f66a006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f66a006be30 0x7f66a006e340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0c85d0 tx=0x7f66ac008040 comp rx=0 tx=0).stop 2024-05-20T22:07:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 -- 172.21.15.55:0/3955507434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0b3c90 msgr2=0x7f66bc1f95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0b3c90 0x7f66bc1f95e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66b802a180 tx=0x7f66b80046c0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 -- 172.21.15.55:0/3955507434 shutdown_connections 2024-05-20T22:07:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f66a006be30 0x7f66a006e340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66bc0d3390 0x7f66bc1e9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66bc0b8550 0x7f66bc1e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 --2- 172.21.15.55:0/3955507434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66bc0b3c90 0x7f66bc1f95e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 -- 172.21.15.55:0/3955507434 >> 172.21.15.55:0/3955507434 conn(0x7f66bc0b65f0 msgr2=0x7f66bc06e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 -- 172.21.15.55:0/3955507434 shutdown_connections 2024-05-20T22:07:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:50.912+0000 7f66c3c32640 1 -- 172.21.15.55:0/3955507434 wait complete. 2024-05-20T22:07:50.924 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T22:07:50.924 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T22:07:51.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.042+0000 7fb455b4b640 1 Processor -- start 2024-05-20T22:07:51.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.042+0000 7fb455b4b640 1 -- start start 2024-05-20T22:07:51.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.043+0000 7fb455b4b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4500b1eb0 0x7fb45006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.043+0000 7fb455b4b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4500b37b0 con 0x7fb4500cfad0 2024-05-20T22:07:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.043+0000 7fb455b4b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb45006d8b0 con 0x7fb4500b2470 2024-05-20T22:07:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.043+0000 7fb455b4b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb4500b2880 con 0x7fb4500b1eb0 2024-05-20T22:07:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.043+0000 7fb44effd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb4500cfad0 0x7fb4500d0b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35724/0 (socket says 172.21.15.55:35724) 2024-05-20T22:07:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.043+0000 7fb44effd640 1 -- 172.21.15.55:0/2503572818 learned_addr learned my addr 172.21.15.55:0/2503572818 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.043+0000 7fb44ffff640 1 --2- 172.21.15.55:0/2503572818 >> v2:172.21.15.138:3300/0 conn(0x7fb4500b1eb0 0x7fb45006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.043+0000 7fb44e7fc640 1 -- 172.21.15.55:0/2503572818 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839961699 0 0) 0x7fb438001110 con 0x7fb4500cfad0 2024-05-20T22:07:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.043+0000 7fb44e7fc640 1 -- 172.21.15.55:0/2503572818 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb43c003520 con 0x7fb4500cfad0 2024-05-20T22:07:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.044+0000 7fb44ffff640 1 -- 172.21.15.55:0/2503572818 >> v1:172.21.15.138:6789/0 conn(0x7fb4500b2470 legacy=0x7fb4500cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.044+0000 7fb44ffff640 1 -- 172.21.15.55:0/2503572818 >> v1:172.21.15.55:6789/0 conn(0x7fb4500cfad0 legacy=0x7fb4500d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.044+0000 7fb44ffff640 1 -- 172.21.15.55:0/2503572818 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45006db90 con 0x7fb4500b1eb0 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.044+0000 7fb44ffff640 1 --2- 172.21.15.55:0/2503572818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b1eb0 0x7fb45006d0b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb4480097a0 tx=0x7fb44802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e33f27ad0196930 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.045+0000 7fb44e7fc640 1 -- 172.21.15.55:0/2503572818 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448037070 con 0x7fb4500b1eb0 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.045+0000 7fb44e7fc640 1 -- 172.21.15.55:0/2503572818 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb448032440 con 0x7fb4500b1eb0 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.045+0000 7fb44e7fc640 1 -- 172.21.15.55:0/2503572818 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448032810 con 0x7fb4500b1eb0 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.045+0000 7fb455b4b640 1 -- 172.21.15.55:0/2503572818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b1eb0 msgr2=0x7fb45006d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.045+0000 7fb455b4b640 1 --2- 172.21.15.55:0/2503572818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b1eb0 0x7fb45006d0b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb4480097a0 tx=0x7fb44802a230 comp rx=0 tx=0).stop 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.045+0000 7fb455b4b640 1 -- 172.21.15.55:0/2503572818 shutdown_connections 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.045+0000 7fb455b4b640 1 --2- 172.21.15.55:0/2503572818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b1eb0 0x7fb45006d0b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.045+0000 7fb455b4b640 1 -- 172.21.15.55:0/2503572818 >> 172.21.15.55:0/2503572818 conn(0x7fb45009be10 msgr2=0x7fb4500d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.045+0000 7fb455b4b640 1 -- 172.21.15.55:0/2503572818 shutdown_connections 2024-05-20T22:07:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.045+0000 7fb455b4b640 1 -- 172.21.15.55:0/2503572818 wait complete. 2024-05-20T22:07:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.047+0000 7fb455b4b640 1 Processor -- start 2024-05-20T22:07:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.047+0000 7fb455b4b640 1 -- start start 2024-05-20T22:07:51.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb455b4b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4500b1eb0 0x7fb45009f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb455b4b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b2470 0x7fb450151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb455b4b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb45009d2b0 0x7fb45009d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb455b4b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb4500b3a30 con 0x7fb4500b1eb0 2024-05-20T22:07:51.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb455b4b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4500b3f50 con 0x7fb4500b2470 2024-05-20T22:07:51.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb455b4b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4501625a0 con 0x7fb45009d2b0 2024-05-20T22:07:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4500b1eb0 0x7fb45009f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4500b1eb0 0x7fb45009f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38426/0 (socket says 172.21.15.55:38426) 2024-05-20T22:07:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44f7fe640 1 -- 172.21.15.55:0/3952497451 learned_addr learned my addr 172.21.15.55:0/3952497451 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44effd640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b2470 0x7fb450151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44ffff640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb45009d2b0 0x7fb45009d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44f7fe640 1 -- 172.21.15.55:0/3952497451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb45009d2b0 msgr2=0x7fb45009d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44f7fe640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb45009d2b0 0x7fb45009d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44f7fe640 1 -- 172.21.15.55:0/3952497451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b2470 msgr2=0x7fb450151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44f7fe640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b2470 0x7fb450151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44f7fe640 1 -- 172.21.15.55:0/3952497451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb450152510 con 0x7fb4500b1eb0 2024-05-20T22:07:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb44f7fe640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4500b1eb0 0x7fb45009f220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb44000c6e0 tx=0x7fb44000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.049+0000 7fb447fff640 1 -- 172.21.15.55:0/3952497451 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440016020 con 0x7fb4500b1eb0 2024-05-20T22:07:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.050+0000 7fb447fff640 1 -- 172.21.15.55:0/3952497451 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb440006d50 con 0x7fb4500b1eb0 2024-05-20T22:07:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.050+0000 7fb447fff640 1 -- 172.21.15.55:0/3952497451 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440003bb0 con 0x7fb4500b1eb0 2024-05-20T22:07:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.050+0000 7fb455b4b640 1 -- 172.21.15.55:0/3952497451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45009dc00 con 0x7fb4500b1eb0 2024-05-20T22:07:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.050+0000 7fb455b4b640 1 -- 172.21.15.55:0/3952497451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45009fee0 con 0x7fb4500b1eb0 2024-05-20T22:07:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.050+0000 7fb455b4b640 1 -- 172.21.15.55:0/3952497451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb450069090 con 0x7fb4500b1eb0 2024-05-20T22:07:51.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.051+0000 7fb447fff640 1 -- 172.21.15.55:0/3952497451 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4400050d0 con 0x7fb4500b1eb0 2024-05-20T22:07:51.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.052+0000 7fb447fff640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb43006be30 0x7fb43006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.052+0000 7fb44effd640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb43006be30 0x7fb43006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.052+0000 7fb447fff640 1 -- 172.21.15.55:0/3952497451 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb44008a780 con 0x7fb4500b1eb0 2024-05-20T22:07:51.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.052+0000 7fb44effd640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb43006be30 0x7fb43006e340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb438007fd0 tx=0x7fb438009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:51.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.056+0000 7fb447fff640 1 -- 172.21.15.55:0/3952497451 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb440020020 con 0x7fb4500b1eb0 2024-05-20T22:07:51.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.244+0000 7fb455b4b640 1 -- 172.21.15.55:0/3952497451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb45008ccf0 con 0x7fb4500b1eb0 2024-05-20T22:07:51.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.632+0000 7fb447fff640 1 -- 172.21.15.55:0/3952497451 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fb45009dc00 con 0x7fb4500b1eb0 2024-05-20T22:07:51.633 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-05-20T22:07:51.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.635+0000 7fb455b4b640 1 -- 172.21.15.55:0/3952497451 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb43006be30 msgr2=0x7fb43006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:51.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.635+0000 7fb455b4b640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb43006be30 0x7fb43006e340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb438007fd0 tx=0x7fb438009040 comp rx=0 tx=0).stop 2024-05-20T22:07:51.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.635+0000 7fb455b4b640 1 -- 172.21.15.55:0/3952497451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4500b1eb0 msgr2=0x7fb45009f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:51.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.635+0000 7fb455b4b640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4500b1eb0 0x7fb45009f220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb44000c6e0 tx=0x7fb44000cba0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.635+0000 7fb455b4b640 1 -- 172.21.15.55:0/3952497451 shutdown_connections 2024-05-20T22:07:51.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.635+0000 7fb455b4b640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb43006be30 0x7fb43006e340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.635+0000 7fb455b4b640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb45009d2b0 0x7fb45009d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.635+0000 7fb455b4b640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4500b2470 0x7fb450151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.635+0000 7fb455b4b640 1 --2- 172.21.15.55:0/3952497451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4500b1eb0 0x7fb45009f220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.635+0000 7fb455b4b640 1 -- 172.21.15.55:0/3952497451 >> 172.21.15.55:0/3952497451 conn(0x7fb45009be10 msgr2=0x7fb4500cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:51.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.636+0000 7fb455b4b640 1 -- 172.21.15.55:0/3952497451 shutdown_connections 2024-05-20T22:07:51.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.636+0000 7fb455b4b640 1 -- 172.21.15.55:0/3952497451 wait complete. 2024-05-20T22:07:51.654 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-20T22:07:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.777+0000 7fc4a0079640 1 Processor -- start 2024-05-20T22:07:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.778+0000 7fc4a0079640 1 -- start start 2024-05-20T22:07:51.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc4a0079640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 0x7fc49815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc4a0079640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4980b35e0 0x7fc49815f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc4a0079640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc49815fd60 0x7fc4980cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc4a0079640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fc498163960 con 0x7fc49815fd60 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc4a0079640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc49806b500 con 0x7fc4980b35e0 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc4a0079640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4980b3e10 con 0x7fc4980b4080 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc49e616640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc49815fd60 0x7fc4980cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc49e616640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc49815fd60 0x7fc4980cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38440/0 (socket says 172.21.15.55:38440) 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc49e616640 1 -- 172.21.15.55:0/3015376993 learned_addr learned my addr 172.21.15.55:0/3015376993 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc49d614640 1 --2- 172.21.15.55:0/3015376993 >> v2:172.21.15.138:3300/0 conn(0x7fc4980b35e0 0x7fc49815f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc49de15640 1 --2- 172.21.15.55:0/3015376993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 0x7fc49815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc49e616640 1 -- 172.21.15.55:0/3015376993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 msgr2=0x7fc49815f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc49e616640 1 --2- 172.21.15.55:0/3015376993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 0x7fc49815f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc49e616640 1 -- 172.21.15.55:0/3015376993 >> v2:172.21.15.138:3300/0 conn(0x7fc4980b35e0 msgr2=0x7fc49815f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc49e616640 1 --2- 172.21.15.55:0/3015376993 >> v2:172.21.15.138:3300/0 conn(0x7fc4980b35e0 0x7fc49815f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.779+0000 7fc49e616640 1 -- 172.21.15.55:0/3015376993 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4980b3b30 con 0x7fc49815fd60 2024-05-20T22:07:51.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.780+0000 7fc49e616640 1 --2- 172.21.15.55:0/3015376993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 0x7fc4980cf620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc49806eb90 tx=0x7fc49402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73093643f77039e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.780+0000 7fc49ce13640 1 -- 172.21.15.55:0/3015376993 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494037070 con 0x7fc49815fd60 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.780+0000 7fc49ce13640 1 -- 172.21.15.55:0/3015376993 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc494032440 con 0x7fc49815fd60 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.780+0000 7fc49ce13640 1 -- 172.21.15.55:0/3015376993 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494032810 con 0x7fc49815fd60 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.781+0000 7fc4a0079640 1 -- 172.21.15.55:0/3015376993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 msgr2=0x7fc4980cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.781+0000 7fc4a0079640 1 --2- 172.21.15.55:0/3015376993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 0x7fc4980cf620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc49806eb90 tx=0x7fc49402a1e0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.781+0000 7fc4a0079640 1 -- 172.21.15.55:0/3015376993 shutdown_connections 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.781+0000 7fc4a0079640 1 --2- 172.21.15.55:0/3015376993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 0x7fc4980cf620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.781+0000 7fc4a0079640 1 --2- 172.21.15.55:0/3015376993 >> v2:172.21.15.138:3300/0 conn(0x7fc4980b35e0 0x7fc49815f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.781+0000 7fc4a0079640 1 --2- 172.21.15.55:0/3015376993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 0x7fc49815f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.781+0000 7fc4a0079640 1 -- 172.21.15.55:0/3015376993 >> 172.21.15.55:0/3015376993 conn(0x7fc4980d51b0 msgr2=0x7fc4980b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.781+0000 7fc4a0079640 1 -- 172.21.15.55:0/3015376993 shutdown_connections 2024-05-20T22:07:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.781+0000 7fc4a0079640 1 -- 172.21.15.55:0/3015376993 wait complete. 2024-05-20T22:07:51.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.783+0000 7fc4a0079640 1 Processor -- start 2024-05-20T22:07:51.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.783+0000 7fc4a0079640 1 -- start start 2024-05-20T22:07:51.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.784+0000 7fc4a0079640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4980b35e0 0x7fc4980ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.784+0000 7fc4a0079640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 0x7fc4980cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.784+0000 7fc4a0079640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 0x7fc4981f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.784+0000 7fc4a0079640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981635f0 con 0x7fc49815fd60 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.784+0000 7fc4a0079640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981604f0 con 0x7fc4980b35e0 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.784+0000 7fc4a0079640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4980cfb60 con 0x7fc4980b4080 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.784+0000 7fc49e616640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 0x7fc4981f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.784+0000 7fc49e616640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 0x7fc4981f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38452/0 (socket says 172.21.15.55:38452) 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.784+0000 7fc49e616640 1 -- 172.21.15.55:0/2674841848 learned_addr learned my addr 172.21.15.55:0/2674841848 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc49de15640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4980b35e0 0x7fc4980ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc49d614640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 0x7fc4980cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc49e616640 1 -- 172.21.15.55:0/2674841848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 msgr2=0x7fc4980cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc49e616640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 0x7fc4980cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc49e616640 1 -- 172.21.15.55:0/2674841848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4980b35e0 msgr2=0x7fc4980ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc49e616640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4980b35e0 0x7fc4980ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:51.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc49e616640 1 -- 172.21.15.55:0/2674841848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4980c88a0 con 0x7fc49815fd60 2024-05-20T22:07:51.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc49d614640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 0x7fc4980cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:07:51.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc49e616640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 0x7fc4981f50a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc4940090b0 tx=0x7fc4940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:51.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc48e7fc640 1 -- 172.21.15.55:0/2674841848 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494037040 con 0x7fc49815fd60 2024-05-20T22:07:51.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc48e7fc640 1 -- 172.21.15.55:0/2674841848 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc494042070 con 0x7fc49815fd60 2024-05-20T22:07:51.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc4a0079640 1 -- 172.21.15.55:0/2674841848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4980c8b80 con 0x7fc49815fd60 2024-05-20T22:07:51.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc48e7fc640 1 -- 172.21.15.55:0/2674841848 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494030920 con 0x7fc49815fd60 2024-05-20T22:07:51.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.785+0000 7fc4a0079640 1 -- 172.21.15.55:0/2674841848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4980c9140 con 0x7fc49815fd60 2024-05-20T22:07:51.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.786+0000 7fc4a0079640 1 -- 172.21.15.55:0/2674841848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc498161eb0 con 0x7fc49815fd60 2024-05-20T22:07:51.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.787+0000 7fc48e7fc640 1 -- 172.21.15.55:0/2674841848 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc494030ac0 con 0x7fc49815fd60 2024-05-20T22:07:51.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.787+0000 7fc48e7fc640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc47c06be30 0x7fc47c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:51.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.787+0000 7fc48e7fc640 1 -- 172.21.15.55:0/2674841848 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fc4940ab5c0 con 0x7fc49815fd60 2024-05-20T22:07:51.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.787+0000 7fc49de15640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc47c06be30 0x7fc47c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:51.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.788+0000 7fc49de15640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc47c06be30 0x7fc47c06e340 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc4980d0980 tx=0x7fc490003990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:51.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.792+0000 7fc48e7fc640 1 -- 172.21.15.55:0/2674841848 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc498161eb0 con 0x7fc49815fd60 2024-05-20T22:07:51.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:51.977+0000 7fc4a0079640 1 -- 172.21.15.55:0/2674841848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fc498069090 con 0x7fc49815fd60 2024-05-20T22:07:52.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.638+0000 7fc48e7fc640 1 -- 172.21.15.55:0/2674841848 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7fc494079a60 con 0x7fc49815fd60 2024-05-20T22:07:52.639 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-05-20T22:07:52.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 -- 172.21.15.55:0/2674841848 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc47c06be30 msgr2=0x7fc47c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:52.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc47c06be30 0x7fc47c06e340 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc4980d0980 tx=0x7fc490003990 comp rx=0 tx=0).stop 2024-05-20T22:07:52.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 -- 172.21.15.55:0/2674841848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 msgr2=0x7fc4981f50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:52.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 0x7fc4981f50a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc4940090b0 tx=0x7fc4940046c0 comp rx=0 tx=0).stop 2024-05-20T22:07:52.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 -- 172.21.15.55:0/2674841848 shutdown_connections 2024-05-20T22:07:52.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc47c06be30 0x7fc47c06e340 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:52.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc49815fd60 0x7fc4981f50a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:52.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4980b4080 0x7fc4980cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:52.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 --2- 172.21.15.55:0/2674841848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4980b35e0 0x7fc4980ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:52.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 -- 172.21.15.55:0/2674841848 >> 172.21.15.55:0/2674841848 conn(0x7fc4980d51b0 msgr2=0x7fc4980ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:52.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 -- 172.21.15.55:0/2674841848 shutdown_connections 2024-05-20T22:07:52.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.641+0000 7fc4a0079640 1 -- 172.21.15.55:0/2674841848 wait complete. 2024-05-20T22:07:52.670 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T22:07:52.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.792+0000 7f0d7fd4c640 1 Processor -- start 2024-05-20T22:07:52.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.792+0000 7f0d7fd4c640 1 -- start start 2024-05-20T22:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.793+0000 7f0d7fd4c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d780cc4c0 0x7f0d780d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.793+0000 7f0d7fd4c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0d780ce860 0x7f0d780d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.793+0000 7f0d7fd4c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d7815f610 con 0x7f0d780b2980 2024-05-20T22:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.793+0000 7f0d7fd4c640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f0d780cf040 con 0x7f0d780ce860 2024-05-20T22:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.793+0000 7f0d7fd4c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0d7809b160 con 0x7f0d780cc4c0 2024-05-20T22:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.793+0000 7f0d7e2e9640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0d780ce860 0x7f0d780d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.793+0000 7f0d7e2e9640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0d780ce860 0x7f0d780d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38468/0 (socket says 172.21.15.55:38468) 2024-05-20T22:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.794+0000 7f0d7e2e9640 1 -- 172.21.15.55:0/34750381 learned_addr learned my addr 172.21.15.55:0/34750381 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.794+0000 7f0d7d2e7640 1 --2- 172.21.15.55:0/34750381 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d780cc4c0 0x7f0d780d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.794+0000 7f0d7e2e9640 1 -- 172.21.15.55:0/34750381 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d780cc4c0 msgr2=0x7f0d780d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:52.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.794+0000 7f0d7e2e9640 1 --2- 172.21.15.55:0/34750381 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d780cc4c0 0x7f0d780d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:52.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.794+0000 7f0d7e2e9640 1 -- 172.21.15.55:0/34750381 >> v1:172.21.15.138:6789/0 conn(0x7f0d780b2980 legacy=0x7f0d780cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:52.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.794+0000 7f0d7e2e9640 1 -- 172.21.15.55:0/34750381 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d780ced60 con 0x7f0d780ce860 2024-05-20T22:07:52.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.795+0000 7f0d7e2e9640 1 --2- 172.21.15.55:0/34750381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 0x7f0d780d1140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009e50 tx=0x7f0d7002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f36c41d0c31314b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:52.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.795+0000 7f0d7cae6640 1 -- 172.21.15.55:0/34750381 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70036070 con 0x7f0d780ce860 2024-05-20T22:07:52.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.795+0000 7f0d7cae6640 1 -- 172.21.15.55:0/34750381 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d70031600 con 0x7f0d780ce860 2024-05-20T22:07:52.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.795+0000 7f0d7cae6640 1 -- 172.21.15.55:0/34750381 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700319d0 con 0x7f0d780ce860 2024-05-20T22:07:52.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.795+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/34750381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 msgr2=0x7f0d780d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:52.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.795+0000 7f0d7fd4c640 1 --2- 172.21.15.55:0/34750381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 0x7f0d780d1140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009e50 tx=0x7f0d7002a680 comp rx=0 tx=0).stop 2024-05-20T22:07:52.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.795+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/34750381 shutdown_connections 2024-05-20T22:07:52.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.795+0000 7f0d7fd4c640 1 --2- 172.21.15.55:0/34750381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 0x7f0d780d1140 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:52.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.795+0000 7f0d7fd4c640 1 --2- 172.21.15.55:0/34750381 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d780cc4c0 0x7f0d780d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:52.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.796+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/34750381 >> 172.21.15.55:0/34750381 conn(0x7f0d780736f0 msgr2=0x7f0d780b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:52.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.796+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/34750381 shutdown_connections 2024-05-20T22:07:52.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.796+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/34750381 wait complete. 2024-05-20T22:07:52.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.797+0000 7f0d7fd4c640 1 Processor -- start 2024-05-20T22:07:52.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.797+0000 7f0d7fd4c640 1 -- start start 2024-05-20T22:07:52.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7fd4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d780b2980 0x7f0d7815c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:52.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7fd4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 0x7f0d781f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:52.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7fd4c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d7806c960 0x7f0d780cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:52.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7fd4c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0d780cd3e0 con 0x7f0d780ce860 2024-05-20T22:07:52.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7fd4c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7809adf0 con 0x7f0d780b2980 2024-05-20T22:07:52.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7fd4c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0d780b2d90 con 0x7f0d7806c960 2024-05-20T22:07:52.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7d2e7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 0x7f0d781f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:52.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7d2e7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 0x7f0d781f4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38476/0 (socket says 172.21.15.55:38476) 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7d2e7640 1 -- 172.21.15.55:0/2166888261 learned_addr learned my addr 172.21.15.55:0/2166888261 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7dae8640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d780b2980 0x7f0d7815c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7e2e9640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d7806c960 0x7f0d780cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7d2e7640 1 -- 172.21.15.55:0/2166888261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d7806c960 msgr2=0x7f0d780cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7d2e7640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d7806c960 0x7f0d780cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7d2e7640 1 -- 172.21.15.55:0/2166888261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d780b2980 msgr2=0x7f0d7815c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7d2e7640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d780b2980 0x7f0d7815c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7d2e7640 1 -- 172.21.15.55:0/2166888261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d78160530 con 0x7f0d780ce860 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d7d2e7640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 0x7f0d781f4cd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800e870 tx=0x7f0d6800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.799+0000 7f0d6e7fc640 1 -- 172.21.15.55:0/2166888261 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68017070 con 0x7f0d780ce860 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.800+0000 7f0d6e7fc640 1 -- 172.21.15.55:0/2166888261 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6800c940 con 0x7f0d780ce860 2024-05-20T22:07:52.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.800+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/2166888261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7806d0c0 con 0x7f0d780ce860 2024-05-20T22:07:52.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.800+0000 7f0d6e7fc640 1 -- 172.21.15.55:0/2166888261 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800cd30 con 0x7f0d780ce860 2024-05-20T22:07:52.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.800+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/2166888261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7806d680 con 0x7f0d780ce860 2024-05-20T22:07:52.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.800+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/2166888261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7806ce30 con 0x7f0d780ce860 2024-05-20T22:07:52.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.801+0000 7f0d6e7fc640 1 -- 172.21.15.55:0/2166888261 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d6801a070 con 0x7f0d780ce860 2024-05-20T22:07:52.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.802+0000 7f0d6e7fc640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0d5c06be60 0x7f0d5c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:52.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.802+0000 7f0d7dae8640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0d5c06be60 0x7f0d5c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:52.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.802+0000 7f0d6e7fc640 1 -- 172.21.15.55:0/2166888261 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f0d6808ac90 con 0x7f0d780ce860 2024-05-20T22:07:52.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.802+0000 7f0d7dae8640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0d5c06be60 0x7f0d5c06e370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d740016e0 tx=0x7f0d74009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:52.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:52.806+0000 7f0d6e7fc640 1 -- 172.21.15.55:0/2166888261 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0d68058fc0 con 0x7f0d780ce860 2024-05-20T22:07:53.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.027+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/2166888261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0d78069090 con 0x7f0d780ce860 2024-05-20T22:07:53.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.640+0000 7f0d6e7fc640 1 -- 172.21.15.55:0/2166888261 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f0d6805d2c0 con 0x7f0d780ce860 2024-05-20T22:07:53.641 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 2 and data pool 3 2024-05-20T22:07:53.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/2166888261 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0d5c06be60 msgr2=0x7f0d5c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:53.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0d5c06be60 0x7f0d5c06e370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d740016e0 tx=0x7f0d74009040 comp rx=0 tx=0).stop 2024-05-20T22:07:53.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/2166888261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 msgr2=0x7f0d781f4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:53.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 0x7f0d781f4cd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800e870 tx=0x7f0d6800ed30 comp rx=0 tx=0).stop 2024-05-20T22:07:53.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/2166888261 shutdown_connections 2024-05-20T22:07:53.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0d5c06be60 0x7f0d5c06e370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:53.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d7806c960 0x7f0d780cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:53.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d780ce860 0x7f0d781f4cd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:53.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 --2- 172.21.15.55:0/2166888261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d780b2980 0x7f0d7815c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:53.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/2166888261 >> 172.21.15.55:0/2166888261 conn(0x7f0d780736f0 msgr2=0x7f0d7815eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:53.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/2166888261 shutdown_connections 2024-05-20T22:07:53.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.643+0000 7f0d7fd4c640 1 -- 172.21.15.55:0/2166888261 wait complete. 2024-05-20T22:07:53.670 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-20T22:07:53.671 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-20T22:07:53.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.799+0000 7f1cae31a640 1 Processor -- start 2024-05-20T22:07:53.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.799+0000 7f1cae31a640 1 -- start start 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.800+0000 7f1cae31a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ca806e0c0 0x7f1ca80ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.800+0000 7f1cae31a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ca80d1ca0 0x7f1ca815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.800+0000 7f1cae31a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca809a260 con 0x7f1ca815cb90 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.800+0000 7f1cae31a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca809a540 con 0x7f1ca80d1ca0 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.800+0000 7f1cae31a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca809a820 con 0x7f1ca806e0c0 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.800+0000 7f1cac8b7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ca80d1ca0 0x7f1ca815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.800+0000 7f1ca77fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1ca815cb90 0x7f1ca80d1590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35730/0 (socket says 172.21.15.55:35730) 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.800+0000 7f1cac8b7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ca80d1ca0 0x7f1ca815cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38490/0 (socket says 172.21.15.55:38490) 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.800+0000 7f1ca77fe640 1 -- 172.21.15.55:0/1110623515 learned_addr learned my addr 172.21.15.55:0/1110623515 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.800+0000 7f1ca7fff640 1 --2- 172.21.15.55:0/1110623515 >> v2:172.21.15.138:3300/0 conn(0x7f1ca806e0c0 0x7f1ca80ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.801+0000 7f1cac8b7640 1 -- 172.21.15.55:0/1110623515 >> v2:172.21.15.138:3300/0 conn(0x7f1ca806e0c0 msgr2=0x7f1ca80ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.801+0000 7f1cac8b7640 1 --2- 172.21.15.55:0/1110623515 >> v2:172.21.15.138:3300/0 conn(0x7f1ca806e0c0 0x7f1ca80ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:53.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.801+0000 7f1cac8b7640 1 -- 172.21.15.55:0/1110623515 >> v1:172.21.15.55:6789/0 conn(0x7f1ca815cb90 legacy=0x7f1ca80d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:53.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.801+0000 7f1ca6ffd640 1 -- 172.21.15.55:0/1110623515 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296611941 0 0) 0x7f1ca809a260 con 0x7f1ca815cb90 2024-05-20T22:07:53.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.801+0000 7f1cac8b7640 1 -- 172.21.15.55:0/1110623515 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca815c670 con 0x7f1ca80d1ca0 2024-05-20T22:07:53.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.801+0000 7f1ca7fff640 1 --2- 172.21.15.55:0/1110623515 >> v2:172.21.15.138:3300/0 conn(0x7f1ca806e0c0 0x7f1ca80ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:07:53.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.801+0000 7f1cac8b7640 1 --2- 172.21.15.55:0/1110623515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca80d1ca0 0x7f1ca815cfa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80ce140 tx=0x7f1ca002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963677f24b222939 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:53.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1ca6ffd640 1 -- 172.21.15.55:0/1110623515 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0007740 con 0x7f1ca80d1ca0 2024-05-20T22:07:53.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1ca6ffd640 1 -- 172.21.15.55:0/1110623515 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca00078e0 con 0x7f1ca80d1ca0 2024-05-20T22:07:53.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1ca6ffd640 1 -- 172.21.15.55:0/1110623515 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0007cb0 con 0x7f1ca80d1ca0 2024-05-20T22:07:53.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1cae31a640 1 -- 172.21.15.55:0/1110623515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca80d1ca0 msgr2=0x7f1ca815cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:53.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1cae31a640 1 --2- 172.21.15.55:0/1110623515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca80d1ca0 0x7f1ca815cfa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80ce140 tx=0x7f1ca002a680 comp rx=0 tx=0).stop 2024-05-20T22:07:53.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1cae31a640 1 -- 172.21.15.55:0/1110623515 shutdown_connections 2024-05-20T22:07:53.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1cae31a640 1 --2- 172.21.15.55:0/1110623515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca80d1ca0 0x7f1ca815cfa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:53.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1cae31a640 1 --2- 172.21.15.55:0/1110623515 >> v2:172.21.15.138:3300/0 conn(0x7f1ca806e0c0 0x7f1ca80ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:53.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1cae31a640 1 -- 172.21.15.55:0/1110623515 >> 172.21.15.55:0/1110623515 conn(0x7f1ca80736f0 msgr2=0x7f1ca80b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:53.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1cae31a640 1 -- 172.21.15.55:0/1110623515 shutdown_connections 2024-05-20T22:07:53.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.802+0000 7f1cae31a640 1 -- 172.21.15.55:0/1110623515 wait complete. 2024-05-20T22:07:53.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.804+0000 7f1cae31a640 1 Processor -- start 2024-05-20T22:07:53.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.804+0000 7f1cae31a640 1 -- start start 2024-05-20T22:07:53.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1cae31a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca806e0c0 0x7f1ca80c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:53.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1cae31a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca815cb90 0x7f1ca8155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1cae31a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ca8155b30 0x7f1ca80b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1cae31a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8156770 con 0x7f1ca815cb90 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1cae31a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca815c360 con 0x7f1ca806e0c0 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1cae31a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca80cf330 con 0x7f1ca8155b30 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca815cb90 0x7f1ca8155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca815cb90 0x7f1ca8155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38492/0 (socket says 172.21.15.55:38492) 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca77fe640 1 -- 172.21.15.55:0/2862385463 learned_addr learned my addr 172.21.15.55:0/2862385463 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca7fff640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca806e0c0 0x7f1ca80c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1cac8b7640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ca8155b30 0x7f1ca80b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca77fe640 1 -- 172.21.15.55:0/2862385463 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ca8155b30 msgr2=0x7f1ca80b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca77fe640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ca8155b30 0x7f1ca80b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca77fe640 1 -- 172.21.15.55:0/2862385463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca806e0c0 msgr2=0x7f1ca80c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:53.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca77fe640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca806e0c0 0x7f1ca80c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:53.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca77fe640 1 -- 172.21.15.55:0/2862385463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca80c9600 con 0x7f1ca815cb90 2024-05-20T22:07:53.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca7fff640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca806e0c0 0x7f1ca80c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:07:53.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca77fe640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca815cb90 0x7f1ca8155410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c410 tx=0x7f1c9c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:53.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca4ff9640 1 -- 172.21.15.55:0/2862385463 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0074e0 con 0x7f1ca815cb90 2024-05-20T22:07:53.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca4ff9640 1 -- 172.21.15.55:0/2862385463 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9c0047e0 con 0x7f1ca815cb90 2024-05-20T22:07:53.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1cae31a640 1 -- 172.21.15.55:0/2862385463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca80c98e0 con 0x7f1ca815cb90 2024-05-20T22:07:53.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1ca4ff9640 1 -- 172.21.15.55:0/2862385463 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c004bd0 con 0x7f1ca815cb90 2024-05-20T22:07:53.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.806+0000 7f1cae31a640 1 -- 172.21.15.55:0/2862385463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca80c5920 con 0x7f1ca815cb90 2024-05-20T22:07:53.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.808+0000 7f1cae31a640 1 -- 172.21.15.55:0/2862385463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca80cb4a0 con 0x7f1ca815cb90 2024-05-20T22:07:53.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.808+0000 7f1ca4ff9640 1 -- 172.21.15.55:0/2862385463 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c9c011070 con 0x7f1ca815cb90 2024-05-20T22:07:53.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.808+0000 7f1ca4ff9640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c9406be60 0x7f1c9406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:53.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.808+0000 7f1ca4ff9640 1 -- 172.21.15.55:0/2862385463 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f1c9c08aa40 con 0x7f1ca815cb90 2024-05-20T22:07:53.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.808+0000 7f1ca7fff640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c9406be60 0x7f1c9406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:53.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.809+0000 7f1ca7fff640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c9406be60 0x7f1c9406e370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c98006fd0 tx=0x7f1c98008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:53.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:53.813+0000 7f1ca4ff9640 1 -- 172.21.15.55:0/2862385463 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c9c00e280 con 0x7f1ca815cb90 2024-05-20T22:07:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.006+0000 7f1cae31a640 1 -- 172.21.15.55:0/2862385463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f1ca8069090 con 0x7f1ca815cb90 2024-05-20T22:07:54.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.646+0000 7f1ca4ff9640 1 -- 172.21.15.55:0/2862385463 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f1c9c0593f0 con 0x7f1ca815cb90 2024-05-20T22:07:54.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.648+0000 7f1cae31a640 1 -- 172.21.15.55:0/2862385463 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c9406be60 msgr2=0x7f1c9406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:54.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.648+0000 7f1cae31a640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c9406be60 0x7f1c9406e370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c98006fd0 tx=0x7f1c98008040 comp rx=0 tx=0).stop 2024-05-20T22:07:54.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.648+0000 7f1cae31a640 1 -- 172.21.15.55:0/2862385463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca815cb90 msgr2=0x7f1ca8155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:54.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.648+0000 7f1cae31a640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca815cb90 0x7f1ca8155410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c410 tx=0x7f1c9c00c8d0 comp rx=0 tx=0).stop 2024-05-20T22:07:54.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.648+0000 7f1cae31a640 1 -- 172.21.15.55:0/2862385463 shutdown_connections 2024-05-20T22:07:54.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.648+0000 7f1cae31a640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c9406be60 0x7f1c9406e370 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:54.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.648+0000 7f1cae31a640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ca8155b30 0x7f1ca80b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:54.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.648+0000 7f1cae31a640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca815cb90 0x7f1ca8155410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:54.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.648+0000 7f1cae31a640 1 --2- 172.21.15.55:0/2862385463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca806e0c0 0x7f1ca80c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:54.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.648+0000 7f1cae31a640 1 -- 172.21.15.55:0/2862385463 >> 172.21.15.55:0/2862385463 conn(0x7f1ca80736f0 msgr2=0x7f1ca815e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:54.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.649+0000 7f1cae31a640 1 -- 172.21.15.55:0/2862385463 shutdown_connections 2024-05-20T22:07:54.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.649+0000 7f1cae31a640 1 -- 172.21.15.55:0/2862385463 wait complete. 2024-05-20T22:07:54.664 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T22:07:54.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.793+0000 7f65238f9640 1 Processor -- start 2024-05-20T22:07:54.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.793+0000 7f65238f9640 1 -- start start 2024-05-20T22:07:54.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.794+0000 7f65238f9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f651c15f090 0x7f651c0d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:54.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.794+0000 7f65238f9640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f651c0cf660 0x7f651c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:54.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f65238f9640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f651c0d3b60 con 0x7f651c0cffb0 2024-05-20T22:07:54.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f65238f9640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f651c06b630 con 0x7f651c0cf660 2024-05-20T22:07:54.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f65238f9640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f651c0d3e40 con 0x7f651c15f090 2024-05-20T22:07:54.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f6520e94640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f651c0cf660 0x7f651c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f6520e94640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f651c0cf660 0x7f651c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38498/0 (socket says 172.21.15.55:38498) 2024-05-20T22:07:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f6520e94640 1 -- 172.21.15.55:0/1572568313 learned_addr learned my addr 172.21.15.55:0/1572568313 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f6521e96640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f651c0cffb0 0x7f651c162b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35736/0 (socket says 172.21.15.55:35736) 2024-05-20T22:07:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f6521695640 1 --2- 172.21.15.55:0/1572568313 >> v2:172.21.15.138:3300/0 conn(0x7f651c15f090 0x7f651c0d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f6520e94640 1 -- 172.21.15.55:0/1572568313 >> v2:172.21.15.138:3300/0 conn(0x7f651c15f090 msgr2=0x7f651c0d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f6520e94640 1 --2- 172.21.15.55:0/1572568313 >> v2:172.21.15.138:3300/0 conn(0x7f651c15f090 0x7f651c0d3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f6520e94640 1 -- 172.21.15.55:0/1572568313 >> v1:172.21.15.55:6789/0 conn(0x7f651c0cffb0 legacy=0x7f651c162b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f6520e94640 1 -- 172.21.15.55:0/1572568313 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651c0d4120 con 0x7f651c0cf660 2024-05-20T22:07:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.795+0000 7f6513fff640 1 -- 172.21.15.55:0/1572568313 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 518866301 0 0) 0x7f651c0d3b60 con 0x7f651c0cffb0 2024-05-20T22:07:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.796+0000 7f6520e94640 1 --2- 172.21.15.55:0/1572568313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cf660 0x7f651c0cfa70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f651c08d520 tx=0x7f650c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e5eebfd1a40023 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.796+0000 7f6513fff640 1 -- 172.21.15.55:0/1572568313 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c037070 con 0x7f651c0cf660 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.796+0000 7f6513fff640 1 -- 172.21.15.55:0/1572568313 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650c032440 con 0x7f651c0cf660 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.796+0000 7f6513fff640 1 -- 172.21.15.55:0/1572568313 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c032810 con 0x7f651c0cf660 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.797+0000 7f65238f9640 1 -- 172.21.15.55:0/1572568313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cf660 msgr2=0x7f651c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.797+0000 7f65238f9640 1 --2- 172.21.15.55:0/1572568313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cf660 0x7f651c0cfa70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f651c08d520 tx=0x7f650c02a270 comp rx=0 tx=0).stop 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.797+0000 7f65238f9640 1 -- 172.21.15.55:0/1572568313 shutdown_connections 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.797+0000 7f65238f9640 1 --2- 172.21.15.55:0/1572568313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cf660 0x7f651c0cfa70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.797+0000 7f65238f9640 1 --2- 172.21.15.55:0/1572568313 >> v2:172.21.15.138:3300/0 conn(0x7f651c15f090 0x7f651c0d3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.797+0000 7f65238f9640 1 -- 172.21.15.55:0/1572568313 >> 172.21.15.55:0/1572568313 conn(0x7f651c0d53c0 msgr2=0x7f651c0b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.797+0000 7f65238f9640 1 -- 172.21.15.55:0/1572568313 shutdown_connections 2024-05-20T22:07:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.797+0000 7f65238f9640 1 -- 172.21.15.55:0/1572568313 wait complete. 2024-05-20T22:07:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.799+0000 7f65238f9640 1 Processor -- start 2024-05-20T22:07:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.799+0000 7f65238f9640 1 -- start start 2024-05-20T22:07:54.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f65238f9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0cf660 0x7f651c15c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:54.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f65238f9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cffb0 0x7f651c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:54.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f65238f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c15f090 0x7f651c162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:54.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f65238f9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f651c0d03c0 con 0x7f651c0cffb0 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f65238f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f651c0d3850 con 0x7f651c15f090 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f65238f9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f651c06cad0 con 0x7f651c0cf660 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6520e94640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cffb0 0x7f651c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6520e94640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cffb0 0x7f651c0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38500/0 (socket says 172.21.15.55:38500) 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6520e94640 1 -- 172.21.15.55:0/2806448066 learned_addr learned my addr 172.21.15.55:0/2806448066 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6521e96640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c15f090 0x7f651c162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6521695640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0cf660 0x7f651c15c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6520e94640 1 -- 172.21.15.55:0/2806448066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0cf660 msgr2=0x7f651c15c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6520e94640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0cf660 0x7f651c15c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6520e94640 1 -- 172.21.15.55:0/2806448066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c15f090 msgr2=0x7f651c162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6520e94640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c15f090 0x7f651c162500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6520e94640 1 -- 172.21.15.55:0/2806448066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651c0cab80 con 0x7f651c0cffb0 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.800+0000 7f6521695640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0cf660 0x7f651c15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:07:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.801+0000 7f6520e94640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cffb0 0x7f651c0c9e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f650c000c00 tx=0x7f650c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:54.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.801+0000 7f6511ffb640 1 -- 172.21.15.55:0/2806448066 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c037040 con 0x7f651c0cffb0 2024-05-20T22:07:54.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.801+0000 7f6511ffb640 1 -- 172.21.15.55:0/2806448066 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650c042070 con 0x7f651c0cffb0 2024-05-20T22:07:54.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.801+0000 7f65238f9640 1 -- 172.21.15.55:0/2806448066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651c0cae60 con 0x7f651c0cffb0 2024-05-20T22:07:54.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.801+0000 7f6511ffb640 1 -- 172.21.15.55:0/2806448066 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c030880 con 0x7f651c0cffb0 2024-05-20T22:07:54.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.801+0000 7f65238f9640 1 -- 172.21.15.55:0/2806448066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651c0cb420 con 0x7f651c0cffb0 2024-05-20T22:07:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.802+0000 7f65238f9640 1 -- 172.21.15.55:0/2806448066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651c163930 con 0x7f651c0cffb0 2024-05-20T22:07:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.803+0000 7f6511ffb640 1 -- 172.21.15.55:0/2806448066 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f650c030a20 con 0x7f651c0cffb0 2024-05-20T22:07:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.803+0000 7f6511ffb640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f650006be30 0x7f650006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.803+0000 7f6511ffb640 1 -- 172.21.15.55:0/2806448066 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f650c0ab7b0 con 0x7f651c0cffb0 2024-05-20T22:07:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.803+0000 7f6521695640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f650006be30 0x7f650006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.804+0000 7f6521695640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f650006be30 0x7f650006e340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f651c0b6fd0 tx=0x7f6518008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:54.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.808+0000 7f6511ffb640 1 -- 172.21.15.55:0/2806448066 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f650c079fa0 con 0x7f651c0cffb0 2024-05-20T22:07:54.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:54.993+0000 7f65238f9640 1 -- 172.21.15.55:0/2806448066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f651c069090 con 0x7f651c0cffb0 2024-05-20T22:07:56.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.686+0000 7f6511ffb640 1 -- 172.21.15.55:0/2806448066 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f650c0798a0 con 0x7f651c0cffb0 2024-05-20T22:07:56.687 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data_ec' created 2024-05-20T22:07:56.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 -- 172.21.15.55:0/2806448066 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f650006be30 msgr2=0x7f650006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:56.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f650006be30 0x7f650006e340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f651c0b6fd0 tx=0x7f6518008040 comp rx=0 tx=0).stop 2024-05-20T22:07:56.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 -- 172.21.15.55:0/2806448066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cffb0 msgr2=0x7f651c0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:56.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cffb0 0x7f651c0c9e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f650c000c00 tx=0x7f650c0046c0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 -- 172.21.15.55:0/2806448066 shutdown_connections 2024-05-20T22:07:56.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f650006be30 0x7f650006e340 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c15f090 0x7f651c162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f651c0cffb0 0x7f651c0c9e10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 --2- 172.21.15.55:0/2806448066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651c0cf660 0x7f651c15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 -- 172.21.15.55:0/2806448066 >> 172.21.15.55:0/2806448066 conn(0x7f651c0d53c0 msgr2=0x7f651c15e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:56.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 -- 172.21.15.55:0/2806448066 shutdown_connections 2024-05-20T22:07:56.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.689+0000 7f65238f9640 1 -- 172.21.15.55:0/2806448066 wait complete. 2024-05-20T22:07:56.711 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-20T22:07:56.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.940+0000 7f95fc884640 1 Processor -- start 2024-05-20T22:07:56.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.940+0000 7f95fc884640 1 -- start start 2024-05-20T22:07:56.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95fc884640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f95f806e630 0x7f95f80afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:56.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95fc884640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f815c4e0 0x7f95f815c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95fc884640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95f806ebd0 con 0x7f95f815ce30 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95fc884640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f95f806eeb0 con 0x7f95f806e630 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95fc884640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f95f806f190 con 0x7f95f815c4e0 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95f659c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f95f806e630 0x7f95f80afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95f659c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f95f806e630 0x7f95f80afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38512/0 (socket says 172.21.15.55:38512) 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95f659c640 1 -- 172.21.15.55:0/3870440862 learned_addr learned my addr 172.21.15.55:0/3870440862 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95f5d9b640 1 --2- 172.21.15.55:0/3870440862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f815c4e0 0x7f95f815c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95f659c640 1 -- 172.21.15.55:0/3870440862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f815c4e0 msgr2=0x7f95f815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95f659c640 1 --2- 172.21.15.55:0/3870440862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f815c4e0 0x7f95f815c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95f659c640 1 -- 172.21.15.55:0/3870440862 >> v1:172.21.15.138:6789/0 conn(0x7f95f815ce30 legacy=0x7f95f806b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:56.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.942+0000 7f95f659c640 1 -- 172.21.15.55:0/3870440862 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f806f470 con 0x7f95f806e630 2024-05-20T22:07:56.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.943+0000 7f95f659c640 1 --2- 172.21.15.55:0/3870440862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f806e630 0x7f95f80afcd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009e50 tx=0x7f95e802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804baf9d50c06986 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:56.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.943+0000 7f95f559a640 1 -- 172.21.15.55:0/3870440862 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8036070 con 0x7f95f806e630 2024-05-20T22:07:56.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.943+0000 7f95f559a640 1 -- 172.21.15.55:0/3870440862 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e8031600 con 0x7f95f806e630 2024-05-20T22:07:56.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.943+0000 7f95f559a640 1 -- 172.21.15.55:0/3870440862 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80319d0 con 0x7f95f806e630 2024-05-20T22:07:56.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.944+0000 7f95fc884640 1 -- 172.21.15.55:0/3870440862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f806e630 msgr2=0x7f95f80afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:56.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.944+0000 7f95fc884640 1 --2- 172.21.15.55:0/3870440862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f806e630 0x7f95f80afcd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009e50 tx=0x7f95e802a6c0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.944+0000 7f95fc884640 1 -- 172.21.15.55:0/3870440862 shutdown_connections 2024-05-20T22:07:56.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.944+0000 7f95fc884640 1 --2- 172.21.15.55:0/3870440862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f815c4e0 0x7f95f815c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.944+0000 7f95fc884640 1 --2- 172.21.15.55:0/3870440862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f806e630 0x7f95f80afcd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.944+0000 7f95fc884640 1 -- 172.21.15.55:0/3870440862 >> 172.21.15.55:0/3870440862 conn(0x7f95f80b3980 msgr2=0x7f95f80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:56.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.944+0000 7f95fc884640 1 -- 172.21.15.55:0/3870440862 shutdown_connections 2024-05-20T22:07:56.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.944+0000 7f95fc884640 1 -- 172.21.15.55:0/3870440862 wait complete. 2024-05-20T22:07:56.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.946+0000 7f95fc884640 1 Processor -- start 2024-05-20T22:07:56.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.946+0000 7f95fc884640 1 -- start start 2024-05-20T22:07:56.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.947+0000 7f95fc884640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f806e630 0x7f95f813ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95fc884640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f815c4e0 0x7f95f8150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95fc884640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f815ce30 0x7f95f8141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95fc884640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81608f0 con 0x7f95f815ce30 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95fc884640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95f80b0210 con 0x7f95f815c4e0 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95fc884640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f95f8160a70 con 0x7f95f806e630 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f6d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f815ce30 0x7f95f8141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f6d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f815ce30 0x7f95f8141110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38524/0 (socket says 172.21.15.55:38524) 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f6d9d640 1 -- 172.21.15.55:0/3271668923 learned_addr learned my addr 172.21.15.55:0/3271668923 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f659c640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f806e630 0x7f95f813ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f5d9b640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f815c4e0 0x7f95f8150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f6d9d640 1 -- 172.21.15.55:0/3271668923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f806e630 msgr2=0x7f95f813ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f6d9d640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f806e630 0x7f95f813ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f6d9d640 1 -- 172.21.15.55:0/3271668923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f815c4e0 msgr2=0x7f95f8150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:56.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f6d9d640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f815c4e0 0x7f95f8150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:56.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f6d9d640 1 -- 172.21.15.55:0/3271668923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f8155430 con 0x7f95f815ce30 2024-05-20T22:07:56.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95f6d9d640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f815ce30 0x7f95f8141110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f95f000b440 tx=0x7f95f000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:56.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95e6ffd640 1 -- 172.21.15.55:0/3271668923 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0015020 con 0x7f95f815ce30 2024-05-20T22:07:56.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95e6ffd640 1 -- 172.21.15.55:0/3271668923 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f0011830 con 0x7f95f815ce30 2024-05-20T22:07:56.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95fc884640 1 -- 172.21.15.55:0/3271668923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f8152ca0 con 0x7f95f815ce30 2024-05-20T22:07:56.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.948+0000 7f95e6ffd640 1 -- 172.21.15.55:0/3271668923 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0011c20 con 0x7f95f815ce30 2024-05-20T22:07:56.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.949+0000 7f95fc884640 1 -- 172.21.15.55:0/3271668923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f8154e70 con 0x7f95f815ce30 2024-05-20T22:07:56.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.950+0000 7f95fc884640 1 -- 172.21.15.55:0/3271668923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f8069090 con 0x7f95f815ce30 2024-05-20T22:07:56.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.950+0000 7f95e6ffd640 1 -- 172.21.15.55:0/3271668923 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95f0018020 con 0x7f95f815ce30 2024-05-20T22:07:56.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.951+0000 7f95e6ffd640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95d806be30 0x7f95d806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:56.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.951+0000 7f95e6ffd640 1 -- 172.21.15.55:0/3271668923 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f95f008ad70 con 0x7f95f815ce30 2024-05-20T22:07:56.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.951+0000 7f95f659c640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95d806be30 0x7f95d806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:56.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.951+0000 7f95f659c640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95d806be30 0x7f95d806e340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f95e8033000 tx=0x7f95e8009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:56.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:56.956+0000 7f95e6ffd640 1 -- 172.21.15.55:0/3271668923 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f95f001f020 con 0x7f95f815ce30 2024-05-20T22:07:57.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.141+0000 7f95fc884640 1 -- 172.21.15.55:0/3271668923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f95f8072e80 con 0x7f95f815ce30 2024-05-20T22:07:57.624 INFO:tasks.ceph.mds.d.smithi138.stderr:2024-05-20T22:07:57.623+0000 7ffaeba0a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T22:07:57.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.699+0000 7f95e6ffd640 1 -- 172.21.15.55:0/3271668923 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f95f0058e40 con 0x7f95f815ce30 2024-05-20T22:07:57.700 INFO:teuthology.orchestra.run.smithi055.stderr:set pool 4 allow_ec_overwrites to true 2024-05-20T22:07:57.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 -- 172.21.15.55:0/3271668923 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95d806be30 msgr2=0x7f95d806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:57.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95d806be30 0x7f95d806e340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f95e8033000 tx=0x7f95e8009000 comp rx=0 tx=0).stop 2024-05-20T22:07:57.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 -- 172.21.15.55:0/3271668923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f815ce30 msgr2=0x7f95f8141110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:57.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f815ce30 0x7f95f8141110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f95f000b440 tx=0x7f95f000b900 comp rx=0 tx=0).stop 2024-05-20T22:07:57.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 -- 172.21.15.55:0/3271668923 shutdown_connections 2024-05-20T22:07:57.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95d806be30 0x7f95d806e340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:57.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f815ce30 0x7f95f8141110 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:57.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f815c4e0 0x7f95f8150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:57.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 --2- 172.21.15.55:0/3271668923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f806e630 0x7f95f813ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:57.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 -- 172.21.15.55:0/3271668923 >> 172.21.15.55:0/3271668923 conn(0x7f95f80b3980 msgr2=0x7f95f80b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:57.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 -- 172.21.15.55:0/3271668923 shutdown_connections 2024-05-20T22:07:57.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.702+0000 7f95fc884640 1 -- 172.21.15.55:0/3271668923 wait complete. 2024-05-20T22:07:57.726 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-20T22:07:57.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.853+0000 7faca8110640 1 Processor -- start 2024-05-20T22:07:57.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.853+0000 7faca8110640 1 -- start start 2024-05-20T22:07:57.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca8110640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faca00d4210 0x7faca00b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:57.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca8110640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d0350 0x7faca006cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:57.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca8110640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca0163980 con 0x7faca00cf7b0 2024-05-20T22:07:57.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca8110640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7faca00d3d80 con 0x7faca00d4210 2024-05-20T22:07:57.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca8110640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faca00d3450 con 0x7faca00d0350 2024-05-20T22:07:57.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca5eac640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faca00d4210 0x7faca00b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:57.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca5eac640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faca00d4210 0x7faca00b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38540/0 (socket says 172.21.15.55:38540) 2024-05-20T22:07:57.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca66ad640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7faca00cf7b0 0x7faca00cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35746/0 (socket says 172.21.15.55:35746) 2024-05-20T22:07:57.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca5eac640 1 -- 172.21.15.55:0/2466000165 learned_addr learned my addr 172.21.15.55:0/2466000165 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:57.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca56ab640 1 --2- 172.21.15.55:0/2466000165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d0350 0x7faca006cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:57.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca5eac640 1 -- 172.21.15.55:0/2466000165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d0350 msgr2=0x7faca006cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:57.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca5eac640 1 --2- 172.21.15.55:0/2466000165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d0350 0x7faca006cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:57.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca5eac640 1 -- 172.21.15.55:0/2466000165 >> v1:172.21.15.55:6789/0 conn(0x7faca00cf7b0 legacy=0x7faca00cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:57.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca4eaa640 1 -- 172.21.15.55:0/2466000165 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3805662782 0 0) 0x7faca0163980 con 0x7faca00cf7b0 2024-05-20T22:07:57.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.855+0000 7faca5eac640 1 -- 172.21.15.55:0/2466000165 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca00d3730 con 0x7faca00d4210 2024-05-20T22:07:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.856+0000 7faca5eac640 1 --2- 172.21.15.55:0/2466000165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00d4210 0x7faca00b3b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fac98009870 tx=0x7fac9802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bcaa86bfd1d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.856+0000 7faca4eaa640 1 -- 172.21.15.55:0/2466000165 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98037070 con 0x7faca00d4210 2024-05-20T22:07:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.856+0000 7faca4eaa640 1 -- 172.21.15.55:0/2466000165 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac98032440 con 0x7faca00d4210 2024-05-20T22:07:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.856+0000 7faca4eaa640 1 -- 172.21.15.55:0/2466000165 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98032810 con 0x7faca00d4210 2024-05-20T22:07:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.857+0000 7faca8110640 1 -- 172.21.15.55:0/2466000165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00d4210 msgr2=0x7faca00b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.857+0000 7faca8110640 1 --2- 172.21.15.55:0/2466000165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00d4210 0x7faca00b3b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fac98009870 tx=0x7fac9802a1c0 comp rx=0 tx=0).stop 2024-05-20T22:07:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.857+0000 7faca8110640 1 -- 172.21.15.55:0/2466000165 shutdown_connections 2024-05-20T22:07:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.857+0000 7faca8110640 1 --2- 172.21.15.55:0/2466000165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d0350 0x7faca006cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.857+0000 7faca8110640 1 --2- 172.21.15.55:0/2466000165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00d4210 0x7faca00b3b20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:57.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.857+0000 7faca8110640 1 -- 172.21.15.55:0/2466000165 >> 172.21.15.55:0/2466000165 conn(0x7faca00d1bb0 msgr2=0x7faca00d4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:57.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.857+0000 7faca8110640 1 -- 172.21.15.55:0/2466000165 shutdown_connections 2024-05-20T22:07:57.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.857+0000 7faca8110640 1 -- 172.21.15.55:0/2466000165 wait complete. 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.859+0000 7faca8110640 1 Processor -- start 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.859+0000 7faca8110640 1 -- start start 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca8110640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00cf7b0 0x7faca01ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca8110640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faca00d0350 0x7faca0152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca8110640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d4210 0x7faca01e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca8110640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7faca0163720 con 0x7faca00cf7b0 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca8110640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faca01635a0 con 0x7faca00d0350 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca8110640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faca015e750 con 0x7faca00d4210 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca5eac640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00cf7b0 0x7faca01ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca5eac640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00cf7b0 0x7faca01ea1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38550/0 (socket says 172.21.15.55:38550) 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca5eac640 1 -- 172.21.15.55:0/1447829542 learned_addr learned my addr 172.21.15.55:0/1447829542 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca66ad640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d4210 0x7faca01e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca56ab640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faca00d0350 0x7faca0152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca5eac640 1 -- 172.21.15.55:0/1447829542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d4210 msgr2=0x7faca01e89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca5eac640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d4210 0x7faca01e89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca5eac640 1 -- 172.21.15.55:0/1447829542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faca00d0350 msgr2=0x7faca0152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca5eac640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faca00d0350 0x7faca0152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca5eac640 1 -- 172.21.15.55:0/1447829542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca0161b00 con 0x7faca00cf7b0 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.860+0000 7faca66ad640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d4210 0x7faca01e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.861+0000 7faca5eac640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00cf7b0 0x7faca01ea1e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac98000c00 tx=0x7fac980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.861+0000 7fac9e7fc640 1 -- 172.21.15.55:0/1447829542 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98037040 con 0x7faca00cf7b0 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.861+0000 7fac9e7fc640 1 -- 172.21.15.55:0/1447829542 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac98042070 con 0x7faca00cf7b0 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.861+0000 7faca8110640 1 -- 172.21.15.55:0/1447829542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca01ea720 con 0x7faca00cf7b0 2024-05-20T22:07:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.861+0000 7fac9e7fc640 1 -- 172.21.15.55:0/1447829542 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98030880 con 0x7faca00cf7b0 2024-05-20T22:07:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.861+0000 7faca8110640 1 -- 172.21.15.55:0/1447829542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca015fcf0 con 0x7faca00cf7b0 2024-05-20T22:07:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.862+0000 7faca8110640 1 -- 172.21.15.55:0/1447829542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca0069090 con 0x7faca00cf7b0 2024-05-20T22:07:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.862+0000 7fac9e7fc640 1 -- 172.21.15.55:0/1447829542 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac9802ad10 con 0x7faca00cf7b0 2024-05-20T22:07:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.863+0000 7fac9e7fc640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac8c06be30 0x7fac8c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.863+0000 7fac9e7fc640 1 -- 172.21.15.55:0/1447829542 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fac980ab7f0 con 0x7faca00cf7b0 2024-05-20T22:07:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.863+0000 7faca56ab640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac8c06be30 0x7fac8c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.863+0000 7faca56ab640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac8c06be30 0x7fac8c06e340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faca01e99b0 tx=0x7fac94008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:57.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:57.868+0000 7fac9e7fc640 1 -- 172.21.15.55:0/1447829542 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faca0161b00 con 0x7faca00cf7b0 2024-05-20T22:07:58.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.090+0000 7faca8110640 1 -- 172.21.15.55:0/1447829542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7faca00d2f70 con 0x7faca00cf7b0 2024-05-20T22:07:58.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.707+0000 7fac9e7fc640 1 -- 172.21.15.55:0/1447829542 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fac980a9e20 con 0x7faca00cf7b0 2024-05-20T22:07:58.708 INFO:teuthology.orchestra.run.smithi055.stderr:added data pool 4 to fsmap 2024-05-20T22:07:58.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 -- 172.21.15.55:0/1447829542 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac8c06be30 msgr2=0x7fac8c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:58.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac8c06be30 0x7fac8c06e340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faca01e99b0 tx=0x7fac94008040 comp rx=0 tx=0).stop 2024-05-20T22:07:58.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 -- 172.21.15.55:0/1447829542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00cf7b0 msgr2=0x7faca01ea1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:58.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00cf7b0 0x7faca01ea1e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac98000c00 tx=0x7fac980046c0 comp rx=0 tx=0).stop 2024-05-20T22:07:58.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 -- 172.21.15.55:0/1447829542 shutdown_connections 2024-05-20T22:07:58.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac8c06be30 0x7fac8c06e340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:58.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faca00d4210 0x7faca01e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:58.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faca00d0350 0x7faca0152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:58.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 --2- 172.21.15.55:0/1447829542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca00cf7b0 0x7faca01ea1e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:58.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 -- 172.21.15.55:0/1447829542 >> 172.21.15.55:0/1447829542 conn(0x7faca00d1bb0 msgr2=0x7faca00b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:58.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 -- 172.21.15.55:0/1447829542 shutdown_connections 2024-05-20T22:07:58.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.710+0000 7faca8110640 1 -- 172.21.15.55:0/1447829542 wait complete. 2024-05-20T22:07:58.724 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:07:58.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.846+0000 7fbabaf8f640 1 Processor -- start 2024-05-20T22:07:58.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.846+0000 7fbabaf8f640 1 -- start start 2024-05-20T22:07:58.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.847+0000 7fbabaf8f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbab406e250 0x7fbab415c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:58.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.847+0000 7fbabaf8f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbab40d1850 0x7fbab40d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.847+0000 7fbabaf8f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab40d21a0 con 0x7fbab409a260 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.847+0000 7fbabaf8f640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fbab40d1410 con 0x7fbab40d1850 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.847+0000 7fbabaf8f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbab406e7c0 con 0x7fbab406e250 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.847+0000 7fbab952c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbab40d1850 0x7fbab40d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.847+0000 7fbab952c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbab40d1850 0x7fbab40d1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52720/0 (socket says 172.21.15.55:52720) 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.847+0000 7fbab952c640 1 -- 172.21.15.55:0/1941444015 learned_addr learned my addr 172.21.15.55:0/1941444015 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.847+0000 7fbab8d2b640 1 --2- 172.21.15.55:0/1941444015 >> v2:172.21.15.138:3300/0 conn(0x7fbab406e250 0x7fbab415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.847+0000 7fbab952c640 1 -- 172.21.15.55:0/1941444015 >> v2:172.21.15.138:3300/0 conn(0x7fbab406e250 msgr2=0x7fbab415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.848+0000 7fbab952c640 1 --2- 172.21.15.55:0/1941444015 >> v2:172.21.15.138:3300/0 conn(0x7fbab406e250 0x7fbab415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.848+0000 7fbab952c640 1 -- 172.21.15.55:0/1941444015 >> v1:172.21.15.138:6789/0 conn(0x7fbab409a260 legacy=0x7fbab40d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:58.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.848+0000 7fbab952c640 1 -- 172.21.15.55:0/1941444015 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab40cff40 con 0x7fbab40d1850 2024-05-20T22:07:58.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.848+0000 7fbab952c640 1 --2- 172.21.15.55:0/1941444015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab40d1850 0x7fbab40d1c60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbab40d1590 tx=0x7fbaac02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40087eeb5b683935 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:58.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.848+0000 7fbab37fe640 1 -- 172.21.15.55:0/1941444015 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac036070 con 0x7fbab40d1850 2024-05-20T22:07:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.848+0000 7fbab37fe640 1 -- 172.21.15.55:0/1941444015 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaac031600 con 0x7fbab40d1850 2024-05-20T22:07:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.848+0000 7fbab37fe640 1 -- 172.21.15.55:0/1941444015 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0319d0 con 0x7fbab40d1850 2024-05-20T22:07:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.849+0000 7fbabaf8f640 1 -- 172.21.15.55:0/1941444015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab40d1850 msgr2=0x7fbab40d1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.849+0000 7fbabaf8f640 1 --2- 172.21.15.55:0/1941444015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab40d1850 0x7fbab40d1c60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbab40d1590 tx=0x7fbaac02a640 comp rx=0 tx=0).stop 2024-05-20T22:07:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.849+0000 7fbabaf8f640 1 -- 172.21.15.55:0/1941444015 shutdown_connections 2024-05-20T22:07:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.849+0000 7fbabaf8f640 1 --2- 172.21.15.55:0/1941444015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab40d1850 0x7fbab40d1c60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.849+0000 7fbabaf8f640 1 --2- 172.21.15.55:0/1941444015 >> v2:172.21.15.138:3300/0 conn(0x7fbab406e250 0x7fbab415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.849+0000 7fbabaf8f640 1 -- 172.21.15.55:0/1941444015 >> 172.21.15.55:0/1941444015 conn(0x7fbab40b6580 msgr2=0x7fbab40b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.849+0000 7fbabaf8f640 1 -- 172.21.15.55:0/1941444015 shutdown_connections 2024-05-20T22:07:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.849+0000 7fbabaf8f640 1 -- 172.21.15.55:0/1941444015 wait complete. 2024-05-20T22:07:58.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.851+0000 7fbabaf8f640 1 Processor -- start 2024-05-20T22:07:58.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.851+0000 7fbabaf8f640 1 -- start start 2024-05-20T22:07:58.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.852+0000 7fbabaf8f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab406e250 0x7fbab415da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbabaf8f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab409a260 0x7fbab40b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbabaf8f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbab40d1850 0x7fbab40af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbabaf8f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbab40d07e0 con 0x7fbab409a260 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbabaf8f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbab40d0500 con 0x7fbab406e250 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbabaf8f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbab409a670 con 0x7fbab40d1850 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab409a260 0x7fbab40b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab409a260 0x7fbab40b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52730/0 (socket says 172.21.15.55:52730) 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab3fff640 1 -- 172.21.15.55:0/2504041907 learned_addr learned my addr 172.21.15.55:0/2504041907 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab8d2b640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab406e250 0x7fbab415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab952c640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbab40d1850 0x7fbab40af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab3fff640 1 -- 172.21.15.55:0/2504041907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbab40d1850 msgr2=0x7fbab40af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab3fff640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbab40d1850 0x7fbab40af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab3fff640 1 -- 172.21.15.55:0/2504041907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab406e250 msgr2=0x7fbab415da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab3fff640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab406e250 0x7fbab415da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab3fff640 1 -- 172.21.15.55:0/2504041907 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab40afc20 con 0x7fbab409a260 2024-05-20T22:07:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab3fff640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab409a260 0x7fbab40b0b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009fd0 tx=0x7fbaa400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab17fa640 1 -- 172.21.15.55:0/2504041907 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa400f8a0 con 0x7fbab409a260 2024-05-20T22:07:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab17fa640 1 -- 172.21.15.55:0/2504041907 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa400fa40 con 0x7fbab409a260 2024-05-20T22:07:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbabaf8f640 1 -- 172.21.15.55:0/2504041907 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab40cda40 con 0x7fbab409a260 2024-05-20T22:07:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbab17fa640 1 -- 172.21.15.55:0/2504041907 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa400fe30 con 0x7fbab409a260 2024-05-20T22:07:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.853+0000 7fbabaf8f640 1 -- 172.21.15.55:0/2504041907 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab40ce000 con 0x7fbab409a260 2024-05-20T22:07:58.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.854+0000 7fbabaf8f640 1 -- 172.21.15.55:0/2504041907 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab4069090 con 0x7fbab409a260 2024-05-20T22:07:58.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.855+0000 7fbab17fa640 1 -- 172.21.15.55:0/2504041907 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbaa4010470 con 0x7fbab409a260 2024-05-20T22:07:58.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.856+0000 7fbab17fa640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba9406be60 0x7fba9406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:58.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.856+0000 7fbab17fa640 1 -- 172.21.15.55:0/2504041907 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbaa408d120 con 0x7fbab409a260 2024-05-20T22:07:58.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.856+0000 7fbab8d2b640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba9406be60 0x7fba9406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:58.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.856+0000 7fbab8d2b640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba9406be60 0x7fba9406e370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba9c006fd0 tx=0x7fba9c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:58.860+0000 7fbab17fa640 1 -- 172.21.15.55:0/2504041907 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbaa4015030 con 0x7fbab409a260 2024-05-20T22:07:59.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.080+0000 7fbabaf8f640 1 -- 172.21.15.55:0/2504041907 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbab40b1080 con 0x7fbab409a260 2024-05-20T22:07:59.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.081+0000 7fbab17fa640 1 -- 172.21.15.55:0/2504041907 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5174 (secure 0 0 0) 0x7fbaa405b660 con 0x7fbab409a260 2024-05-20T22:07:59.082 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:07:59.082 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:07:53.028952+0000","modified":"2024-05-20T22:07:58.091673+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-05-20T22:07:59.082 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 7 2024-05-20T22:07:59.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 -- 172.21.15.55:0/2504041907 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba9406be60 msgr2=0x7fba9406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba9406be60 0x7fba9406e370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba9c006fd0 tx=0x7fba9c008040 comp rx=0 tx=0).stop 2024-05-20T22:07:59.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 -- 172.21.15.55:0/2504041907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab409a260 msgr2=0x7fbab40b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab409a260 0x7fbab40b0b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009fd0 tx=0x7fbaa400b870 comp rx=0 tx=0).stop 2024-05-20T22:07:59.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 -- 172.21.15.55:0/2504041907 shutdown_connections 2024-05-20T22:07:59.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba9406be60 0x7fba9406e370 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbab40d1850 0x7fbab40af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbab409a260 0x7fbab40b0b40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 --2- 172.21.15.55:0/2504041907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab406e250 0x7fbab415da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 -- 172.21.15.55:0/2504041907 >> 172.21.15.55:0/2504041907 conn(0x7fbab40b6580 msgr2=0x7fbab40d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:59.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 -- 172.21.15.55:0/2504041907 shutdown_connections 2024-05-20T22:07:59.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.083+0000 7fbabaf8f640 1 -- 172.21.15.55:0/2504041907 wait complete. 2024-05-20T22:07:59.095 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:07:59.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.208+0000 7f05d41bc640 1 Processor -- start 2024-05-20T22:07:59.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.208+0000 7f05d41bc640 1 -- start start 2024-05-20T22:07:59.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d41bc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 0x7f05cc169f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d41bc640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05cc0b83f0 con 0x7f05cc0b3b30 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d41bc640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05cc0d3dd0 con 0x7f05cc15f100 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d41bc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05cc06cea0 con 0x7f05cc0b7fe0 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d1f58640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f05cc0b3b30 0x7f05cc06c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47296/0 (socket says 172.21.15.55:47296) 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d1f58640 1 -- 172.21.15.55:0/2454963902 learned_addr learned my addr 172.21.15.55:0/2454963902 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d2759640 1 --2- 172.21.15.55:0/2454963902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 0x7f05cc169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d0f56640 1 -- 172.21.15.55:0/2454963902 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2466247852 0 0) 0x7f05cc0b83f0 con 0x7f05cc0b3b30 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d0f56640 1 -- 172.21.15.55:0/2454963902 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05b8003520 con 0x7f05cc0b3b30 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d0f56640 1 -- 172.21.15.55:0/2454963902 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2774154483 0 0) 0x7f05cc0d3dd0 con 0x7f05cc15f100 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d0f56640 1 -- 172.21.15.55:0/2454963902 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05cc0b83f0 con 0x7f05cc15f100 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d0f56640 1 -- 172.21.15.55:0/2454963902 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2492662086 0 0) 0x7f05b8003520 con 0x7f05cc0b3b30 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d2759640 1 -- 172.21.15.55:0/2454963902 >> v1:172.21.15.138:6789/0 conn(0x7f05cc15f100 legacy=0x7f05cc06c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d2759640 1 -- 172.21.15.55:0/2454963902 >> v1:172.21.15.55:6789/0 conn(0x7f05cc0b3b30 legacy=0x7f05cc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.210+0000 7f05d2759640 1 -- 172.21.15.55:0/2454963902 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05cc0d40b0 con 0x7f05cc0b7fe0 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.211+0000 7f05d2759640 1 --2- 172.21.15.55:0/2454963902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 0x7f05cc169f00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05c40097d0 tx=0x7f05c402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e6106856e7aaee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.211+0000 7f05d0f56640 1 -- 172.21.15.55:0/2454963902 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4037070 con 0x7f05cc0b7fe0 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.211+0000 7f05d0f56640 1 -- 172.21.15.55:0/2454963902 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c402ac30 con 0x7f05cc0b7fe0 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.211+0000 7f05d0f56640 1 -- 172.21.15.55:0/2454963902 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8001a10 con 0x7f05cc0b7fe0 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.212+0000 7f05d41bc640 1 -- 172.21.15.55:0/2454963902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 msgr2=0x7f05cc169f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.212+0000 7f05d41bc640 1 --2- 172.21.15.55:0/2454963902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 0x7f05cc169f00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05c40097d0 tx=0x7f05c402a1a0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.212+0000 7f05d41bc640 1 -- 172.21.15.55:0/2454963902 shutdown_connections 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.212+0000 7f05d41bc640 1 --2- 172.21.15.55:0/2454963902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 0x7f05cc169f00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.212+0000 7f05d41bc640 1 -- 172.21.15.55:0/2454963902 >> 172.21.15.55:0/2454963902 conn(0x7f05cc0d51d0 msgr2=0x7f05cc0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.212+0000 7f05d41bc640 1 -- 172.21.15.55:0/2454963902 shutdown_connections 2024-05-20T22:07:59.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.212+0000 7f05d41bc640 1 -- 172.21.15.55:0/2454963902 wait complete. 2024-05-20T22:07:59.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.214+0000 7f05d41bc640 1 Processor -- start 2024-05-20T22:07:59.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.214+0000 7f05d41bc640 1 -- start start 2024-05-20T22:07:59.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.215+0000 7f05d41bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0b3b30 0x7f05cc0be610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.215+0000 7f05d41bc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 0x7f05cc161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.215+0000 7f05d41bc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05cc15f100 0x7f05cc161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.215+0000 7f05d41bc640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc06b880 con 0x7f05cc0b3b30 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.215+0000 7f05d41bc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc06b5a0 con 0x7f05cc15f100 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.215+0000 7f05d41bc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05cc06bb60 con 0x7f05cc0b7fe0 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d1f58640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0b3b30 0x7f05cc0be610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d1f58640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0b3b30 0x7f05cc0be610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52746/0 (socket says 172.21.15.55:52746) 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d1f58640 1 -- 172.21.15.55:0/1064976008 learned_addr learned my addr 172.21.15.55:0/1064976008 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d1757640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 0x7f05cc161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d2759640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05cc15f100 0x7f05cc161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d1f58640 1 -- 172.21.15.55:0/1064976008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 msgr2=0x7f05cc161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d1f58640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 0x7f05cc161250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d1f58640 1 -- 172.21.15.55:0/1064976008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05cc15f100 msgr2=0x7f05cc161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d1f58640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05cc15f100 0x7f05cc161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d1f58640 1 -- 172.21.15.55:0/1064976008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05cc1f5e60 con 0x7f05cc0b3b30 2024-05-20T22:07:59.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d1f58640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0b3b30 0x7f05cc0be610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f05c800cbc0 tx=0x7f05c800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:59.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05c27fc640 1 -- 172.21.15.55:0/1064976008 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8017070 con 0x7f05cc0b3b30 2024-05-20T22:07:59.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05c27fc640 1 -- 172.21.15.55:0/1064976008 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c800ace0 con 0x7f05cc0b3b30 2024-05-20T22:07:59.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05c27fc640 1 -- 172.21.15.55:0/1064976008 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8012600 con 0x7f05cc0b3b30 2024-05-20T22:07:59.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d41bc640 1 -- 172.21.15.55:0/1064976008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05cc157920 con 0x7f05cc0b3b30 2024-05-20T22:07:59.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.216+0000 7f05d41bc640 1 -- 172.21.15.55:0/1064976008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05cc157ee0 con 0x7f05cc0b3b30 2024-05-20T22:07:59.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.217+0000 7f05d41bc640 1 -- 172.21.15.55:0/1064976008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05cc0d2ca0 con 0x7f05cc0b3b30 2024-05-20T22:07:59.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.218+0000 7f05c27fc640 1 -- 172.21.15.55:0/1064976008 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05c801a070 con 0x7f05cc0b3b30 2024-05-20T22:07:59.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.218+0000 7f05c27fc640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b006be30 0x7f05b006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.218+0000 7f05c27fc640 1 -- 172.21.15.55:0/1064976008 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f05c808afd0 con 0x7f05cc0b3b30 2024-05-20T22:07:59.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.219+0000 7f05d1757640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b006be30 0x7f05b006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.219+0000 7f05d1757640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b006be30 0x7f05b006e340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0bed40 tx=0x7f05bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:59.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.223+0000 7f05c27fc640 1 -- 172.21.15.55:0/1064976008 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f05cc0d2ca0 con 0x7f05cc0b3b30 2024-05-20T22:07:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.406+0000 7f05d41bc640 1 -- 172.21.15.55:0/1064976008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05cc069090 con 0x7f05cc0b3b30 2024-05-20T22:07:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.407+0000 7f05c27fc640 1 -- 172.21.15.55:0/1064976008 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f05c8058f50 con 0x7f05cc0b3b30 2024-05-20T22:07:59.409 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:07:59.409 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:07:59.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 -- 172.21.15.55:0/1064976008 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b006be30 msgr2=0x7f05b006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b006be30 0x7f05b006e340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0bed40 tx=0x7f05bc009040 comp rx=0 tx=0).stop 2024-05-20T22:07:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 -- 172.21.15.55:0/1064976008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0b3b30 msgr2=0x7f05cc0be610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0b3b30 0x7f05cc0be610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f05c800cbc0 tx=0x7f05c800a430 comp rx=0 tx=0).stop 2024-05-20T22:07:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 -- 172.21.15.55:0/1064976008 shutdown_connections 2024-05-20T22:07:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b006be30 0x7f05b006e340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05cc15f100 0x7f05cc161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05cc0b7fe0 0x7f05cc161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 --2- 172.21.15.55:0/1064976008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0b3b30 0x7f05cc0be610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 -- 172.21.15.55:0/1064976008 >> 172.21.15.55:0/1064976008 conn(0x7f05cc0d51d0 msgr2=0x7f05cc06c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 -- 172.21.15.55:0/1064976008 shutdown_connections 2024-05-20T22:07:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.410+0000 7f05d41bc640 1 -- 172.21.15.55:0/1064976008 wait complete. 2024-05-20T22:07:59.422 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:07:59.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.536+0000 7f585a83d640 1 Processor -- start 2024-05-20T22:07:59.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.536+0000 7f585a83d640 1 -- start start 2024-05-20T22:07:59.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f585a83d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f58540d3390 0x7f585406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f585a83d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58540b3c90 0x7f58540b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f585a83d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f585406ce50 con 0x7f58540b8550 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f585a83d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f585406b870 con 0x7f58540b3c90 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f585a83d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f585406bb50 con 0x7f58540d3390 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f58537fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58540b3c90 0x7f58540b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f5858dda640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f58540b8550 0x7f585415e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47310/0 (socket says 172.21.15.55:47310) 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f5858dda640 1 -- 172.21.15.55:0/3621245342 learned_addr learned my addr 172.21.15.55:0/3621245342 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f58537fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58540b3c90 0x7f58540b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52750/0 (socket says 172.21.15.55:52750) 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f5853fff640 1 --2- 172.21.15.55:0/3621245342 >> v2:172.21.15.138:3300/0 conn(0x7f58540d3390 0x7f585406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f58537fe640 1 -- 172.21.15.55:0/3621245342 >> v2:172.21.15.138:3300/0 conn(0x7f58540d3390 msgr2=0x7f585406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f58537fe640 1 --2- 172.21.15.55:0/3621245342 >> v2:172.21.15.138:3300/0 conn(0x7f58540d3390 0x7f585406c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f58537fe640 1 -- 172.21.15.55:0/3621245342 >> v1:172.21.15.55:6789/0 conn(0x7f58540b8550 legacy=0x7f585415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f5852ffd640 1 -- 172.21.15.55:0/3621245342 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1851037728 0 0) 0x7f585406ce50 con 0x7f58540b8550 2024-05-20T22:07:59.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.538+0000 7f58537fe640 1 -- 172.21.15.55:0/3621245342 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585406be30 con 0x7f58540b3c90 2024-05-20T22:07:59.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.539+0000 7f58537fe640 1 --2- 172.21.15.55:0/3621245342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540b3c90 0x7f58540b8010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f585408d060 tx=0x7f584802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e689fea34134812 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:59.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.539+0000 7f5852ffd640 1 -- 172.21.15.55:0/3621245342 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848037070 con 0x7f58540b3c90 2024-05-20T22:07:59.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.539+0000 7f5852ffd640 1 -- 172.21.15.55:0/3621245342 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5848032440 con 0x7f58540b3c90 2024-05-20T22:07:59.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.539+0000 7f5852ffd640 1 -- 172.21.15.55:0/3621245342 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848032810 con 0x7f58540b3c90 2024-05-20T22:07:59.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.540+0000 7f585a83d640 1 -- 172.21.15.55:0/3621245342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540b3c90 msgr2=0x7f58540b8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.540+0000 7f585a83d640 1 --2- 172.21.15.55:0/3621245342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540b3c90 0x7f58540b8010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f585408d060 tx=0x7f584802a1a0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.540+0000 7f585a83d640 1 -- 172.21.15.55:0/3621245342 shutdown_connections 2024-05-20T22:07:59.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.540+0000 7f585a83d640 1 --2- 172.21.15.55:0/3621245342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540b3c90 0x7f58540b8010 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.540+0000 7f585a83d640 1 --2- 172.21.15.55:0/3621245342 >> v2:172.21.15.138:3300/0 conn(0x7f58540d3390 0x7f585406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.540+0000 7f585a83d640 1 -- 172.21.15.55:0/3621245342 >> 172.21.15.55:0/3621245342 conn(0x7f58540b65f0 msgr2=0x7f58540b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:59.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.540+0000 7f585a83d640 1 -- 172.21.15.55:0/3621245342 shutdown_connections 2024-05-20T22:07:59.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.540+0000 7f585a83d640 1 -- 172.21.15.55:0/3621245342 wait complete. 2024-05-20T22:07:59.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.542+0000 7f585a83d640 1 Processor -- start 2024-05-20T22:07:59.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.542+0000 7f585a83d640 1 -- start start 2024-05-20T22:07:59.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.543+0000 7f585a83d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58540b3c90 0x7f58540ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.543+0000 7f585a83d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58540b8550 0x7f58540b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.543+0000 7f585a83d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540d3390 0x7f5854154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.543+0000 7f585a83d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f585406b500 con 0x7f58540d3390 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.543+0000 7f585a83d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f585406c3f0 con 0x7f58540b8550 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.543+0000 7f585a83d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f58540b40a0 con 0x7f58540b3c90 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.543+0000 7f5858dda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540d3390 0x7f5854154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.543+0000 7f5858dda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540d3390 0x7f5854154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52764/0 (socket says 172.21.15.55:52764) 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.543+0000 7f5858dda640 1 -- 172.21.15.55:0/3936595371 learned_addr learned my addr 172.21.15.55:0/3936595371 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f58537fe640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58540b8550 0x7f58540b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5853fff640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58540b3c90 0x7f58540ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5858dda640 1 -- 172.21.15.55:0/3936595371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58540b3c90 msgr2=0x7f58540ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5858dda640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58540b3c90 0x7f58540ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5858dda640 1 -- 172.21.15.55:0/3936595371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58540b8550 msgr2=0x7f58540b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5858dda640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58540b8550 0x7f58540b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5858dda640 1 -- 172.21.15.55:0/3936595371 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5854152fa0 con 0x7f58540d3390 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f58537fe640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58540b8550 0x7f58540b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:07:59.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5853fff640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58540b3c90 0x7f58540ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:07:59.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5858dda640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540d3390 0x7f5854154040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f584c00c700 tx=0x7f584c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:59.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5850ff9640 1 -- 172.21.15.55:0/3936595371 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c018070 con 0x7f58540d3390 2024-05-20T22:07:59.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5850ff9640 1 -- 172.21.15.55:0/3936595371 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584c00ce10 con 0x7f58540d3390 2024-05-20T22:07:59.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f585a83d640 1 -- 172.21.15.55:0/3936595371 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5854153280 con 0x7f58540d3390 2024-05-20T22:07:59.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f5850ff9640 1 -- 172.21.15.55:0/3936595371 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c00a820 con 0x7f58540d3390 2024-05-20T22:07:59.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.544+0000 7f585a83d640 1 -- 172.21.15.55:0/3936595371 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5854153840 con 0x7f58540d3390 2024-05-20T22:07:59.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.545+0000 7f585a83d640 1 -- 172.21.15.55:0/3936595371 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5854069090 con 0x7f58540d3390 2024-05-20T22:07:59.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.546+0000 7f5850ff9640 1 -- 172.21.15.55:0/3936595371 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f584c01b070 con 0x7f58540d3390 2024-05-20T22:07:59.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.546+0000 7f5850ff9640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f584006be30 0x7f584006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.546+0000 7f5850ff9640 1 -- 172.21.15.55:0/3936595371 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f584c08a770 con 0x7f58540d3390 2024-05-20T22:07:59.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.546+0000 7f5853fff640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f584006be30 0x7f584006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.547+0000 7f5853fff640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f584006be30 0x7f584006e340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f585408d370 tx=0x7f5844008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:59.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.551+0000 7f5850ff9640 1 -- 172.21.15.55:0/3936595371 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f584c058d50 con 0x7f58540d3390 2024-05-20T22:07:59.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.739+0000 7f585a83d640 1 -- 172.21.15.55:0/3936595371 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58540d2eb0 con 0x7f58540d3390 2024-05-20T22:07:59.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.740+0000 7f5850ff9640 1 -- 172.21.15.55:0/3936595371 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f584c058650 con 0x7f58540d3390 2024-05-20T22:07:59.741 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:07:59.741 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:07:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 -- 172.21.15.55:0/3936595371 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f584006be30 msgr2=0x7f584006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f584006be30 0x7f584006e340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f585408d370 tx=0x7f5844008040 comp rx=0 tx=0).stop 2024-05-20T22:07:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 -- 172.21.15.55:0/3936595371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540d3390 msgr2=0x7f5854154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540d3390 0x7f5854154040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f584c00c700 tx=0x7f584c00cbc0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 -- 172.21.15.55:0/3936595371 shutdown_connections 2024-05-20T22:07:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f584006be30 0x7f584006e340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58540d3390 0x7f5854154040 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58540b8550 0x7f58540b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 --2- 172.21.15.55:0/3936595371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58540b3c90 0x7f58540ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 -- 172.21.15.55:0/3936595371 >> 172.21.15.55:0/3936595371 conn(0x7f58540b65f0 msgr2=0x7f58540b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:59.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 -- 172.21.15.55:0/3936595371 shutdown_connections 2024-05-20T22:07:59.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.742+0000 7f585a83d640 1 -- 172.21.15.55:0/3936595371 wait complete. 2024-05-20T22:07:59.770 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:07:59.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.884+0000 7f4367261640 1 Processor -- start 2024-05-20T22:07:59.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.884+0000 7f4367261640 1 -- start start 2024-05-20T22:07:59.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f4367261640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f43600cedc0 0x7f43601618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f4367261640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436015d6c0 con 0x7f43600b7b00 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f4367261640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43600d11a0 con 0x7f4360160dc0 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f4367261640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f43600d0f50 con 0x7f43600cedc0 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f4364ffd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f43600b7b00 0x7f436015cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47326/0 (socket says 172.21.15.55:47326) 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f4364ffd640 1 -- 172.21.15.55:0/45746177 learned_addr learned my addr 172.21.15.55:0/45746177 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f43657fe640 1 --2- 172.21.15.55:0/45746177 >> v2:172.21.15.138:3300/0 conn(0x7f43600cedc0 0x7f43601618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4028079924 0 0) 0x7f436015d6c0 con 0x7f43600b7b00 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4348003520 con 0x7f43600b7b00 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3981041314 0 0) 0x7f4348003520 con 0x7f43600b7b00 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 >> v2:172.21.15.138:3300/0 conn(0x7f43600cedc0 msgr2=0x7f43601618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f435f7fe640 1 --2- 172.21.15.55:0/45746177 >> v2:172.21.15.138:3300/0 conn(0x7f43600cedc0 0x7f43601618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 >> v1:172.21.15.138:6789/0 conn(0x7f4360160dc0 legacy=0x7f43601611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43600d1480 con 0x7f43600b7b00 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f43500017f0 con 0x7f43600b7b00 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.886+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 >> v1:172.21.15.55:6789/0 conn(0x7f43600b7b00 legacy=0x7f436015cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435f7fe640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43480039b0 0x7f4348003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435f7fe640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f4348004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435f7fe640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4348005250 0x7f434800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f43600d0f50 con 0x7f4348004800 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f434800c1e0 con 0x7f4348005250 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f434800c3e0 con 0x7f43480039b0 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435ffff640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f4348004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f4364ffd640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43480039b0 0x7f4348003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f43657fe640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4348005250 0x7f434800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435ffff640 1 -- 172.21.15.55:0/45746177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43480039b0 msgr2=0x7f4348003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:59.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435ffff640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43480039b0 0x7f4348003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435ffff640 1 -- 172.21.15.55:0/45746177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4348005250 msgr2=0x7f434800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:07:59.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435ffff640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4348005250 0x7f434800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f435ffff640 1 -- 172.21.15.55:0/45746177 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4348003520 con 0x7f4348004800 2024-05-20T22:07:59.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f4364ffd640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43480039b0 0x7f4348003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:07:59.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f43657fe640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4348005250 0x7f434800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:07:59.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.887+0000 7f43657fe640 1 -- 172.21.15.55:0/45746177 reap_dead start 2024-05-20T22:07:59.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.888+0000 7f435ffff640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f4348004c10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4354001750 tx=0x7f4348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7283b30dfeb3a9c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:59.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.888+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354001370 con 0x7f4348004800 2024-05-20T22:07:59.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.888+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43540052d0 con 0x7f4348004800 2024-05-20T22:07:59.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.888+0000 7f435f7fe640 1 -- 172.21.15.55:0/45746177 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435402abc0 con 0x7f4348004800 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.888+0000 7f4367261640 1 -- 172.21.15.55:0/45746177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 msgr2=0x7f4348004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.888+0000 7f4367261640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f4348004c10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4354001750 tx=0x7f4348005830 comp rx=0 tx=0).stop 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.889+0000 7f4367261640 1 -- 172.21.15.55:0/45746177 shutdown_connections 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.889+0000 7f4367261640 1 --2- 172.21.15.55:0/45746177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f4348004c10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.889+0000 7f4367261640 1 -- 172.21.15.55:0/45746177 >> 172.21.15.55:0/45746177 conn(0x7f43600b4460 msgr2=0x7f43600b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.889+0000 7f4367261640 1 -- 172.21.15.55:0/45746177 shutdown_connections 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.889+0000 7f4367261640 1 -- 172.21.15.55:0/45746177 wait complete. 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.890+0000 7f4367261640 1 Processor -- start 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.890+0000 7f4367261640 1 -- start start 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4367261640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f436006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4367261640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43600c9a20 0x7f436006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4367261640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43600cb830 0x7f43600cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4367261640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f43600d1a40 con 0x7f4348004800 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4367261640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43600cd920 con 0x7f43600cb830 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4367261640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43600d1bc0 con 0x7f43600c9a20 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4364ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f436006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4364ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f436006cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52780/0 (socket says 172.21.15.55:52780) 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4364ffd640 1 -- 172.21.15.55:0/2695570433 learned_addr learned my addr 172.21.15.55:0/2695570433 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f435ffff640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43600c9a20 0x7f436006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f43657fe640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43600cb830 0x7f43600cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4364ffd640 1 -- 172.21.15.55:0/2695570433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43600c9a20 msgr2=0x7f436006d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4364ffd640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43600c9a20 0x7f436006d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4364ffd640 1 -- 172.21.15.55:0/2695570433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43600cb830 msgr2=0x7f43600cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4364ffd640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43600cb830 0x7f43600cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.892+0000 7f4364ffd640 1 -- 172.21.15.55:0/2695570433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43600caba0 con 0x7f4348004800 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.893+0000 7f4364ffd640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f436006cea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4350000ef0 tx=0x7f435000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.893+0000 7f435d7fa640 1 -- 172.21.15.55:0/2695570433 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43500070d0 con 0x7f4348004800 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.893+0000 7f435d7fa640 1 -- 172.21.15.55:0/2695570433 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4350018070 con 0x7f4348004800 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.893+0000 7f435d7fa640 1 -- 172.21.15.55:0/2695570433 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350007470 con 0x7f4348004800 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.893+0000 7f4367261640 1 -- 172.21.15.55:0/2695570433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43601f91f0 con 0x7f4348004800 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.893+0000 7f4367261640 1 -- 172.21.15.55:0/2695570433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436006dc10 con 0x7f4348004800 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.894+0000 7f4367261640 1 -- 172.21.15.55:0/2695570433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4360069090 con 0x7f4348004800 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.894+0000 7f435d7fa640 1 -- 172.21.15.55:0/2695570433 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43500057a0 con 0x7f4348004800 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.895+0000 7f435d7fa640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f434c06be30 0x7f434c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.895+0000 7f435ffff640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f434c06be30 0x7f434c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.895+0000 7f435d7fa640 1 -- 172.21.15.55:0/2695570433 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f435008af60 con 0x7f4348004800 2024-05-20T22:07:59.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.896+0000 7f435ffff640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f434c06be30 0x7f434c06e340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4354032000 tx=0x7f435402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:07:59.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:07:59.900+0000 7f435d7fa640 1 -- 172.21.15.55:0/2695570433 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4350059690 con 0x7f4348004800 2024-05-20T22:08:00.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.121+0000 7f4367261640 1 -- 172.21.15.55:0/2695570433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43600b51c0 con 0x7f4348004800 2024-05-20T22:08:00.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.122+0000 7f435d7fa640 1 -- 172.21.15.55:0/2695570433 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5174 (secure 0 0 0) 0x7f4350058f90 con 0x7f4348004800 2024-05-20T22:08:00.123 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:00.123 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:07:53.028952+0000","modified":"2024-05-20T22:07:58.091673+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-05-20T22:08:00.123 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 7 2024-05-20T22:08:00.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 -- 172.21.15.55:0/2695570433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f434c06be30 msgr2=0x7f434c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f434c06be30 0x7f434c06e340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4354032000 tx=0x7f435402e000 comp rx=0 tx=0).stop 2024-05-20T22:08:00.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 -- 172.21.15.55:0/2695570433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 msgr2=0x7f436006cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f436006cea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4350000ef0 tx=0x7f435000ec10 comp rx=0 tx=0).stop 2024-05-20T22:08:00.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 -- 172.21.15.55:0/2695570433 shutdown_connections 2024-05-20T22:08:00.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f434c06be30 0x7f434c06e340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43600cb830 0x7f43600cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43600c9a20 0x7f436006d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 --2- 172.21.15.55:0/2695570433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4348004800 0x7f436006cea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 -- 172.21.15.55:0/2695570433 >> 172.21.15.55:0/2695570433 conn(0x7f43600b4460 msgr2=0x7f43601611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:00.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 -- 172.21.15.55:0/2695570433 shutdown_connections 2024-05-20T22:08:00.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.124+0000 7f4367261640 1 -- 172.21.15.55:0/2695570433 wait complete. 2024-05-20T22:08:00.136 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:00.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.250+0000 7fcd19f96640 1 Processor -- start 2024-05-20T22:08:00.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.250+0000 7fcd19f96640 1 -- start start 2024-05-20T22:08:00.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd19f96640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd140b3c90 0x7fcd140b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:00.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd19f96640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1406b870 con 0x7fcd140d3390 2024-05-20T22:08:00.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd19f96640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1406bb50 con 0x7fcd140b8550 2024-05-20T22:08:00.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd19f96640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1406be30 con 0x7fcd140b3c90 2024-05-20T22:08:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd12ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd140b3c90 0x7fcd140b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd137fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcd140d3390 0x7fcd1406c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47334/0 (socket says 172.21.15.55:47334) 2024-05-20T22:08:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd12ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd140b3c90 0x7fcd140b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52784/0 (socket says 172.21.15.55:52784) 2024-05-20T22:08:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd137fe640 1 -- 172.21.15.55:0/737777780 learned_addr learned my addr 172.21.15.55:0/737777780 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd12ffd640 1 -- 172.21.15.55:0/737777780 >> v1:172.21.15.138:6789/0 conn(0x7fcd140b8550 legacy=0x7fcd1415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd12ffd640 1 -- 172.21.15.55:0/737777780 >> v1:172.21.15.55:6789/0 conn(0x7fcd140d3390 legacy=0x7fcd1406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd12ffd640 1 -- 172.21.15.55:0/737777780 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1406c110 con 0x7fcd140b3c90 2024-05-20T22:08:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.251+0000 7fcd127fc640 1 -- 172.21.15.55:0/737777780 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 187738034 0 0) 0x7fcd1406b870 con 0x7fcd140d3390 2024-05-20T22:08:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.252+0000 7fcd12ffd640 1 --2- 172.21.15.55:0/737777780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140b3c90 0x7fcd140b8010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097a0 tx=0x7fcd0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533a196d49cfbe1f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.252+0000 7fcd127fc640 1 -- 172.21.15.55:0/737777780 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04037070 con 0x7fcd140b3c90 2024-05-20T22:08:00.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.252+0000 7fcd127fc640 1 -- 172.21.15.55:0/737777780 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd04032440 con 0x7fcd140b3c90 2024-05-20T22:08:00.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.252+0000 7fcd127fc640 1 -- 172.21.15.55:0/737777780 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04032810 con 0x7fcd140b3c90 2024-05-20T22:08:00.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.253+0000 7fcd19f96640 1 -- 172.21.15.55:0/737777780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140b3c90 msgr2=0x7fcd140b8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.253+0000 7fcd19f96640 1 --2- 172.21.15.55:0/737777780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140b3c90 0x7fcd140b8010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097a0 tx=0x7fcd0402a1a0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.253+0000 7fcd19f96640 1 -- 172.21.15.55:0/737777780 shutdown_connections 2024-05-20T22:08:00.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.253+0000 7fcd19f96640 1 --2- 172.21.15.55:0/737777780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140b3c90 0x7fcd140b8010 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.253+0000 7fcd19f96640 1 -- 172.21.15.55:0/737777780 >> 172.21.15.55:0/737777780 conn(0x7fcd140b65f0 msgr2=0x7fcd140b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:00.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.253+0000 7fcd19f96640 1 -- 172.21.15.55:0/737777780 shutdown_connections 2024-05-20T22:08:00.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.253+0000 7fcd19f96640 1 -- 172.21.15.55:0/737777780 wait complete. 2024-05-20T22:08:00.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.255+0000 7fcd19f96640 1 Processor -- start 2024-05-20T22:08:00.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.255+0000 7fcd19f96640 1 -- start start 2024-05-20T22:08:00.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.256+0000 7fcd19f96640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd140b3c90 0x7fcd140ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:00.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.256+0000 7fcd19f96640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd140b8550 0x7fcd1415f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:00.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.256+0000 7fcd19f96640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140d3390 0x7fcd140cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.256+0000 7fcd19f96640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcd140b40a0 con 0x7fcd140d3390 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.256+0000 7fcd19f96640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1406b500 con 0x7fcd140b3c90 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.256+0000 7fcd19f96640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd140b4220 con 0x7fcd140b8550 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.256+0000 7fcd13fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140d3390 0x7fcd140cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.256+0000 7fcd13fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140d3390 0x7fcd140cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52792/0 (socket says 172.21.15.55:52792) 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.256+0000 7fcd13fff640 1 -- 172.21.15.55:0/3503843789 learned_addr learned my addr 172.21.15.55:0/3503843789 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd137fe640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd140b3c90 0x7fcd140ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd12ffd640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd140b8550 0x7fcd1415f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd13fff640 1 -- 172.21.15.55:0/3503843789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd140b8550 msgr2=0x7fcd1415f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd13fff640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd140b8550 0x7fcd1415f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd13fff640 1 -- 172.21.15.55:0/3503843789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd140b3c90 msgr2=0x7fcd140ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd13fff640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd140b3c90 0x7fcd140ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd13fff640 1 -- 172.21.15.55:0/3503843789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd140cbf90 con 0x7fcd140d3390 2024-05-20T22:08:00.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd12ffd640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd140b8550 0x7fcd1415f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:00.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd137fe640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd140b3c90 0x7fcd140ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:00.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd13fff640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140d3390 0x7fcd140cd030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c6b0 tx=0x7fcd0c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:00.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd0bfff640 1 -- 172.21.15.55:0/3503843789 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c003b70 con 0x7fcd140d3390 2024-05-20T22:08:00.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd0bfff640 1 -- 172.21.15.55:0/3503843789 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0c003d30 con 0x7fcd140d3390 2024-05-20T22:08:00.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd19f96640 1 -- 172.21.15.55:0/3503843789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd140cc270 con 0x7fcd140d3390 2024-05-20T22:08:00.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd0bfff640 1 -- 172.21.15.55:0/3503843789 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c005310 con 0x7fcd140d3390 2024-05-20T22:08:00.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.257+0000 7fcd19f96640 1 -- 172.21.15.55:0/3503843789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd140cc830 con 0x7fcd140d3390 2024-05-20T22:08:00.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.258+0000 7fcd19f96640 1 -- 172.21.15.55:0/3503843789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd140cbd00 con 0x7fcd140d3390 2024-05-20T22:08:00.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.259+0000 7fcd0bfff640 1 -- 172.21.15.55:0/3503843789 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd0c0054b0 con 0x7fcd140d3390 2024-05-20T22:08:00.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.259+0000 7fcd0bfff640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccf406be30 0x7fccf406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:00.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.260+0000 7fcd0bfff640 1 -- 172.21.15.55:0/3503843789 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcd0c08afd0 con 0x7fcd140d3390 2024-05-20T22:08:00.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.260+0000 7fcd137fe640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccf406be30 0x7fccf406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:00.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.260+0000 7fcd137fe640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccf406be30 0x7fccf406e340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fccfc007fd0 tx=0x7fccfc007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.264+0000 7fcd0bfff640 1 -- 172.21.15.55:0/3503843789 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcd0c0596d0 con 0x7fcd140d3390 2024-05-20T22:08:00.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.452+0000 7fcd19f96640 1 -- 172.21.15.55:0/3503843789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd14069090 con 0x7fcd140d3390 2024-05-20T22:08:00.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.453+0000 7fcd0bfff640 1 -- 172.21.15.55:0/3503843789 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fcd0c058fd0 con 0x7fcd140d3390 2024-05-20T22:08:00.454 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:00.455 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:00.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 -- 172.21.15.55:0/3503843789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccf406be30 msgr2=0x7fccf406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccf406be30 0x7fccf406e340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fccfc007fd0 tx=0x7fccfc007dc0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 -- 172.21.15.55:0/3503843789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140d3390 msgr2=0x7fcd140cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140d3390 0x7fcd140cd030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c6b0 tx=0x7fcd0c00cb70 comp rx=0 tx=0).stop 2024-05-20T22:08:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 -- 172.21.15.55:0/3503843789 shutdown_connections 2024-05-20T22:08:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccf406be30 0x7fccf406e340 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd140d3390 0x7fcd140cd030 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd140b8550 0x7fcd1415f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 --2- 172.21.15.55:0/3503843789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd140b3c90 0x7fcd140ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 -- 172.21.15.55:0/3503843789 >> 172.21.15.55:0/3503843789 conn(0x7fcd140b65f0 msgr2=0x7fcd140cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 -- 172.21.15.55:0/3503843789 shutdown_connections 2024-05-20T22:08:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.455+0000 7fcd19f96640 1 -- 172.21.15.55:0/3503843789 wait complete. 2024-05-20T22:08:00.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:07:53.028952+0000', 'modified': '2024-05-20T22:07:58.091673+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6835/2341014737', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6833', 'nonce': 2341014737}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 2341014737}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4288, 'qdb_cluster': [4288]} 2024-05-20T22:08:00.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T22:08:00.871 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-20T22:08:00.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.989+0000 7fd05cb6b640 1 Processor -- start 2024-05-20T22:08:00.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.989+0000 7fd05cb6b640 1 -- start start 2024-05-20T22:08:00.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd05cb6b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0580b4080 0x7fd05815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:00.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd05cb6b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd0580b35e0 0x7fd05815f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:00.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd05cb6b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd05806b500 con 0x7fd05815fd60 2024-05-20T22:08:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd05cb6b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fd058163960 con 0x7fd0580b4080 2024-05-20T22:08:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd05cb6b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd0580b3e10 con 0x7fd0580b35e0 2024-05-20T22:08:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd05659c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0580b4080 0x7fd05815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd05659c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0580b4080 0x7fd05815f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52794/0 (socket says 172.21.15.55:52794) 2024-05-20T22:08:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd05659c640 1 -- 172.21.15.55:0/3855574421 learned_addr learned my addr 172.21.15.55:0/3855574421 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd055d9b640 1 --2- 172.21.15.55:0/3855574421 >> v2:172.21.15.138:3300/0 conn(0x7fd0580b35e0 0x7fd05815f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd05659c640 1 -- 172.21.15.55:0/3855574421 >> v2:172.21.15.138:3300/0 conn(0x7fd0580b35e0 msgr2=0x7fd05815f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.991+0000 7fd05659c640 1 --2- 172.21.15.55:0/3855574421 >> v2:172.21.15.138:3300/0 conn(0x7fd0580b35e0 0x7fd05815f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.992+0000 7fd05659c640 1 -- 172.21.15.55:0/3855574421 >> v1:172.21.15.138:6789/0 conn(0x7fd05815fd60 legacy=0x7fd0580cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.992+0000 7fd05659c640 1 -- 172.21.15.55:0/3855574421 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0580b3b30 con 0x7fd0580b4080 2024-05-20T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.992+0000 7fd05659c640 1 --2- 172.21.15.55:0/3855574421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 0x7fd05815f2e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd0580d06e0 tx=0x7fd04802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1915bcc16009fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.992+0000 7fd05559a640 1 -- 172.21.15.55:0/3855574421 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048036070 con 0x7fd0580b4080 2024-05-20T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.992+0000 7fd05559a640 1 -- 172.21.15.55:0/3855574421 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd048031600 con 0x7fd0580b4080 2024-05-20T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.992+0000 7fd05559a640 1 -- 172.21.15.55:0/3855574421 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0480319d0 con 0x7fd0580b4080 2024-05-20T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.993+0000 7fd05cb6b640 1 -- 172.21.15.55:0/3855574421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 msgr2=0x7fd05815f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.993+0000 7fd05cb6b640 1 --2- 172.21.15.55:0/3855574421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 0x7fd05815f2e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd0580d06e0 tx=0x7fd04802a730 comp rx=0 tx=0).stop 2024-05-20T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.993+0000 7fd05cb6b640 1 -- 172.21.15.55:0/3855574421 shutdown_connections 2024-05-20T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.993+0000 7fd05cb6b640 1 --2- 172.21.15.55:0/3855574421 >> v2:172.21.15.138:3300/0 conn(0x7fd0580b35e0 0x7fd05815f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.993+0000 7fd05cb6b640 1 --2- 172.21.15.55:0/3855574421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 0x7fd05815f2e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.993+0000 7fd05cb6b640 1 -- 172.21.15.55:0/3855574421 >> 172.21.15.55:0/3855574421 conn(0x7fd0580d51b0 msgr2=0x7fd0580b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.993+0000 7fd05cb6b640 1 -- 172.21.15.55:0/3855574421 shutdown_connections 2024-05-20T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.993+0000 7fd05cb6b640 1 -- 172.21.15.55:0/3855574421 wait complete. 2024-05-20T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.995+0000 7fd05cb6b640 1 Processor -- start 2024-05-20T22:08:00.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.995+0000 7fd05cb6b640 1 -- start start 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd05cb6b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0580b35e0 0x7fd058151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd05cb6b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 0x7fd0580cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd05cb6b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd05815fd60 0x7fd0580ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd05cb6b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581635f0 con 0x7fd0580b4080 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd05cb6b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581604f0 con 0x7fd0580b35e0 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd05cb6b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd0580d42a0 con 0x7fd05815fd60 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd055d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 0x7fd0580cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd055d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 0x7fd0580cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52806/0 (socket says 172.21.15.55:52806) 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd055d9b640 1 -- 172.21.15.55:0/1297536248 learned_addr learned my addr 172.21.15.55:0/1297536248 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd05659c640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0580b35e0 0x7fd058151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd056d9d640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd05815fd60 0x7fd0580ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd055d9b640 1 -- 172.21.15.55:0/1297536248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd05815fd60 msgr2=0x7fd0580ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd055d9b640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd05815fd60 0x7fd0580ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd055d9b640 1 -- 172.21.15.55:0/1297536248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0580b35e0 msgr2=0x7fd058151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd055d9b640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0580b35e0 0x7fd058151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd055d9b640 1 -- 172.21.15.55:0/1297536248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd058151b60 con 0x7fd0580b4080 2024-05-20T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd055d9b640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 0x7fd0580cca40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd04000e870 tx=0x7fd04000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd04effd640 1 -- 172.21.15.55:0/1297536248 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040017070 con 0x7fd0580b4080 2024-05-20T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd04effd640 1 -- 172.21.15.55:0/1297536248 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd04000ee60 con 0x7fd0580b4080 2024-05-20T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.997+0000 7fd05cb6b640 1 -- 172.21.15.55:0/1297536248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0580cb9c0 con 0x7fd0580b4080 2024-05-20T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.998+0000 7fd04effd640 1 -- 172.21.15.55:0/1297536248 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04000cb00 con 0x7fd0580b4080 2024-05-20T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:00.998+0000 7fd05cb6b640 1 -- 172.21.15.55:0/1297536248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0580cbc80 con 0x7fd0580b4080 2024-05-20T22:08:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.000+0000 7fd04effd640 1 -- 172.21.15.55:0/1297536248 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd04001a070 con 0x7fd0580b4080 2024-05-20T22:08:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.000+0000 7fd04effd640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd03806be60 0x7fd03806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.000+0000 7fd04effd640 1 -- 172.21.15.55:0/1297536248 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd04008b320 con 0x7fd0580b4080 2024-05-20T22:08:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.000+0000 7fd05659c640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd03806be60 0x7fd03806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.001+0000 7fd05659c640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd03806be60 0x7fd03806e370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd048033000 tx=0x7fd048009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.001+0000 7fd063a0b740 1 Processor -- start 2024-05-20T22:08:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.001+0000 7fd063a0b740 1 -- start start 2024-05-20T22:08:01.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66c3270 0x556eb6a8e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 0x556eb6a8b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66a2590 0x556eb6a64360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd063a0b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556eb69c74d0 con 0x556eb66a2590 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd063a0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556eb696f3e0 con 0x556eb66c3270 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd063a0b740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556eb69c3490 con 0x556eb669a3b0 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66a2590 0x556eb6a64360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66a2590 0x556eb6a64360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52812/0 (socket says 172.21.15.55:52812) 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd056d9d640 1 -- 172.21.15.55:0/3332636329 learned_addr learned my addr 172.21.15.55:0/3332636329 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd055d9b640 1 --2- 172.21.15.55:0/3332636329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 0x556eb6a8b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd05659c640 1 --2- 172.21.15.55:0/3332636329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66c3270 0x556eb6a8e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd056d9d640 1 -- 172.21.15.55:0/3332636329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 msgr2=0x556eb6a8b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd056d9d640 1 --2- 172.21.15.55:0/3332636329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 0x556eb6a8b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd056d9d640 1 -- 172.21.15.55:0/3332636329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66c3270 msgr2=0x556eb6a8e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd056d9d640 1 --2- 172.21.15.55:0/3332636329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66c3270 0x556eb6a8e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd056d9d640 1 -- 172.21.15.55:0/3332636329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb66db2d0 con 0x556eb66a2590 2024-05-20T22:08:01.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd056d9d640 1 --2- 172.21.15.55:0/3332636329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66a2590 0x556eb6a64360 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd050001720 tx=0x7fd050007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5baa789cbb009902 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:01.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.003+0000 7fd037fff640 1 -- 172.21.15.55:0/3332636329 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500052e0 con 0x556eb66a2590 2024-05-20T22:08:01.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.004+0000 7fd037fff640 1 -- 172.21.15.55:0/3332636329 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd050018070 con 0x556eb66a2590 2024-05-20T22:08:01.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.004+0000 7fd037fff640 1 -- 172.21.15.55:0/3332636329 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05000f950 con 0x556eb66a2590 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.004+0000 7fd063a0b740 1 -- 172.21.15.55:0/3332636329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66a2590 msgr2=0x556eb6a64360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.004+0000 7fd063a0b740 1 --2- 172.21.15.55:0/3332636329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66a2590 0x556eb6a64360 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd050001720 tx=0x7fd050007db0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.004+0000 7fd063a0b740 1 -- 172.21.15.55:0/3332636329 shutdown_connections 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.004+0000 7fd063a0b740 1 --2- 172.21.15.55:0/3332636329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66a2590 0x556eb6a64360 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.004+0000 7fd063a0b740 1 --2- 172.21.15.55:0/3332636329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 0x556eb6a8b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.004+0000 7fd063a0b740 1 --2- 172.21.15.55:0/3332636329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66c3270 0x556eb6a8e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.004+0000 7fd063a0b740 1 -- 172.21.15.55:0/3332636329 >> 172.21.15.55:0/3332636329 conn(0x556eb66b73c0 msgr2=0x556eb6a63af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.004+0000 7fd063a0b740 1 -- 172.21.15.55:0/3332636329 shutdown_connections 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.005+0000 7fd063a0b740 1 -- 172.21.15.55:0/3332636329 wait complete. 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.005+0000 7fd063a0b740 1 -- start start 2024-05-20T22:08:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.005+0000 7fd063a0b740 1 Processor -- start 2024-05-20T22:08:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 0x556eb6aa8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 0x556eb6abbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a8b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd063a0b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556eb69828e0 con 0x556eb66c3270 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd063a0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556eb6a85e00 con 0x556eb66a2590 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd063a0b740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556eb6a8dd90 con 0x556eb669a3b0 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd063a0b740 10 client.0 _ll_register_callbacks cb 0x556eb694f9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a8b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a8b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52820/0 (socket says 172.21.15.55:52820) 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd056d9d640 1 -- 172.21.15.55:0/2288474309 learned_addr learned my addr 172.21.15.55:0/2288474309 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd055d9b640 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 0x556eb6abbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd05659c640 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 0x556eb6aa8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd056d9d640 1 -- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 msgr2=0x556eb6aa8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd056d9d640 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 0x556eb6aa8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd056d9d640 1 -- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 msgr2=0x556eb6abbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd056d9d640 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 0x556eb6abbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd056d9d640 1 -- 172.21.15.55:0/2288474309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb67a56b0 con 0x556eb66c3270 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.007+0000 7fd05659c640 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 0x556eb6aa8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd056d9d640 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a8b010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x556eb6aa0280 tx=0x7fd050000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd6d3ef628d9683 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd0357fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd0357fa640 1 -- 172.21.15.55:0/2288474309 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050010c70 con 0x556eb66c3270 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd0357fa640 1 -- 172.21.15.55:0/2288474309 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd050007030 con 0x556eb66c3270 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd063a0b740 1 -- 172.21.15.55:0/2288474309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd050018070 con 0x556eb66c3270 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd0357fa640 1 -- 172.21.15.55:0/2288474309 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fd05002e050 con 0x556eb66c3270 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd063a0b740 10 client.4437 fetch_fsmap learned FSMap version 7 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd063a0b740 1 -- 172.21.15.55:0/2288474309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x556eb6aa8260 con 0x556eb66c3270 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd0357fa640 1 -- 172.21.15.55:0/2288474309 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd050018070 con 0x556eb66c3270 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd0357fa640 1 -- 172.21.15.55:0/2288474309 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fd050031020 con 0x556eb66c3270 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd063a0b740 10 client.4437 fetch_fsmap finished waiting for FSMap version 7 2024-05-20T22:08:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd063a0b740 10 client.4437 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd063a0b740 20 client.4437 populate_metadata read hostname 'smithi055' 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x556eb6681e40 0x556eb6a91820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd063a0b740 4 client.4437 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd063a0b740 1 -- 172.21.15.55:0/2288474309 --> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eb69d4c00 con 0x556eb6681e40 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.008+0000 7fd05659c640 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x556eb6681e40 0x556eb6a91820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.009+0000 7fd0357fa640 10 client.4437 ms_handle_connect on v2:172.21.15.138:6833/2341014737 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.009+0000 7fd05659c640 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x556eb6681e40 0x556eb6a91820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d16d6b9b87cb643 server_cookie=d1eaa73b416b3183 in_seq=0 out_seq=0 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd0357fa640 1 -- 172.21.15.55:0/2288474309 <== mds.0 v2:172.21.15.138:6833/2341014737 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556eb69d4c00 con 0x556eb6681e40 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd0357fa640 10 client.4437 handle_command_reply: tid=0 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 client.4437 shutdown 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 -- 172.21.15.55:0/2288474309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 msgr2=0x556eb6a8b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a8b010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x556eb6aa0280 tx=0x7fd050000ed0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 -- 172.21.15.55:0/2288474309 shutdown_connections 2024-05-20T22:08:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x556eb6681e40 0x556eb6a91820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a8b010 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 0x556eb6abbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd0357fa640 0 client.4437 ms_handle_reset on v2:172.21.15.138:6833/2341014737 2024-05-20T22:08:01.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2288474309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb669a3b0 0x556eb6aa8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 -- 172.21.15.55:0/2288474309 >> 172.21.15.55:0/2288474309 conn(0x556eb66b73c0 msgr2=0x556eb6a5de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:01.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 -- 172.21.15.55:0/2288474309 shutdown_connections 2024-05-20T22:08:01.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.010+0000 7fd063a0b740 1 -- 172.21.15.55:0/2288474309 wait complete. 2024-05-20T22:08:01.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.011+0000 7fd063a0b740 20 client.4437 trim_cache size 0 max 16384 2024-05-20T22:08:01.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.030+0000 7fd063a0b740 1 Processor -- start 2024-05-20T22:08:01.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.030+0000 7fd063a0b740 1 -- start start 2024-05-20T22:08:01.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb669a3b0 0x556eb6a91770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 0x556eb6a62070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb66c3270 0x556eb6aa9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd063a0b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd058202210 con 0x556eb669a3b0 2024-05-20T22:08:01.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd063a0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0580b52f0 con 0x556eb66a2590 2024-05-20T22:08:01.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd063a0b740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd058201bf0 con 0x556eb66c3270 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd05659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb669a3b0 0x556eb6a91770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd05659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb669a3b0 0x556eb6a91770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52822/0 (socket says 172.21.15.55:52822) 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd05659c640 1 -- 172.21.15.55:0/2786926481 learned_addr learned my addr 172.21.15.55:0/2786926481 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd055d9b640 1 --2- 172.21.15.55:0/2786926481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 0x556eb6a62070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd05659c640 1 -- 172.21.15.55:0/2786926481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb66c3270 msgr2=0x556eb6aa9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd05659c640 1 --2- 172.21.15.55:0/2786926481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb66c3270 0x556eb6aa9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd05659c640 1 -- 172.21.15.55:0/2786926481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 msgr2=0x556eb6a62070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd05659c640 1 --2- 172.21.15.55:0/2786926481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 0x556eb6a62070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd05659c640 1 -- 172.21.15.55:0/2786926481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb6a77330 con 0x556eb669a3b0 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd055d9b640 1 --2- 172.21.15.55:0/2786926481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 0x556eb6a62070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.031+0000 7fd05659c640 1 --2- 172.21.15.55:0/2786926481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb669a3b0 0x556eb6a91770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd04802ad20 tx=0x7fd04802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5270362c4029c986 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd0377fe640 1 -- 172.21.15.55:0/2786926481 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048051070 con 0x556eb669a3b0 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd0377fe640 1 -- 172.21.15.55:0/2786926481 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd048037040 con 0x556eb669a3b0 2024-05-20T22:08:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd0377fe640 1 -- 172.21.15.55:0/2786926481 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048036040 con 0x556eb669a3b0 2024-05-20T22:08:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd063a0b740 1 -- 172.21.15.55:0/2786926481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb669a3b0 msgr2=0x556eb6a91770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2786926481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb669a3b0 0x556eb6a91770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd04802ad20 tx=0x7fd04802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd063a0b740 1 -- 172.21.15.55:0/2786926481 shutdown_connections 2024-05-20T22:08:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2786926481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb66c3270 0x556eb6aa9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2786926481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb66a2590 0x556eb6a62070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2786926481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb669a3b0 0x556eb6a91770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd063a0b740 1 -- 172.21.15.55:0/2786926481 >> 172.21.15.55:0/2786926481 conn(0x556eb66b73c0 msgr2=0x556eb6a59940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd063a0b740 1 -- 172.21.15.55:0/2786926481 shutdown_connections 2024-05-20T22:08:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.032+0000 7fd063a0b740 1 -- 172.21.15.55:0/2786926481 wait complete. 2024-05-20T22:08:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.033+0000 7fd063a0b740 1 -- start start 2024-05-20T22:08:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.033+0000 7fd063a0b740 1 Processor -- start 2024-05-20T22:08:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb669a3b0 0x556eb6aaa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb66a2590 0x556eb6b0ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd063a0b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a91770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd063a0b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556eb6a85c30 con 0x556eb66c3270 2024-05-20T22:08:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd063a0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556eb6a8b550 con 0x556eb669a3b0 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd063a0b740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556eb6a764e0 con 0x556eb66a2590 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd063a0b740 10 client.0 _ll_register_callbacks cb 0x7fd04802af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a91770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a91770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52830/0 (socket says 172.21.15.55:52830) 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd056d9d640 1 -- 172.21.15.55:0/2359364060 learned_addr learned my addr 172.21.15.55:0/2359364060 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd05659c640 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb669a3b0 0x556eb6aaa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd055d9b640 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb66a2590 0x556eb6b0ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd056d9d640 1 -- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb66a2590 msgr2=0x556eb6b0ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd056d9d640 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb66a2590 0x556eb6b0ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd056d9d640 1 -- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb669a3b0 msgr2=0x556eb6aaa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd056d9d640 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb669a3b0 0x556eb6aaa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd056d9d640 1 -- 172.21.15.55:0/2359364060 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb6ac3310 con 0x556eb66c3270 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd05659c640 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb669a3b0 0x556eb6aaa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:01.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd056d9d640 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a91770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd050001200 tx=0x7fd050001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63edca6d1153f941 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd0357fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd0357fa640 1 -- 172.21.15.55:0/2359364060 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050010480 con 0x556eb66c3270 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd0357fa640 1 -- 172.21.15.55:0/2359364060 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd050010620 con 0x556eb66c3270 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.035+0000 7fd063a0b740 1 -- 172.21.15.55:0/2359364060 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd048037040 con 0x556eb66c3270 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd0357fa640 1 -- 172.21.15.55:0/2359364060 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fd050001590 con 0x556eb66c3270 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd063a0b740 10 client.4443 fetch_fsmap learned FSMap version 7 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd063a0b740 1 -- 172.21.15.55:0/2359364060 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x556eb6a8eb40 con 0x556eb66c3270 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd0357fa640 1 -- 172.21.15.55:0/2359364060 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd048037040 con 0x556eb66c3270 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd0357fa640 1 -- 172.21.15.55:0/2359364060 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fd050031040 con 0x556eb66c3270 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd063a0b740 10 client.4443 fetch_fsmap finished waiting for FSMap version 7 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd063a0b740 10 client.4443 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd063a0b740 20 client.4443 populate_metadata read hostname 'smithi055' 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x556eb6abaef0 0x556eb6acab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd063a0b740 4 client.4443 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd063a0b740 1 -- 172.21.15.55:0/2359364060 --> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] -- command(tid 0: {"prefix": "status"}) -- 0x556eb69cef50 con 0x556eb6abaef0 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.036+0000 7fd05659c640 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x556eb6abaef0 0x556eb6acab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd0357fa640 10 client.4443 ms_handle_connect on v2:172.21.15.138:6833/2341014737 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd05659c640 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x556eb6abaef0 0x556eb6acab60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0d1fcc51118a2d server_cookie=ebe6bbdbc2d7640f in_seq=0 out_seq=0 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd0357fa640 1 -- 172.21.15.55:0/2359364060 <== mds.0 v2:172.21.15.138:6833/2341014737 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x556eb69cef50 con 0x556eb6abaef0 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd0357fa640 10 client.4443 handle_command_reply: tid=0 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 client.4443 shutdown 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 -- 172.21.15.55:0/2359364060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 msgr2=0x556eb6a91770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a91770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd050001200 tx=0x7fd050001080 comp rx=0 tx=0).stop 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 -- 172.21.15.55:0/2359364060 shutdown_connections 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x556eb6abaef0 0x556eb6acab60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556eb66c3270 0x556eb6a91770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556eb66a2590 0x556eb6b0ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd0357fa640 0 client.4443 ms_handle_reset on v2:172.21.15.138:6833/2341014737 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 --2- 172.21.15.55:0/2359364060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eb669a3b0 0x556eb6aaa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 -- 172.21.15.55:0/2359364060 >> 172.21.15.55:0/2359364060 conn(0x556eb66b73c0 msgr2=0x556eb6a59940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 -- 172.21.15.55:0/2359364060 shutdown_connections 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 1 -- 172.21.15.55:0/2359364060 wait complete. 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.037+0000 7fd063a0b740 20 client.4443 trim_cache size 0 max 16384 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "de8ca569-3fd1-4cb2-8bdc-99f367745927", 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-20T22:08:01.043 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 4288, 2024-05-20T22:08:01.044 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-20T22:08:01.044 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-20T22:08:01.044 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-20T22:08:01.044 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 7.3958319750000001, 2024-05-20T22:08:01.044 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 7, 2024-05-20T22:08:01.044 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 24, 2024-05-20T22:08:01.044 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 24, 2024-05-20T22:08:01.044 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 14.41381968 2024-05-20T22:08:01.044 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 -- 172.21.15.55:0/1297536248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd03806be60 msgr2=0x7fd03806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd03806be60 0x7fd03806e370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd048033000 tx=0x7fd048009000 comp rx=0 tx=0).stop 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 -- 172.21.15.55:0/1297536248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 msgr2=0x7fd0580cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 0x7fd0580cca40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd04000e870 tx=0x7fd04000ed30 comp rx=0 tx=0).stop 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 -- 172.21.15.55:0/1297536248 shutdown_connections 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd03806be60 0x7fd03806e370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd05815fd60 0x7fd0580ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0580b4080 0x7fd0580cca40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 --2- 172.21.15.55:0/1297536248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0580b35e0 0x7fd058151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 -- 172.21.15.55:0/1297536248 >> 172.21.15.55:0/1297536248 conn(0x7fd0580d51b0 msgr2=0x7fd058159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 -- 172.21.15.55:0/1297536248 shutdown_connections 2024-05-20T22:08:01.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.038+0000 7fd05cb6b640 1 -- 172.21.15.55:0/1297536248 wait complete. 2024-05-20T22:08:01.050 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T22:08:01.050 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-20T22:08:01.050 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T22:08:01.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.163+0000 7f8160c5f640 1 Processor -- start 2024-05-20T22:08:01.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.163+0000 7f8160c5f640 1 -- start start 2024-05-20T22:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.164+0000 7f8160c5f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f815c0b2ab0 0x7f815c162990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.164+0000 7f8160c5f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0cd290 0x7f815c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.164+0000 7f8160c5f640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f815c0b2350 con 0x7f815c0b10e0 2024-05-20T22:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.164+0000 7f8160c5f640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f815c162ed0 con 0x7f815c0b2ab0 2024-05-20T22:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.164+0000 7f8160c5f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f815c0b2fb0 con 0x7f815c0cd290 2024-05-20T22:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.164+0000 7f815a59c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f815c0b2ab0 0x7f815c162990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.164+0000 7f8159d9b640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f815c0b10e0 0x7f815c0b14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47348/0 (socket says 172.21.15.55:47348) 2024-05-20T22:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.164+0000 7f8159d9b640 1 -- 172.21.15.55:0/548683440 learned_addr learned my addr 172.21.15.55:0/548683440 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.164+0000 7f815a59c640 1 --2- 172.21.15.55:0/548683440 >> v2:172.21.15.55:3300/0 conn(0x7f815c0b2ab0 0x7f815c162990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52846/0 (socket says 172.21.15.55:52846) 2024-05-20T22:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.165+0000 7f815ad9d640 1 --2- 172.21.15.55:0/548683440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0cd290 0x7f815c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.165+0000 7f815a59c640 1 -- 172.21.15.55:0/548683440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0cd290 msgr2=0x7f815c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.165+0000 7f815a59c640 1 --2- 172.21.15.55:0/548683440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0cd290 0x7f815c0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.165+0000 7f815a59c640 1 -- 172.21.15.55:0/548683440 >> v1:172.21.15.55:6789/0 conn(0x7f815c0b10e0 legacy=0x7f815c0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.165+0000 7f815a59c640 1 -- 172.21.15.55:0/548683440 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c0b1d90 con 0x7f815c0b2ab0 2024-05-20T22:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.165+0000 7f815959a640 1 -- 172.21.15.55:0/548683440 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1060467143 0 0) 0x7f815c0b2350 con 0x7f815c0b10e0 2024-05-20T22:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.165+0000 7f815ad9d640 1 --2- 172.21.15.55:0/548683440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0cd290 0x7f815c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:01.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.165+0000 7f815a59c640 1 --2- 172.21.15.55:0/548683440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b2ab0 0x7f815c162990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f814c0097a0 tx=0x7f814c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55aa908f0b4bebf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:01.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.166+0000 7f815959a640 1 -- 172.21.15.55:0/548683440 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c037070 con 0x7f815c0b2ab0 2024-05-20T22:08:01.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.166+0000 7f815959a640 1 -- 172.21.15.55:0/548683440 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f814c02ae50 con 0x7f815c0b2ab0 2024-05-20T22:08:01.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.166+0000 7f815959a640 1 -- 172.21.15.55:0/548683440 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c032660 con 0x7f815c0b2ab0 2024-05-20T22:08:01.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.166+0000 7f8160c5f640 1 -- 172.21.15.55:0/548683440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b2ab0 msgr2=0x7f815c162990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.166+0000 7f8160c5f640 1 --2- 172.21.15.55:0/548683440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b2ab0 0x7f815c162990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f814c0097a0 tx=0x7f814c02a100 comp rx=0 tx=0).stop 2024-05-20T22:08:01.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.166+0000 7f8160c5f640 1 -- 172.21.15.55:0/548683440 shutdown_connections 2024-05-20T22:08:01.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.166+0000 7f8160c5f640 1 --2- 172.21.15.55:0/548683440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0cd290 0x7f815c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.166+0000 7f8160c5f640 1 --2- 172.21.15.55:0/548683440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b2ab0 0x7f815c162990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.166+0000 7f8160c5f640 1 -- 172.21.15.55:0/548683440 >> 172.21.15.55:0/548683440 conn(0x7f815c0b6580 msgr2=0x7f815c0b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:01.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.167+0000 7f8160c5f640 1 -- 172.21.15.55:0/548683440 shutdown_connections 2024-05-20T22:08:01.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.167+0000 7f8160c5f640 1 -- 172.21.15.55:0/548683440 wait complete. 2024-05-20T22:08:01.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.168+0000 7f8160c5f640 1 Processor -- start 2024-05-20T22:08:01.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.168+0000 7f8160c5f640 1 -- start start 2024-05-20T22:08:01.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f8160c5f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b10e0 0x7f815c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f8160c5f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0b2ab0 0x7f815c06c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f8160c5f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f815c0cd290 0x7f815c1e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f8160c5f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f815c0ce210 con 0x7f815c0b10e0 2024-05-20T22:08:01.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f8160c5f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f815c0ce090 con 0x7f815c0cd290 2024-05-20T22:08:01.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f8160c5f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f815c0ce390 con 0x7f815c0b2ab0 2024-05-20T22:08:01.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f815a59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b10e0 0x7f815c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f815a59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b10e0 0x7f815c155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52854/0 (socket says 172.21.15.55:52854) 2024-05-20T22:08:01.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f815a59c640 1 -- 172.21.15.55:0/2676761099 learned_addr learned my addr 172.21.15.55:0/2676761099 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f8159d9b640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0b2ab0 0x7f815c06c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f815ad9d640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f815c0cd290 0x7f815c1e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f815a59c640 1 -- 172.21.15.55:0/2676761099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0b2ab0 msgr2=0x7f815c06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f815a59c640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0b2ab0 0x7f815c06c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f815a59c640 1 -- 172.21.15.55:0/2676761099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f815c0cd290 msgr2=0x7f815c1e8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f815a59c640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f815c0cd290 0x7f815c1e8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f815a59c640 1 -- 172.21.15.55:0/2676761099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c0c8dc0 con 0x7f815c0b10e0 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.170+0000 7f815a59c640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b10e0 0x7f815c155110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f814c0098c0 tx=0x7f814c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.171+0000 7f814affd640 1 -- 172.21.15.55:0/2676761099 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c041020 con 0x7f815c0b10e0 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.171+0000 7f814affd640 1 -- 172.21.15.55:0/2676761099 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f814c03c870 con 0x7f815c0b10e0 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.171+0000 7f8160c5f640 1 -- 172.21.15.55:0/2676761099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c0c90a0 con 0x7f815c0b10e0 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.171+0000 7f814affd640 1 -- 172.21.15.55:0/2676761099 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c037040 con 0x7f815c0b10e0 2024-05-20T22:08:01.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.171+0000 7f8160c5f640 1 -- 172.21.15.55:0/2676761099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c0c9660 con 0x7f815c0b10e0 2024-05-20T22:08:01.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.172+0000 7f8160c5f640 1 -- 172.21.15.55:0/2676761099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815c09a770 con 0x7f815c0b10e0 2024-05-20T22:08:01.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.173+0000 7f814affd640 1 -- 172.21.15.55:0/2676761099 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f814c044020 con 0x7f815c0b10e0 2024-05-20T22:08:01.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.173+0000 7f814affd640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f813c06be30 0x7f813c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:01.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.173+0000 7f814affd640 1 -- 172.21.15.55:0/2676761099 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f814c0ab940 con 0x7f815c0b10e0 2024-05-20T22:08:01.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.173+0000 7f8159d9b640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f813c06be30 0x7f813c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:01.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.174+0000 7f8159d9b640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f813c06be30 0x7f813c06e340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f815c06d5a0 tx=0x7f8150007e40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:01.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.178+0000 7f814affd640 1 -- 172.21.15.55:0/2676761099 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f814c04b030 con 0x7f815c0b10e0 2024-05-20T22:08:01.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.398+0000 7f8160c5f640 1 -- 172.21.15.55:0/2676761099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f815c069090 con 0x7f815c0b10e0 2024-05-20T22:08:01.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.399+0000 7f814affd640 1 -- 172.21.15.55:0/2676761099 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f814c0798d0 con 0x7f815c0b10e0 2024-05-20T22:08:01.400 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 -- 172.21.15.55:0/2676761099 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f813c06be30 msgr2=0x7f813c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f813c06be30 0x7f813c06e340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f815c06d5a0 tx=0x7f8150007e40 comp rx=0 tx=0).stop 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 -- 172.21.15.55:0/2676761099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b10e0 msgr2=0x7f815c155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b10e0 0x7f815c155110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f814c0098c0 tx=0x7f814c0046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 -- 172.21.15.55:0/2676761099 shutdown_connections 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f813c06be30 0x7f813c06e340 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f815c0cd290 0x7f815c1e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f815c0b2ab0 0x7f815c06c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 --2- 172.21.15.55:0/2676761099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f815c0b10e0 0x7f815c155110 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 -- 172.21.15.55:0/2676761099 >> 172.21.15.55:0/2676761099 conn(0x7f815c0b6580 msgr2=0x7f815c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:01.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 -- 172.21.15.55:0/2676761099 shutdown_connections 2024-05-20T22:08:01.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01.401+0000 7f8160c5f640 1 -- 172.21.15.55:0/2676761099 wait complete. 2024-05-20T22:08:01.412 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T22:08:01.413 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T22:08:01.413 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-20T22:08:01.413 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-20T22:08:01.413 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-20T22:08:01.413 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T22:08:01.413 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T22:08:01.413 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T22:08:01.413 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-20T22:08:01.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:01.413 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-20T22:08:01.429 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-20T22:08:01.430 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T22:08:01.431 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:01.431 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-20T22:08:01.431 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T22:08:01.431 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42710sec preferred_lft 42710sec 2024-05-20T22:08:01.431 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-20T22:08:01.431 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:01.431 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:01.431 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:01.431 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-20T22:08:01.432 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-20T22:08:01.432 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:01.432 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:01.432 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add name ceph-brx type bridge 2024-05-20T22:08:01.432 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr flush dev ceph-brx 2024-05-20T22:08:01.432 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set ceph-brx up 2024-05-20T22:08:01.432 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-20T22:08:01.432 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:01.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:01.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:01.625 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T22:08:01.651 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-20T22:08:01.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:01.653 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-20T22:08:01.721 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-20T22:08:01.721 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T22:08:01.721 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T22:08:01.722 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T22:08:01.722 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T22:08:01.722 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:01.722 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:01.722 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T22:08:01.722 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T22:08:01.722 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T22:08:01.722 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:01.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:01.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:01.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:01.994 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-20T22:08:02.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:02.011 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list-id 2024-05-20T22:08:02.067 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:02.067 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:02.067 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T22:08:02.067 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-20T22:08:02.067 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:02.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:02.191 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-20T22:08:02.191 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:02.192 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:02.192 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-20T22:08:02.192 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-20T22:08:02.192 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-20T22:08:02.192 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-20T22:08:02.192 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-20T22:08:02.192 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:02.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:02.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:02.325 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:02.325 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:02.325 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-20T22:08:02.325 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set dev brx.0 master ceph-brx 2024-05-20T22:08:02.325 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:02.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:02.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:02.406 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-20T22:08:02.406 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:08:02.406 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-20T22:08:02.422 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-20T22:08:02.423 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:08:02.423 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-20T22:08:02.478 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:08:02.546 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:02.602 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:02.603 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:08:02.604 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:08:02.605 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:08:02.605 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:08:02.605 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T22:08:02.605 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T22:08:02.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:02.606 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:08:02.662 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-20T22:08:02.663 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-20T22:08:02.705 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:08:02.732 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:02.789 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:08:02.789 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:08:02.790 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:08:02.791 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:08:02.792 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:08:02.792 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T22:08:02.792 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T22:08:02.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:02.793 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:08:02.856 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi055.stderr:2024-05-20T22:08:02.854+0000 7f42c4126480 -1 init, newargv = 0x56283f0730a0 newargc=15 2024-05-20T22:08:02.856 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi055.stderr:ceph-fuse[44456]: starting ceph client 2024-05-20T22:08:02.864 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi055.stderr:ceph-fuse[44456]: starting fuse 2024-05-20T22:08:03.952 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:08:03.985 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:04.045 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:08:04.045 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:08:04.045 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:08:04.046 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:08:04.046 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:08:04.046 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:08:04.046 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:08:04.046 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:08:04.046 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:08:04.046 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:08:04.046 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:08:04.046 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:08:04.046 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:08:04.047 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:08:04.048 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:08:04.048 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:08:04.048 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T22:08:04.048 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T22:08:04.048 INFO:teuthology.orchestra.run.smithi055.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-20T22:08:04.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:04.048 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:08:04.104 INFO:teuthology.orchestra.run.smithi055.stdout:40 2024-05-20T22:08:04.104 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-20T22:08:04.104 DEBUG:teuthology.orchestra.run.smithi055:> sudo stdin-killer -- python3 -c ' 2024-05-20T22:08:04.104 DEBUG:teuthology.orchestra.run.smithi055:> import glob 2024-05-20T22:08:04.104 DEBUG:teuthology.orchestra.run.smithi055:> import re 2024-05-20T22:08:04.104 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> import subprocess 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> def _find_admin_socket(client_name): 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> files = glob.glob(asok_path) 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> # Given a non-glob path, it better be there 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> if "*" not in asok_path: 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> assert(len(files) == 1) 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> return files[0] 2024-05-20T22:08:04.105 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:04.106 DEBUG:teuthology.orchestra.run.smithi055:> for f in files: 2024-05-20T22:08:04.106 DEBUG:teuthology.orchestra.run.smithi055:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-20T22:08:04.106 DEBUG:teuthology.orchestra.run.smithi055:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-20T22:08:04.106 DEBUG:teuthology.orchestra.run.smithi055:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-20T22:08:04.106 DEBUG:teuthology.orchestra.run.smithi055:> contents = proc_f.read() 2024-05-20T22:08:04.106 DEBUG:teuthology.orchestra.run.smithi055:> if mountpoint in contents: 2024-05-20T22:08:04.106 DEBUG:teuthology.orchestra.run.smithi055:> return f 2024-05-20T22:08:04.106 DEBUG:teuthology.orchestra.run.smithi055:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-20T22:08:04.106 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:04.106 DEBUG:teuthology.orchestra.run.smithi055:> print(_find_admin_socket("client.admin")) 2024-05-20T22:08:04.107 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:08:04.234 INFO:teuthology.orchestra.run.smithi055.stdout:/var/run/ceph/ceph-client.admin.44456.asok 2024-05-20T22:08:04.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:04.245 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44456.asok 2024-05-20T22:08:04.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:04.246 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44456.asok status 2024-05-20T22:08:04.395 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:08:04.395 INFO:teuthology.orchestra.run.smithi055.stdout: "metadata": { 2024-05-20T22:08:04.395 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_sha1": "d2b507c724ef6c8295ee0a35d185cdab167fd61d", 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "entity_id": "admin", 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": "smithi055", 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "pid": "44456", 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "root": "/" 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_count": 0, 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_pinned_count": 0, 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 4461, 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "inst": { 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "name": { 2024-05-20T22:08:04.396 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "client", 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "num": 4461 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.1:0", 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 2223020694 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.1:0", 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 2223020694 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "inst_str": "client.4461 192.168.0.1:0/2223020694", 2024-05-20T22:08:04.397 INFO:teuthology.orchestra.run.smithi055.stdout: "addr_str": "192.168.0.1:0/2223020694", 2024-05-20T22:08:04.398 INFO:teuthology.orchestra.run.smithi055.stdout: "inode_count": 1, 2024-05-20T22:08:04.398 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_epoch": 7, 2024-05-20T22:08:04.398 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch": 24, 2024-05-20T22:08:04.398 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch_barrier": 0, 2024-05-20T22:08:04.398 INFO:teuthology.orchestra.run.smithi055.stdout: "blocklisted": false, 2024-05-20T22:08:04.398 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs" 2024-05-20T22:08:04.398 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:08:04.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:04.404 DEBUG:teuthology.orchestra.run.smithi055:> stat --file-system '--printf=%T 2024-05-20T22:08:04.404 DEBUG:teuthology.orchestra.run.smithi055:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-20T22:08:04.420 INFO:teuthology.orchestra.run.smithi055.stdout:fuseblk 2024-05-20T22:08:04.421 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-20T22:08:04.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:04.421 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-20T22:08:04.487 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-20T22:08:04.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T22:08:04.598 INFO:teuthology.orchestra.run.smithi055.stdout:# file: . 2024-05-20T22:08:04.599 INFO:teuthology.orchestra.run.smithi055.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-20T22:08:04.599 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:04.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:04.605 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:08:04.606 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:04.623 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.776+0000 7f4b678a2640 1 Processor -- start 2024-05-20T22:08:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.777+0000 7f4b678a2640 1 -- start start 2024-05-20T22:08:04.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b678a2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4b600b3b30 0x7f4b600b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:04.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b678a2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 0x7f4b6006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:04.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b678a2640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b6006c5c0 0x7f4b6006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:04.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b678a2640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f4b600d3dd0 con 0x7f4b6006c5c0 2024-05-20T22:08:04.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b678a2640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4b600d40b0 con 0x7f4b600b3b30 2024-05-20T22:08:04.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b678a2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b600d4390 con 0x7f4b6015f100 2024-05-20T22:08:04.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b65e3f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b6006c5c0 0x7f4b6006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:04.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b65e3f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b6006c5c0 0x7f4b6006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52860/0 (socket says 172.21.15.55:52860) 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b65e3f640 1 -- 172.21.15.55:0/1707518233 learned_addr learned my addr 172.21.15.55:0/1707518233 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b6563e640 1 --2- 172.21.15.55:0/1707518233 >> v2:172.21.15.138:3300/0 conn(0x7f4b600b3b30 0x7f4b600b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b64e3d640 1 --2- 172.21.15.55:0/1707518233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 0x7f4b6006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b65e3f640 1 -- 172.21.15.55:0/1707518233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 msgr2=0x7f4b6006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b65e3f640 1 --2- 172.21.15.55:0/1707518233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 0x7f4b6006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b65e3f640 1 -- 172.21.15.55:0/1707518233 >> v2:172.21.15.138:3300/0 conn(0x7f4b600b3b30 msgr2=0x7f4b600b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.778+0000 7f4b65e3f640 1 --2- 172.21.15.55:0/1707518233 >> v2:172.21.15.138:3300/0 conn(0x7f4b600b3b30 0x7f4b600b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.779+0000 7f4b65e3f640 1 -- 172.21.15.55:0/1707518233 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6006b5a0 con 0x7f4b6006c5c0 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.779+0000 7f4b64e3d640 1 --2- 172.21.15.55:0/1707518233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 0x7f4b6006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.779+0000 7f4b65e3f640 1 --2- 172.21.15.55:0/1707518233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 0x7f4b6006c9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b600d27f0 tx=0x7f4b5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1ce2cfc66d7579 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.779+0000 7f4b57fff640 1 -- 172.21.15.55:0/1707518233 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58037070 con 0x7f4b6006c5c0 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.779+0000 7f4b57fff640 1 -- 172.21.15.55:0/1707518233 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b58032440 con 0x7f4b6006c5c0 2024-05-20T22:08:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.779+0000 7f4b57fff640 1 -- 172.21.15.55:0/1707518233 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58032810 con 0x7f4b6006c5c0 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.780+0000 7f4b678a2640 1 -- 172.21.15.55:0/1707518233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 msgr2=0x7f4b6006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.780+0000 7f4b678a2640 1 --2- 172.21.15.55:0/1707518233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 0x7f4b6006c9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b600d27f0 tx=0x7f4b5802a1e0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.780+0000 7f4b678a2640 1 -- 172.21.15.55:0/1707518233 shutdown_connections 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.780+0000 7f4b678a2640 1 --2- 172.21.15.55:0/1707518233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 0x7f4b6006c9d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.780+0000 7f4b678a2640 1 --2- 172.21.15.55:0/1707518233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 0x7f4b6006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.780+0000 7f4b678a2640 1 --2- 172.21.15.55:0/1707518233 >> v2:172.21.15.138:3300/0 conn(0x7f4b600b3b30 0x7f4b600b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.780+0000 7f4b678a2640 1 -- 172.21.15.55:0/1707518233 >> 172.21.15.55:0/1707518233 conn(0x7f4b600d51d0 msgr2=0x7f4b600b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.780+0000 7f4b678a2640 1 -- 172.21.15.55:0/1707518233 shutdown_connections 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.780+0000 7f4b678a2640 1 -- 172.21.15.55:0/1707518233 wait complete. 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.782+0000 7f4b678a2640 1 Processor -- start 2024-05-20T22:08:04.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.782+0000 7f4b678a2640 1 -- start start 2024-05-20T22:08:04.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.783+0000 7f4b678a2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 0x7f4b600cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:04.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.783+0000 7f4b678a2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b600b3b30 0x7f4b6014bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.783+0000 7f4b678a2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 0x7f4b6014c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.783+0000 7f4b678a2640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60162b70 con 0x7f4b6006c5c0 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.783+0000 7f4b678a2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6006bb60 con 0x7f4b600b3b30 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.783+0000 7f4b678a2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b60162cf0 con 0x7f4b6015f100 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.783+0000 7f4b6563e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 0x7f4b600cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.783+0000 7f4b6563e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 0x7f4b600cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52862/0 (socket says 172.21.15.55:52862) 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.783+0000 7f4b6563e640 1 -- 172.21.15.55:0/2936850017 learned_addr learned my addr 172.21.15.55:0/2936850017 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b64e3d640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b600b3b30 0x7f4b6014bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b65e3f640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 0x7f4b6014c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b6563e640 1 -- 172.21.15.55:0/2936850017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 msgr2=0x7f4b6014c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b6563e640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 0x7f4b6014c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b6563e640 1 -- 172.21.15.55:0/2936850017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b600b3b30 msgr2=0x7f4b6014bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:04.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b6563e640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b600b3b30 0x7f4b6014bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b6563e640 1 -- 172.21.15.55:0/2936850017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b60149b70 con 0x7f4b6006c5c0 2024-05-20T22:08:04.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b6563e640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 0x7f4b600cfe70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c007e60 tx=0x7f4b5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:04.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b55ffb640 1 -- 172.21.15.55:0/2936850017 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c01a070 con 0x7f4b6006c5c0 2024-05-20T22:08:04.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b55ffb640 1 -- 172.21.15.55:0/2936850017 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b5c003990 con 0x7f4b6006c5c0 2024-05-20T22:08:04.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b678a2640 1 -- 172.21.15.55:0/2936850017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b60149e50 con 0x7f4b6006c5c0 2024-05-20T22:08:04.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b55ffb640 1 -- 172.21.15.55:0/2936850017 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c003d60 con 0x7f4b6006c5c0 2024-05-20T22:08:04.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.784+0000 7f4b678a2640 1 -- 172.21.15.55:0/2936850017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6014a410 con 0x7f4b6006c5c0 2024-05-20T22:08:04.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.786+0000 7f4b678a2640 1 -- 172.21.15.55:0/2936850017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b600b9b00 con 0x7f4b6006c5c0 2024-05-20T22:08:04.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.786+0000 7f4b55ffb640 1 -- 172.21.15.55:0/2936850017 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b5c01d070 con 0x7f4b6006c5c0 2024-05-20T22:08:04.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.787+0000 7f4b55ffb640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4b4406be30 0x7f4b4406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:04.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.787+0000 7f4b55ffb640 1 -- 172.21.15.55:0/2936850017 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4b5c08b280 con 0x7f4b6006c5c0 2024-05-20T22:08:04.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.787+0000 7f4b64e3d640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4b4406be30 0x7f4b4406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:04.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.787+0000 7f4b64e3d640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4b4406be30 0x7f4b4406e340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b60067b10 tx=0x7f4b50008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:04.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.792+0000 7f4b55ffb640 1 -- 172.21.15.55:0/2936850017 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4b600b9b00 con 0x7f4b6006c5c0 2024-05-20T22:08:04.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.979+0000 7f4b678a2640 1 -- 172.21.15.55:0/2936850017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b60069090 con 0x7f4b6006c5c0 2024-05-20T22:08:04.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.981+0000 7f4b55ffb640 1 -- 172.21.15.55:0/2936850017 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f4b5c0592a0 con 0x7f4b6006c5c0 2024-05-20T22:08:04.983 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:04.983 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:04.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 -- 172.21.15.55:0/2936850017 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4b4406be30 msgr2=0x7f4b4406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:04.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4b4406be30 0x7f4b4406e340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b60067b10 tx=0x7f4b50008040 comp rx=0 tx=0).stop 2024-05-20T22:08:04.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 -- 172.21.15.55:0/2936850017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 msgr2=0x7f4b600cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:04.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 0x7f4b600cfe70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c007e60 tx=0x7f4b5c00e4b0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 -- 172.21.15.55:0/2936850017 shutdown_connections 2024-05-20T22:08:04.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4b4406be30 0x7f4b4406e340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b6015f100 0x7f4b6014c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b600b3b30 0x7f4b6014bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 --2- 172.21.15.55:0/2936850017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b6006c5c0 0x7f4b600cfe70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:04.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 -- 172.21.15.55:0/2936850017 >> 172.21.15.55:0/2936850017 conn(0x7f4b600d51d0 msgr2=0x7f4b600b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:04.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 -- 172.21.15.55:0/2936850017 shutdown_connections 2024-05-20T22:08:04.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:04.983+0000 7f4b678a2640 1 -- 172.21.15.55:0/2936850017 wait complete. 2024-05-20T22:08:04.996 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:04.997 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:05.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.114+0000 7f31e4f85640 1 Processor -- start 2024-05-20T22:08:05.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.114+0000 7f31e4f85640 1 -- start start 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.115+0000 7f31e4f85640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f31e015d170 0x7f31e00cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.115+0000 7f31e4f85640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e00d1e00 0x7f31e0166b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.115+0000 7f31e4f85640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e00cf000 con 0x7f31e00ce220 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.115+0000 7f31e4f85640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f31e00ce760 con 0x7f31e015d170 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.115+0000 7f31e4f85640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f31e00cea40 con 0x7f31e00d1e00 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.115+0000 7f31d5d9b640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f31e00ce220 0x7f31e00d16f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47362/0 (socket says 172.21.15.55:47362) 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.115+0000 7f31d5d9b640 1 -- 172.21.15.55:0/2482339388 learned_addr learned my addr 172.21.15.55:0/2482339388 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.116+0000 7f31ded9d640 1 --2- 172.21.15.55:0/2482339388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e00d1e00 0x7f31e0166b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.116+0000 7f31de59c640 1 --2- 172.21.15.55:0/2482339388 >> v2:172.21.15.138:3300/0 conn(0x7f31e015d170 0x7f31e00cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.116+0000 7f31ddd9b640 1 -- 172.21.15.55:0/2482339388 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3259592177 0 0) 0x7f31e00cf000 con 0x7f31e00ce220 2024-05-20T22:08:05.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.116+0000 7f31ddd9b640 1 -- 172.21.15.55:0/2482339388 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31cc003520 con 0x7f31e00ce220 2024-05-20T22:08:05.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.116+0000 7f31de59c640 1 -- 172.21.15.55:0/2482339388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e00d1e00 msgr2=0x7f31e0166b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:05.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.116+0000 7f31ddd9b640 1 -- 172.21.15.55:0/2482339388 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3066687794 0 0) 0x7f31cc003520 con 0x7f31e00ce220 2024-05-20T22:08:05.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.116+0000 7f31de59c640 1 --2- 172.21.15.55:0/2482339388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e00d1e00 0x7f31e0166b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.116+0000 7f31de59c640 1 -- 172.21.15.55:0/2482339388 >> v1:172.21.15.55:6789/0 conn(0x7f31e00ce220 legacy=0x7f31e00d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.116+0000 7f31de59c640 1 -- 172.21.15.55:0/2482339388 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e00ced20 con 0x7f31e015d170 2024-05-20T22:08:05.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.116+0000 7f31ded9d640 1 --2- 172.21.15.55:0/2482339388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e00d1e00 0x7f31e0166b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:05.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.117+0000 7f31de59c640 1 --2- 172.21.15.55:0/2482339388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e015d170 0x7f31e00cdce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f31d00097a0 tx=0x7f31d002a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9d75ef26ddb5c02 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:05.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.117+0000 7f31ddd9b640 1 -- 172.21.15.55:0/2482339388 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0037070 con 0x7f31e015d170 2024-05-20T22:08:05.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.117+0000 7f31ddd9b640 1 -- 172.21.15.55:0/2482339388 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d002aa90 con 0x7f31e015d170 2024-05-20T22:08:05.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.117+0000 7f31ddd9b640 1 -- 172.21.15.55:0/2482339388 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8001670 con 0x7f31e015d170 2024-05-20T22:08:05.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.118+0000 7f31e4f85640 1 -- 172.21.15.55:0/2482339388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e015d170 msgr2=0x7f31e00cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.118+0000 7f31e4f85640 1 --2- 172.21.15.55:0/2482339388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e015d170 0x7f31e00cdce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f31d00097a0 tx=0x7f31d002a000 comp rx=0 tx=0).stop 2024-05-20T22:08:05.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.118+0000 7f31e4f85640 1 -- 172.21.15.55:0/2482339388 shutdown_connections 2024-05-20T22:08:05.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.118+0000 7f31e4f85640 1 --2- 172.21.15.55:0/2482339388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e00d1e00 0x7f31e0166b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.118+0000 7f31e4f85640 1 --2- 172.21.15.55:0/2482339388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e015d170 0x7f31e00cdce0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.118+0000 7f31e4f85640 1 -- 172.21.15.55:0/2482339388 >> 172.21.15.55:0/2482339388 conn(0x7f31e00736f0 msgr2=0x7f31e00b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:05.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.118+0000 7f31e4f85640 1 -- 172.21.15.55:0/2482339388 shutdown_connections 2024-05-20T22:08:05.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.118+0000 7f31e4f85640 1 -- 172.21.15.55:0/2482339388 wait complete. 2024-05-20T22:08:05.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.120+0000 7f31e4f85640 1 Processor -- start 2024-05-20T22:08:05.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.120+0000 7f31e4f85640 1 -- start start 2024-05-20T22:08:05.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31e4f85640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e00ce220 0x7f31e015ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31e4f85640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e015d170 0x7f31e014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31e4f85640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e014e840 0x7f31e014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31e4f85640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f31e006d530 con 0x7f31e00ce220 2024-05-20T22:08:05.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31e4f85640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31e006d3b0 con 0x7f31e014e840 2024-05-20T22:08:05.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31e4f85640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f31e006d6b0 con 0x7f31e015d170 2024-05-20T22:08:05.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31de59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e00ce220 0x7f31e015ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31de59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e00ce220 0x7f31e015ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52866/0 (socket says 172.21.15.55:52866) 2024-05-20T22:08:05.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31de59c640 1 -- 172.21.15.55:0/3341379239 learned_addr learned my addr 172.21.15.55:0/3341379239 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31d5d9b640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e015d170 0x7f31e014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.121+0000 7f31ded9d640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e014e840 0x7f31e014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31de59c640 1 -- 172.21.15.55:0/3341379239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e015d170 msgr2=0x7f31e014db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31de59c640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e015d170 0x7f31e014db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31de59c640 1 -- 172.21.15.55:0/3341379239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e014e840 msgr2=0x7f31e014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31de59c640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e014e840 0x7f31e014e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31de59c640 1 -- 172.21.15.55:0/3341379239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e015d820 con 0x7f31e00ce220 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31de59c640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e00ce220 0x7f31e015ff60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f31d0035040 tx=0x7f31d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31d77fe640 1 -- 172.21.15.55:0/3341379239 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0037050 con 0x7f31e00ce220 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31d77fe640 1 -- 172.21.15.55:0/3341379239 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d0043070 con 0x7f31e00ce220 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31e4f85640 1 -- 172.21.15.55:0/3341379239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e015db00 con 0x7f31e00ce220 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31d77fe640 1 -- 172.21.15.55:0/3341379239 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0006e10 con 0x7f31e00ce220 2024-05-20T22:08:05.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.122+0000 7f31e4f85640 1 -- 172.21.15.55:0/3341379239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e014bda0 con 0x7f31e00ce220 2024-05-20T22:08:05.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.123+0000 7f31e4f85640 1 -- 172.21.15.55:0/3341379239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e0069090 con 0x7f31e00ce220 2024-05-20T22:08:05.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.124+0000 7f31d77fe640 1 -- 172.21.15.55:0/3341379239 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31d003ca10 con 0x7f31e00ce220 2024-05-20T22:08:05.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.124+0000 7f31d77fe640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c006be30 0x7f31c006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.124+0000 7f31d77fe640 1 -- 172.21.15.55:0/3341379239 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f31d00abbb0 con 0x7f31e00ce220 2024-05-20T22:08:05.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.124+0000 7f31d5d9b640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c006be30 0x7f31c006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.125+0000 7f31d5d9b640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c006be30 0x7f31c006e340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31c8005880 tx=0x7f31c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:05.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.129+0000 7f31d77fe640 1 -- 172.21.15.55:0/3341379239 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f31d0079bd0 con 0x7f31e00ce220 2024-05-20T22:08:05.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.312+0000 7f31e4f85640 1 -- 172.21.15.55:0/3341379239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31e00b6880 con 0x7f31e00ce220 2024-05-20T22:08:05.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.313+0000 7f31d77fe640 1 -- 172.21.15.55:0/3341379239 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f31d007ded0 con 0x7f31e00ce220 2024-05-20T22:08:05.315 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 -- 172.21.15.55:0/3341379239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c006be30 msgr2=0x7f31c006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c006be30 0x7f31c006e340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31c8005880 tx=0x7f31c8009040 comp rx=0 tx=0).stop 2024-05-20T22:08:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 -- 172.21.15.55:0/3341379239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e00ce220 msgr2=0x7f31e015ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e00ce220 0x7f31e015ff60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f31d0035040 tx=0x7f31d00046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 -- 172.21.15.55:0/3341379239 shutdown_connections 2024-05-20T22:08:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c006be30 0x7f31c006e340 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e014e840 0x7f31e014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e015d170 0x7f31e014db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 --2- 172.21.15.55:0/3341379239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e00ce220 0x7f31e015ff60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 -- 172.21.15.55:0/3341379239 >> 172.21.15.55:0/3341379239 conn(0x7f31e00736f0 msgr2=0x7f31e00b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 -- 172.21.15.55:0/3341379239 shutdown_connections 2024-05-20T22:08:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.315+0000 7f31e4f85640 1 -- 172.21.15.55:0/3341379239 wait complete. 2024-05-20T22:08:05.328 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:05.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.444+0000 7fbe13779640 1 Processor -- start 2024-05-20T22:08:05.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.444+0000 7fbe13779640 1 -- start start 2024-05-20T22:08:05.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.445+0000 7fbe13779640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe0c0d32a0 0x7fbe0c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.445+0000 7fbe13779640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 0x7fbe0c06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe13779640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe0c0d3780 con 0x7fbe0c0cf690 2024-05-20T22:08:05.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe13779640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbe0c0d3a60 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe13779640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0c0d3d40 con 0x7fbe0c0d01b0 2024-05-20T22:08:05.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe11d16640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 0x7fbe0c06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe11515640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe0c0d32a0 0x7fbe0c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe11d16640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 0x7fbe0c06b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:39592/0 (socket says 172.21.15.55:39592) 2024-05-20T22:08:05.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe10d14640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fbe0c0cf690 0x7fbe0c0cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:48800/0 (socket says 172.21.15.55:48800) 2024-05-20T22:08:05.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe11515640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe0c0d32a0 0x7fbe0c15f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:47106/0 (socket says 172.21.15.55:47106) 2024-05-20T22:08:05.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe11d16640 1 -- 172.21.15.55:0/719629885 learned_addr learned my addr 172.21.15.55:0/719629885 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:05.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe11d16640 1 -- 172.21.15.55:0/719629885 >> v2:172.21.15.138:3300/0 conn(0x7fbe0c0d32a0 msgr2=0x7fbe0c15f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:05.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe11d16640 1 --2- 172.21.15.55:0/719629885 >> v2:172.21.15.138:3300/0 conn(0x7fbe0c0d32a0 0x7fbe0c15f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe11d16640 1 -- 172.21.15.55:0/719629885 >> v1:172.21.15.138:6789/0 conn(0x7fbe0c0cf690 legacy=0x7fbe0c0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe11d16640 1 -- 172.21.15.55:0/719629885 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c0d4020 con 0x7fbe0c0d01b0 2024-05-20T22:08:05.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.446+0000 7fbe11515640 1 --2- 172.21.15.55:0/719629885 >> v2:172.21.15.138:3300/0 conn(0x7fbe0c0d32a0 0x7fbe0c15f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:05.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.447+0000 7fbe11d16640 1 --2- 172.21.15.55:0/719629885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 0x7fbe0c06b500 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009870 tx=0x7fbe0402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32fc6c80687dd63e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.447+0000 7fbdfffff640 1 -- 172.21.15.55:0/719629885 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04037070 con 0x7fbe0c0d01b0 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.447+0000 7fbdfffff640 1 -- 172.21.15.55:0/719629885 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe04032440 con 0x7fbe0c0d01b0 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.447+0000 7fbdfffff640 1 -- 172.21.15.55:0/719629885 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04032810 con 0x7fbe0c0d01b0 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.448+0000 7fbe13779640 1 -- 172.21.15.55:0/719629885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 msgr2=0x7fbe0c06b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.448+0000 7fbe13779640 1 --2- 172.21.15.55:0/719629885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 0x7fbe0c06b500 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009870 tx=0x7fbe0402a270 comp rx=0 tx=0).stop 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.448+0000 7fbe13779640 1 -- 172.21.15.55:0/719629885 shutdown_connections 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.448+0000 7fbe13779640 1 --2- 172.21.15.55:0/719629885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 0x7fbe0c06b500 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.448+0000 7fbe13779640 1 --2- 172.21.15.55:0/719629885 >> v2:172.21.15.138:3300/0 conn(0x7fbe0c0d32a0 0x7fbe0c15f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.448+0000 7fbe13779640 1 -- 172.21.15.55:0/719629885 >> 172.21.15.55:0/719629885 conn(0x7fbe0c0b6de0 msgr2=0x7fbe0c0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.448+0000 7fbe13779640 1 -- 172.21.15.55:0/719629885 shutdown_connections 2024-05-20T22:08:05.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.448+0000 7fbe13779640 1 -- 172.21.15.55:0/719629885 wait complete. 2024-05-20T22:08:05.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.450+0000 7fbe13779640 1 Processor -- start 2024-05-20T22:08:05.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.450+0000 7fbe13779640 1 -- start start 2024-05-20T22:08:05.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.451+0000 7fbe13779640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c0cf690 0x7fbe0c14e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.451+0000 7fbe13779640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 0x7fbe0c1f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.451+0000 7fbe13779640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c0d32a0 0x7fbe0c1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.451+0000 7fbe13779640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c0d4820 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.451+0000 7fbe13779640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c0b4350 con 0x7fbe0c0cf690 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.451+0000 7fbe13779640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0c0cb2d0 con 0x7fbe0c0d01b0 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.451+0000 7fbe11d16640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c0d32a0 0x7fbe0c1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.451+0000 7fbe11d16640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c0d32a0 0x7fbe0c1f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52874/0 (socket says 172.21.15.55:52874) 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe11d16640 1 -- 172.21.15.55:0/1300063118 learned_addr learned my addr 172.21.15.55:0/1300063118 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe10d14640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 0x7fbe0c1f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe11515640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c0cf690 0x7fbe0c14e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe11d16640 1 -- 172.21.15.55:0/1300063118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 msgr2=0x7fbe0c1f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe11d16640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 0x7fbe0c1f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe11d16640 1 -- 172.21.15.55:0/1300063118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c0cf690 msgr2=0x7fbe0c14e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe11d16640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c0cf690 0x7fbe0c14e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe11d16640 1 -- 172.21.15.55:0/1300063118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c1600b0 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe11d16640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c0d32a0 0x7fbe0c1f5a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009870 tx=0x7fbe040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbdfdffb640 1 -- 172.21.15.55:0/1300063118 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04041020 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbdfdffb640 1 -- 172.21.15.55:0/1300063118 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0403d640 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe13779640 1 -- 172.21.15.55:0/1300063118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c15f2f0 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbdfdffb640 1 -- 172.21.15.55:0/1300063118 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04037040 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.452+0000 7fbe13779640 1 -- 172.21.15.55:0/1300063118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c15f8b0 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.454+0000 7fbe13779640 1 -- 172.21.15.55:0/1300063118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0c1f6060 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.454+0000 7fbdfdffb640 1 -- 172.21.15.55:0/1300063118 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe04044020 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.455+0000 7fbdfdffb640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbdf006be30 0x7fbdf006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.455+0000 7fbdfdffb640 1 -- 172.21.15.55:0/1300063118 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbe040ab7d0 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.455+0000 7fbe11515640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbdf006be30 0x7fbdf006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.455+0000 7fbe11515640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbdf006be30 0x7fbdf006e340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009bd0 tx=0x7fbe0800b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.460+0000 7fbdfdffb640 1 -- 172.21.15.55:0/1300063118 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbe0404b030 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.642+0000 7fbe13779640 1 -- 172.21.15.55:0/1300063118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbe0c069090 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.642+0000 7fbdfdffb640 1 -- 172.21.15.55:0/1300063118 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fbe04079770 con 0x7fbe0c0d32a0 2024-05-20T22:08:05.646 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:05.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 -- 172.21.15.55:0/1300063118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbdf006be30 msgr2=0x7fbdf006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbdf006be30 0x7fbdf006e340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009bd0 tx=0x7fbe0800b040 comp rx=0 tx=0).stop 2024-05-20T22:08:05.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 -- 172.21.15.55:0/1300063118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c0d32a0 msgr2=0x7fbe0c1f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c0d32a0 0x7fbe0c1f5a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009870 tx=0x7fbe040046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 -- 172.21.15.55:0/1300063118 shutdown_connections 2024-05-20T22:08:05.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbdf006be30 0x7fbdf006e340 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c0d32a0 0x7fbe0c1f5a60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe0c0d01b0 0x7fbe0c1f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 --2- 172.21.15.55:0/1300063118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c0cf690 0x7fbe0c14e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 -- 172.21.15.55:0/1300063118 >> 172.21.15.55:0/1300063118 conn(0x7fbe0c0b6de0 msgr2=0x7fbe0c0b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:05.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 -- 172.21.15.55:0/1300063118 shutdown_connections 2024-05-20T22:08:05.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.644+0000 7fbe13779640 1 -- 172.21.15.55:0/1300063118 wait complete. 2024-05-20T22:08:05.657 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:05.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.775+0000 7f1318d1a640 1 Processor -- start 2024-05-20T22:08:05.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.775+0000 7f1318d1a640 1 -- start start 2024-05-20T22:08:05.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.776+0000 7f1318d1a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13140b2980 0x7f13140cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.776+0000 7f1318d1a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2d90 0x7f13140cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.776+0000 7f1318d1a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f131415f610 con 0x7f13140cd850 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.776+0000 7f1318d1a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f13140cf040 con 0x7f13140b2980 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.776+0000 7f1318d1a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f131409b160 con 0x7f13140b2d90 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.776+0000 7f131259c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13140b2980 0x7f13140cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.777+0000 7f1309d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2d90 0x7f13140cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.777+0000 7f1312d9d640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f13140cd850 0x7f13140cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:48814/0 (socket says 172.21.15.55:48814) 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.777+0000 7f1312d9d640 1 -- 172.21.15.55:0/2700391459 learned_addr learned my addr 172.21.15.55:0/2700391459 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.777+0000 7f131259c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13140b2980 0x7f13140cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:47132/0 (socket says 172.21.15.55:47132) 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.777+0000 7f1309d9b640 1 -- 172.21.15.55:0/2700391459 >> v2:172.21.15.138:3300/0 conn(0x7f13140b2980 msgr2=0x7f13140cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.777+0000 7f1309d9b640 1 --2- 172.21.15.55:0/2700391459 >> v2:172.21.15.138:3300/0 conn(0x7f13140b2980 0x7f13140cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.777+0000 7f1309d9b640 1 -- 172.21.15.55:0/2700391459 >> v1:172.21.15.138:6789/0 conn(0x7f13140cd850 legacy=0x7f13140cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.777+0000 7f1309d9b640 1 -- 172.21.15.55:0/2700391459 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13140ced60 con 0x7f13140b2d90 2024-05-20T22:08:05.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.777+0000 7f131259c640 1 --2- 172.21.15.55:0/2700391459 >> v2:172.21.15.138:3300/0 conn(0x7f13140b2980 0x7f13140cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:05.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.778+0000 7f1309d9b640 1 --2- 172.21.15.55:0/2700391459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2d90 0x7f13140cc2f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0097a0 tx=0x7f12fc02a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d499ca61a707cf3f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:05.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.778+0000 7f1311d9b640 1 -- 172.21.15.55:0/2700391459 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc037070 con 0x7f13140b2d90 2024-05-20T22:08:05.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.778+0000 7f1311d9b640 1 -- 172.21.15.55:0/2700391459 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12fc02ac90 con 0x7f13140b2d90 2024-05-20T22:08:05.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.778+0000 7f1311d9b640 1 -- 172.21.15.55:0/2700391459 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc032660 con 0x7f13140b2d90 2024-05-20T22:08:05.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.778+0000 7f1318d1a640 1 -- 172.21.15.55:0/2700391459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2d90 msgr2=0x7f13140cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.778+0000 7f1318d1a640 1 --2- 172.21.15.55:0/2700391459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2d90 0x7f13140cc2f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0097a0 tx=0x7f12fc02a000 comp rx=0 tx=0).stop 2024-05-20T22:08:05.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.779+0000 7f1318d1a640 1 -- 172.21.15.55:0/2700391459 shutdown_connections 2024-05-20T22:08:05.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.779+0000 7f1318d1a640 1 --2- 172.21.15.55:0/2700391459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2d90 0x7f13140cc2f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.779+0000 7f1318d1a640 1 --2- 172.21.15.55:0/2700391459 >> v2:172.21.15.138:3300/0 conn(0x7f13140b2980 0x7f13140cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.779+0000 7f1318d1a640 1 -- 172.21.15.55:0/2700391459 >> 172.21.15.55:0/2700391459 conn(0x7f13140736f0 msgr2=0x7f13140b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:05.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.779+0000 7f1318d1a640 1 -- 172.21.15.55:0/2700391459 shutdown_connections 2024-05-20T22:08:05.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.779+0000 7f1318d1a640 1 -- 172.21.15.55:0/2700391459 wait complete. 2024-05-20T22:08:05.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.781+0000 7f1318d1a640 1 Processor -- start 2024-05-20T22:08:05.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.781+0000 7f1318d1a640 1 -- start start 2024-05-20T22:08:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f1318d1a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2980 0x7f13140cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f1318d1a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13140b2d90 0x7f13141f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f1318d1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13140cd850 0x7f13141f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f1318d1a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f13140cd3e0 con 0x7f13140b2d90 2024-05-20T22:08:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f1318d1a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f131409adf0 con 0x7f13140cd850 2024-05-20T22:08:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f1318d1a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f13140d1c10 con 0x7f13140b2980 2024-05-20T22:08:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f1309d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13140b2d90 0x7f13141f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f1309d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13140b2d90 0x7f13141f8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52888/0 (socket says 172.21.15.55:52888) 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f1309d9b640 1 -- 172.21.15.55:0/1973967431 learned_addr learned my addr 172.21.15.55:0/1973967431 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f1312d9d640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13140cd850 0x7f13141f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.782+0000 7f131259c640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2980 0x7f13140cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f1309d9b640 1 -- 172.21.15.55:0/1973967431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2980 msgr2=0x7f13140cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f1309d9b640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2980 0x7f13140cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f1309d9b640 1 -- 172.21.15.55:0/1973967431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13140cd850 msgr2=0x7f13141f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f1309d9b640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13140cd850 0x7f13141f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f1309d9b640 1 -- 172.21.15.55:0/1973967431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131406bdf0 con 0x7f13140b2d90 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f1309d9b640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13140b2d90 0x7f13141f8ec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0098c0 tx=0x7f12fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f130b7fe640 1 -- 172.21.15.55:0/1973967431 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc037040 con 0x7f13140b2d90 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f130b7fe640 1 -- 172.21.15.55:0/1973967431 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12fc004030 con 0x7f13140b2d90 2024-05-20T22:08:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f1318d1a640 1 -- 172.21.15.55:0/1973967431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13140c8940 con 0x7f13140b2d90 2024-05-20T22:08:05.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f130b7fe640 1 -- 172.21.15.55:0/1973967431 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc004400 con 0x7f13140b2d90 2024-05-20T22:08:05.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.783+0000 7f1318d1a640 1 -- 172.21.15.55:0/1973967431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131406d600 con 0x7f13140b2d90 2024-05-20T22:08:05.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.784+0000 7f1318d1a640 1 -- 172.21.15.55:0/1973967431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1314069090 con 0x7f13140b2d90 2024-05-20T22:08:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.785+0000 7f130b7fe640 1 -- 172.21.15.55:0/1973967431 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12fc003610 con 0x7f13140b2d90 2024-05-20T22:08:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.785+0000 7f130b7fe640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12f406be30 0x7f12f406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.785+0000 7f130b7fe640 1 -- 172.21.15.55:0/1973967431 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f12fc0ab9a0 con 0x7f13140b2d90 2024-05-20T22:08:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.785+0000 7f131259c640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12f406be30 0x7f12f406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.786+0000 7f131259c640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12f406be30 0x7f12f406e340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1304009a30 tx=0x7f1304008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:05.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.790+0000 7f130b7fe640 1 -- 172.21.15.55:0/1973967431 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f12fc04b070 con 0x7f13140b2d90 2024-05-20T22:08:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.973+0000 7f1318d1a640 1 -- 172.21.15.55:0/1973967431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f13140b6860 con 0x7f13140b2d90 2024-05-20T22:08:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.973+0000 7f130b7fe640 1 -- 172.21.15.55:0/1973967431 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f12fc07a010 con 0x7f13140b2d90 2024-05-20T22:08:05.975 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:05.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.975+0000 7f1318d1a640 1 -- 172.21.15.55:0/1973967431 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12f406be30 msgr2=0x7f12f406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.975+0000 7f1318d1a640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12f406be30 0x7f12f406e340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1304009a30 tx=0x7f1304008040 comp rx=0 tx=0).stop 2024-05-20T22:08:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.975+0000 7f1318d1a640 1 -- 172.21.15.55:0/1973967431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13140b2d90 msgr2=0x7f13141f8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.975+0000 7f1318d1a640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13140b2d90 0x7f13141f8ec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0098c0 tx=0x7f12fc0046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.975+0000 7f1318d1a640 1 -- 172.21.15.55:0/1973967431 shutdown_connections 2024-05-20T22:08:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.975+0000 7f1318d1a640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12f406be30 0x7f12f406e340 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.975+0000 7f1318d1a640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13140cd850 0x7f13141f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.975+0000 7f1318d1a640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13140b2d90 0x7f13141f8ec0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.975+0000 7f1318d1a640 1 --2- 172.21.15.55:0/1973967431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13140b2980 0x7f13140cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.975+0000 7f1318d1a640 1 -- 172.21.15.55:0/1973967431 >> 172.21.15.55:0/1973967431 conn(0x7f13140736f0 msgr2=0x7f13140b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.976+0000 7f1318d1a640 1 -- 172.21.15.55:0/1973967431 shutdown_connections 2024-05-20T22:08:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:05.976+0000 7f1318d1a640 1 -- 172.21.15.55:0/1973967431 wait complete. 2024-05-20T22:08:05.988 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:06.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.105+0000 7ff9ead2f640 1 Processor -- start 2024-05-20T22:08:06.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.105+0000 7ff9ead2f640 1 -- start start 2024-05-20T22:08:06.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.106+0000 7ff9ead2f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9e40b4080 0x7ff9e415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.106+0000 7ff9ead2f640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e406b500 con 0x7ff9e40b35e0 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.106+0000 7ff9ead2f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e40b3e10 con 0x7ff9e40d3290 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.106+0000 7ff9ead2f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff9e4163960 con 0x7ff9e40b4080 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.106+0000 7ff9dbfff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff9e40b35e0 0x7ff9e415f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47366/0 (socket says 172.21.15.55:47366) 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.106+0000 7ff9dbfff640 1 -- 172.21.15.55:0/1196613838 learned_addr learned my addr 172.21.15.55:0/1196613838 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9e8acb640 1 --2- 172.21.15.55:0/1196613838 >> v2:172.21.15.138:3300/0 conn(0x7ff9e40b4080 0x7ff9e415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3315926969 0 0) 0x7ff9d4001110 con 0x7ff9e40b35e0 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9cc003520 con 0x7ff9e40b35e0 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1824108471 0 0) 0x7ff9cc003520 con 0x7ff9e40b35e0 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 >> v2:172.21.15.138:3300/0 conn(0x7ff9e40b4080 msgr2=0x7ff9e415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 --2- 172.21.15.55:0/1196613838 >> v2:172.21.15.138:3300/0 conn(0x7ff9e40b4080 0x7ff9e415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 >> v1:172.21.15.138:6789/0 conn(0x7ff9e40d3290 legacy=0x7ff9e40d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e40b3b30 con 0x7ff9e40b35e0 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7ff9d4001840 con 0x7ff9e40b35e0 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 >> v1:172.21.15.55:6789/0 conn(0x7ff9e40b35e0 legacy=0x7ff9e415f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9cc0039b0 0x7ff9cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9cc004800 0x7ff9cc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0051c0 0x7ff9cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e4163960 con 0x7ff9cc0051c0 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc00c160 con 0x7ff9cc004800 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff9cc00c360 con 0x7ff9cc0039b0 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9e92cc640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0051c0 0x7ff9cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9dbfff640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9cc004800 0x7ff9cc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.107+0000 7ff9e8acb640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9cc0039b0 0x7ff9cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.108+0000 7ff9e92cc640 1 -- 172.21.15.55:0/1196613838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9cc0039b0 msgr2=0x7ff9cc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.108+0000 7ff9e92cc640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9cc0039b0 0x7ff9cc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.108+0000 7ff9e92cc640 1 -- 172.21.15.55:0/1196613838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9cc004800 msgr2=0x7ff9cc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.108+0000 7ff9e92cc640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9cc004800 0x7ff9cc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.108+0000 7ff9e92cc640 1 -- 172.21.15.55:0/1196613838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc003520 con 0x7ff9cc0051c0 2024-05-20T22:08:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.108+0000 7ff9e92cc640 1 -- 172.21.15.55:0/1196613838 reap_dead start 2024-05-20T22:08:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.108+0000 7ff9e92cc640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0051c0 0x7ff9cc00b9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40b7510 tx=0x7ff9cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719afd8ea68ef377 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.108+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc001a70 con 0x7ff9cc0051c0 2024-05-20T22:08:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.108+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9dc02ad40 con 0x7ff9cc0051c0 2024-05-20T22:08:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.108+0000 7ff9db7fe640 1 -- 172.21.15.55:0/1196613838 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc003b90 con 0x7ff9cc0051c0 2024-05-20T22:08:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.109+0000 7ff9ead2f640 1 -- 172.21.15.55:0/1196613838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0051c0 msgr2=0x7ff9cc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.109+0000 7ff9ead2f640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0051c0 0x7ff9cc00b9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40b7510 tx=0x7ff9cc005830 comp rx=0 tx=0).stop 2024-05-20T22:08:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.109+0000 7ff9ead2f640 1 -- 172.21.15.55:0/1196613838 shutdown_connections 2024-05-20T22:08:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.109+0000 7ff9ead2f640 1 --2- 172.21.15.55:0/1196613838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0051c0 0x7ff9cc00b9c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.109+0000 7ff9ead2f640 1 -- 172.21.15.55:0/1196613838 >> 172.21.15.55:0/1196613838 conn(0x7ff9e40d51b0 msgr2=0x7ff9e40b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.109+0000 7ff9ead2f640 1 -- 172.21.15.55:0/1196613838 shutdown_connections 2024-05-20T22:08:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.109+0000 7ff9ead2f640 1 -- 172.21.15.55:0/1196613838 wait complete. 2024-05-20T22:08:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.111+0000 7ff9ead2f640 1 Processor -- start 2024-05-20T22:08:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.111+0000 7ff9ead2f640 1 -- start start 2024-05-20T22:08:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.112+0000 7ff9ead2f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9cc0051c0 0x7ff9e40b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.112+0000 7ff9ead2f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e415b470 0x7ff9e41f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.112+0000 7ff9ead2f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e415dc80 0x7ff9e415e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.112+0000 7ff9ead2f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e41635f0 con 0x7ff9e415dc80 2024-05-20T22:08:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.112+0000 7ff9ead2f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e41604f0 con 0x7ff9cc0051c0 2024-05-20T22:08:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.112+0000 7ff9ead2f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e415f2e0 con 0x7ff9e415b470 2024-05-20T22:08:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9e92cc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e415dc80 0x7ff9e415e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9e92cc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e415dc80 0x7ff9e415e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52912/0 (socket says 172.21.15.55:52912) 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9e92cc640 1 -- 172.21.15.55:0/2409758595 learned_addr learned my addr 172.21.15.55:0/2409758595 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9dbfff640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e415b470 0x7ff9e41f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9e8acb640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9cc0051c0 0x7ff9e40b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9e92cc640 1 -- 172.21.15.55:0/2409758595 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e415b470 msgr2=0x7ff9e41f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9e92cc640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e415b470 0x7ff9e41f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9e92cc640 1 -- 172.21.15.55:0/2409758595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9cc0051c0 msgr2=0x7ff9e40b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9e92cc640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9cc0051c0 0x7ff9e40b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9e92cc640 1 -- 172.21.15.55:0/2409758595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e415e5d0 con 0x7ff9e415dc80 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9e92cc640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e415dc80 0x7ff9e415e090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc02a7f0 tx=0x7ff9dc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9d97fa640 1 -- 172.21.15.55:0/2409758595 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc041070 con 0x7ff9e415dc80 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9d97fa640 1 -- 172.21.15.55:0/2409758595 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9dc02ab40 con 0x7ff9e415dc80 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9d97fa640 1 -- 172.21.15.55:0/2409758595 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc001a70 con 0x7ff9e415dc80 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9ead2f640 1 -- 172.21.15.55:0/2409758595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e41f5620 con 0x7ff9e415dc80 2024-05-20T22:08:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.113+0000 7ff9ead2f640 1 -- 172.21.15.55:0/2409758595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e414af20 con 0x7ff9e415dc80 2024-05-20T22:08:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.114+0000 7ff9ead2f640 1 -- 172.21.15.55:0/2409758595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e4069090 con 0x7ff9e415dc80 2024-05-20T22:08:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.115+0000 7ff9d97fa640 1 -- 172.21.15.55:0/2409758595 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9dc044070 con 0x7ff9e415dc80 2024-05-20T22:08:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.116+0000 7ff9d97fa640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff9d006be60 0x7ff9d006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.116+0000 7ff9d97fa640 1 -- 172.21.15.55:0/2409758595 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff9dc0aa2d0 con 0x7ff9e415dc80 2024-05-20T22:08:06.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.116+0000 7ff9e8acb640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff9d006be60 0x7ff9d006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.116+0000 7ff9e8acb640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff9d006be60 0x7ff9d006e370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40d0980 tx=0x7ff9e0009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:06.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.120+0000 7ff9d97fa640 1 -- 172.21.15.55:0/2409758595 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff9e415e5d0 con 0x7ff9e415dc80 2024-05-20T22:08:06.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.304+0000 7ff9ead2f640 1 -- 172.21.15.55:0/2409758595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff9e415eb20 con 0x7ff9e415dc80 2024-05-20T22:08:06.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.304+0000 7ff9d97fa640 1 -- 172.21.15.55:0/2409758595 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7ff9dc0aa0f0 con 0x7ff9e415dc80 2024-05-20T22:08:06.307 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:06.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 -- 172.21.15.55:0/2409758595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff9d006be60 msgr2=0x7ff9d006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff9d006be60 0x7ff9d006e370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40d0980 tx=0x7ff9e0009190 comp rx=0 tx=0).stop 2024-05-20T22:08:06.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 -- 172.21.15.55:0/2409758595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e415dc80 msgr2=0x7ff9e415e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e415dc80 0x7ff9e415e090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc02a7f0 tx=0x7ff9dc0014a0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 -- 172.21.15.55:0/2409758595 shutdown_connections 2024-05-20T22:08:06.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff9d006be60 0x7ff9d006e370 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e415dc80 0x7ff9e415e090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9e415b470 0x7ff9e41f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 --2- 172.21.15.55:0/2409758595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9cc0051c0 0x7ff9e40b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 -- 172.21.15.55:0/2409758595 >> 172.21.15.55:0/2409758595 conn(0x7ff9e40d51b0 msgr2=0x7ff9e40ba450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 -- 172.21.15.55:0/2409758595 shutdown_connections 2024-05-20T22:08:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.307+0000 7ff9ead2f640 1 -- 172.21.15.55:0/2409758595 wait complete. 2024-05-20T22:08:06.320 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:08:06.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.434+0000 7f581fb77640 1 Processor -- start 2024-05-20T22:08:06.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.434+0000 7f581fb77640 1 -- start start 2024-05-20T22:08:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581fb77640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5818167ff0 0x7f581816a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581fb77640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58180d04b0 con 0x7f581815f150 2024-05-20T22:08:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581fb77640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58180d3780 con 0x7f58180d32a0 2024-05-20T22:08:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581fb77640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f581806c510 con 0x7f5818167ff0 2024-05-20T22:08:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581d112640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f581815f150 0x7f58180cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47380/0 (socket says 172.21.15.55:47380) 2024-05-20T22:08:06.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581d112640 1 -- 172.21.15.55:0/1655244788 learned_addr learned my addr 172.21.15.55:0/1655244788 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:06.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581e114640 1 --2- 172.21.15.55:0/1655244788 >> v2:172.21.15.138:3300/0 conn(0x7f5818167ff0 0x7f581816a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581c911640 1 -- 172.21.15.55:0/1655244788 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 260364224 0 0) 0x7f58180d04b0 con 0x7f581815f150 2024-05-20T22:08:06.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581c911640 1 -- 172.21.15.55:0/1655244788 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5804003520 con 0x7f581815f150 2024-05-20T22:08:06.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581c911640 1 -- 172.21.15.55:0/1655244788 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3344783258 0 0) 0x7f58180d3780 con 0x7f58180d32a0 2024-05-20T22:08:06.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581c911640 1 -- 172.21.15.55:0/1655244788 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58180d04b0 con 0x7f58180d32a0 2024-05-20T22:08:06.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581e114640 1 -- 172.21.15.55:0/1655244788 >> v1:172.21.15.138:6789/0 conn(0x7f58180d32a0 legacy=0x7f58180cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581e114640 1 -- 172.21.15.55:0/1655244788 >> v1:172.21.15.55:6789/0 conn(0x7f581815f150 legacy=0x7f58180cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.436+0000 7f581e114640 1 -- 172.21.15.55:0/1655244788 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58180d3a60 con 0x7f5818167ff0 2024-05-20T22:08:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.437+0000 7f581e114640 1 --2- 172.21.15.55:0/1655244788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5818167ff0 0x7f581816a410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5810009d90 tx=0x7f581002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61b91da479af6b89 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.437+0000 7f581c911640 1 -- 172.21.15.55:0/1655244788 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810007740 con 0x7f5818167ff0 2024-05-20T22:08:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.437+0000 7f581c911640 1 -- 172.21.15.55:0/1655244788 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58100078e0 con 0x7f5818167ff0 2024-05-20T22:08:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.437+0000 7f581c911640 1 -- 172.21.15.55:0/1655244788 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810007cb0 con 0x7f5818167ff0 2024-05-20T22:08:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.438+0000 7f581fb77640 1 -- 172.21.15.55:0/1655244788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5818167ff0 msgr2=0x7f581816a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.438+0000 7f581fb77640 1 --2- 172.21.15.55:0/1655244788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5818167ff0 0x7f581816a410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5810009d90 tx=0x7f581002a6f0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.438+0000 7f581fb77640 1 -- 172.21.15.55:0/1655244788 shutdown_connections 2024-05-20T22:08:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.438+0000 7f581fb77640 1 --2- 172.21.15.55:0/1655244788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5818167ff0 0x7f581816a410 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.438+0000 7f581fb77640 1 -- 172.21.15.55:0/1655244788 >> 172.21.15.55:0/1655244788 conn(0x7f58180b6de0 msgr2=0x7f58180d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.438+0000 7f581fb77640 1 -- 172.21.15.55:0/1655244788 shutdown_connections 2024-05-20T22:08:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.438+0000 7f581fb77640 1 -- 172.21.15.55:0/1655244788 wait complete. 2024-05-20T22:08:06.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.440+0000 7f581fb77640 1 Processor -- start 2024-05-20T22:08:06.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.440+0000 7f581fb77640 1 -- start start 2024-05-20T22:08:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581fb77640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58180d32a0 0x7f5818155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581fb77640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f581815f150 0x7f5818161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581fb77640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818167ff0 0x7f58181617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581fb77640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f58180d4300 con 0x7f5818167ff0 2024-05-20T22:08:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581fb77640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58180d4020 con 0x7f58180d32a0 2024-05-20T22:08:06.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581fb77640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f581815f560 con 0x7f581815f150 2024-05-20T22:08:06.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581e114640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818167ff0 0x7f58181617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581e114640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818167ff0 0x7f58181617c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52916/0 (socket says 172.21.15.55:52916) 2024-05-20T22:08:06.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581e114640 1 -- 172.21.15.55:0/3945311497 learned_addr learned my addr 172.21.15.55:0/3945311497 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:06.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581d913640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58180d32a0 0x7f5818155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.441+0000 7f581d112640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f581815f150 0x7f5818161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f581e114640 1 -- 172.21.15.55:0/3945311497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f581815f150 msgr2=0x7f5818161280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f581e114640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f581815f150 0x7f5818161280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f581e114640 1 -- 172.21.15.55:0/3945311497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58180d32a0 msgr2=0x7f5818155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f581e114640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58180d32a0 0x7f5818155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f581e114640 1 -- 172.21.15.55:0/3945311497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58180cb550 con 0x7f5818167ff0 2024-05-20T22:08:06.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f581e114640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818167ff0 0x7f58181617c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f581002ace0 tx=0x7f5810004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:06.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f580e7fc640 1 -- 172.21.15.55:0/3945311497 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581003f070 con 0x7f5818167ff0 2024-05-20T22:08:06.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f580e7fc640 1 -- 172.21.15.55:0/3945311497 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5810031bc0 con 0x7f5818167ff0 2024-05-20T22:08:06.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f581fb77640 1 -- 172.21.15.55:0/3945311497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5818160220 con 0x7f5818167ff0 2024-05-20T22:08:06.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f580e7fc640 1 -- 172.21.15.55:0/3945311497 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810009450 con 0x7f5818167ff0 2024-05-20T22:08:06.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.442+0000 7f581fb77640 1 -- 172.21.15.55:0/3945311497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181607e0 con 0x7f5818167ff0 2024-05-20T22:08:06.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.443+0000 7f581fb77640 1 -- 172.21.15.55:0/3945311497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5818069090 con 0x7f5818167ff0 2024-05-20T22:08:06.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.444+0000 7f580e7fc640 1 -- 172.21.15.55:0/3945311497 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5810003990 con 0x7f5818167ff0 2024-05-20T22:08:06.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.444+0000 7f580e7fc640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57fc06be60 0x7f57fc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.444+0000 7f581d913640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57fc06be60 0x7f57fc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.444+0000 7f580e7fc640 1 -- 172.21.15.55:0/3945311497 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f58100ac740 con 0x7f5818167ff0 2024-05-20T22:08:06.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.445+0000 7f581d913640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57fc06be60 0x7f57fc06e370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5814007fd0 tx=0x7f5814009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.449+0000 7f580e7fc640 1 -- 172.21.15.55:0/3945311497 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5810049060 con 0x7f5818167ff0 2024-05-20T22:08:06.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.647+0000 7f581fb77640 1 -- 172.21.15.55:0/3945311497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f58180cb2e0 con 0x7f5818167ff0 2024-05-20T22:08:06.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.647+0000 7f580e7fc640 1 -- 172.21.15.55:0/3945311497 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f581007a760 con 0x7f5818167ff0 2024-05-20T22:08:06.649 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-20T22:08:06.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 -- 172.21.15.55:0/3945311497 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57fc06be60 msgr2=0x7f57fc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57fc06be60 0x7f57fc06e370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5814007fd0 tx=0x7f5814009040 comp rx=0 tx=0).stop 2024-05-20T22:08:06.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 -- 172.21.15.55:0/3945311497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818167ff0 msgr2=0x7f58181617c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818167ff0 0x7f58181617c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f581002ace0 tx=0x7f5810004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:06.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 -- 172.21.15.55:0/3945311497 shutdown_connections 2024-05-20T22:08:06.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57fc06be60 0x7f57fc06e370 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818167ff0 0x7f58181617c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f581815f150 0x7f5818161280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 --2- 172.21.15.55:0/3945311497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58180d32a0 0x7f5818155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 -- 172.21.15.55:0/3945311497 >> 172.21.15.55:0/3945311497 conn(0x7f58180b6de0 msgr2=0x7f58180b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:06.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 -- 172.21.15.55:0/3945311497 shutdown_connections 2024-05-20T22:08:06.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.649+0000 7f581fb77640 1 -- 172.21.15.55:0/3945311497 wait complete. 2024-05-20T22:08:06.663 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:08:06.663 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:06.679 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:06.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.834+0000 7f30b1196640 1 Processor -- start 2024-05-20T22:08:06.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.834+0000 7f30b1196640 1 -- start start 2024-05-20T22:08:06.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.835+0000 7f30b1196640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30ac06e520 0x7f30ac0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.835+0000 7f30b1196640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30ac0d1d70 0x7f30ac0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.835+0000 7f30b1196640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30ac0d2180 con 0x7f30ac06dfd0 2024-05-20T22:08:06.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.835+0000 7f30b1196640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f30ac06bd90 con 0x7f30ac0d1d70 2024-05-20T22:08:06.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.835+0000 7f30b1196640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f30ac0cd330 con 0x7f30ac06e520 2024-05-20T22:08:06.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.835+0000 7f30ab59e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30ac0d1d70 0x7f30ac0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.835+0000 7f30ab59e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30ac0d1d70 0x7f30ac0b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52930/0 (socket says 172.21.15.55:52930) 2024-05-20T22:08:06.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.835+0000 7f30ab59e640 1 -- 172.21.15.55:0/1436219934 learned_addr learned my addr 172.21.15.55:0/1436219934 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:06.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.835+0000 7f30aad9d640 1 --2- 172.21.15.55:0/1436219934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30ac06e520 0x7f30ac0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.836+0000 7f30ab59e640 1 -- 172.21.15.55:0/1436219934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30ac06e520 msgr2=0x7f30ac0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:06.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.836+0000 7f30ab59e640 1 --2- 172.21.15.55:0/1436219934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30ac06e520 0x7f30ac0d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.836+0000 7f30ab59e640 1 -- 172.21.15.55:0/1436219934 >> v1:172.21.15.138:6789/0 conn(0x7f30ac06dfd0 legacy=0x7f30ac0b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.836+0000 7f30ab59e640 1 -- 172.21.15.55:0/1436219934 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac0b2720 con 0x7f30ac0d1d70 2024-05-20T22:08:06.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.836+0000 7f30ab59e640 1 --2- 172.21.15.55:0/1436219934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 0x7f30ac0b1860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009870 tx=0x7f30a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57686ab9b2cd6806 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:06.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30a9d9b640 1 -- 172.21.15.55:0/1436219934 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4037070 con 0x7f30ac0d1d70 2024-05-20T22:08:06.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30a9d9b640 1 -- 172.21.15.55:0/1436219934 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a4032440 con 0x7f30ac0d1d70 2024-05-20T22:08:06.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30a9d9b640 1 -- 172.21.15.55:0/1436219934 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4032810 con 0x7f30ac0d1d70 2024-05-20T22:08:06.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30b1196640 1 -- 172.21.15.55:0/1436219934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 msgr2=0x7f30ac0b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30b1196640 1 --2- 172.21.15.55:0/1436219934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 0x7f30ac0b1860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009870 tx=0x7f30a402a270 comp rx=0 tx=0).stop 2024-05-20T22:08:06.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30b1196640 1 -- 172.21.15.55:0/1436219934 shutdown_connections 2024-05-20T22:08:06.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30b1196640 1 --2- 172.21.15.55:0/1436219934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 0x7f30ac0b1860 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30b1196640 1 --2- 172.21.15.55:0/1436219934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30ac06e520 0x7f30ac0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30b1196640 1 -- 172.21.15.55:0/1436219934 >> 172.21.15.55:0/1436219934 conn(0x7f30ac0b6580 msgr2=0x7f30ac0b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:06.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30b1196640 1 -- 172.21.15.55:0/1436219934 shutdown_connections 2024-05-20T22:08:06.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.837+0000 7f30b1196640 1 -- 172.21.15.55:0/1436219934 wait complete. 2024-05-20T22:08:06.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.839+0000 7f30b1196640 1 Processor -- start 2024-05-20T22:08:06.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.839+0000 7f30b1196640 1 -- start start 2024-05-20T22:08:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30b1196640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30ac06dfd0 0x7f30ac152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30b1196640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30ac06e520 0x7f30ac09b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30b1196640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 0x7f30ac0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30b1196640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac0cd800 con 0x7f30ac0d1d70 2024-05-20T22:08:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30b1196640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac0cd560 con 0x7f30ac06e520 2024-05-20T22:08:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30b1196640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f30ac15d4f0 con 0x7f30ac06dfd0 2024-05-20T22:08:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30ab59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 0x7f30ac0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30ab59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 0x7f30ac0ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52932/0 (socket says 172.21.15.55:52932) 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30ab59e640 1 -- 172.21.15.55:0/991683722 learned_addr learned my addr 172.21.15.55:0/991683722 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30aad9d640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30ac06dfd0 0x7f30ac152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30aa59c640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30ac06e520 0x7f30ac09b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30ab59e640 1 -- 172.21.15.55:0/991683722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30ac06dfd0 msgr2=0x7f30ac152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30ab59e640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30ac06dfd0 0x7f30ac152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30ab59e640 1 -- 172.21.15.55:0/991683722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30ac06e520 msgr2=0x7f30ac09b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30ab59e640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30ac06e520 0x7f30ac09b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30ab59e640 1 -- 172.21.15.55:0/991683722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac153440 con 0x7f30ac0d1d70 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30aa59c640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30ac06e520 0x7f30ac09b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.841+0000 7f30ab59e640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 0x7f30ac0ca360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f30a402a2b0 tx=0x7f30a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.842+0000 7f30a37fe640 1 -- 172.21.15.55:0/991683722 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4041020 con 0x7f30ac0d1d70 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.842+0000 7f30a37fe640 1 -- 172.21.15.55:0/991683722 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a40306c0 con 0x7f30ac0d1d70 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.842+0000 7f30a37fe640 1 -- 172.21.15.55:0/991683722 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4037040 con 0x7f30ac0d1d70 2024-05-20T22:08:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.842+0000 7f30b1196640 1 -- 172.21.15.55:0/991683722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac152ed0 con 0x7f30ac0d1d70 2024-05-20T22:08:06.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.842+0000 7f30b1196640 1 -- 172.21.15.55:0/991683722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac09ea40 con 0x7f30ac0d1d70 2024-05-20T22:08:06.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.843+0000 7f30b1196640 1 -- 172.21.15.55:0/991683722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ac069090 con 0x7f30ac0d1d70 2024-05-20T22:08:06.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.843+0000 7f30a37fe640 1 -- 172.21.15.55:0/991683722 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30a4044020 con 0x7f30ac0d1d70 2024-05-20T22:08:06.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.844+0000 7f30a37fe640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f308c06be30 0x7f308c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:06.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.844+0000 7f30a37fe640 1 -- 172.21.15.55:0/991683722 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f30a40ab900 con 0x7f30ac0d1d70 2024-05-20T22:08:06.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.844+0000 7f30aad9d640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f308c06be30 0x7f308c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:06.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.844+0000 7f30aad9d640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f308c06be30 0x7f308c06e340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f30ac09f9e0 tx=0x7f309c009830 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:06.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:06.848+0000 7f30a37fe640 1 -- 172.21.15.55:0/991683722 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f30a404b030 con 0x7f30ac0d1d70 2024-05-20T22:08:07.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.037+0000 7f30b1196640 1 -- 172.21.15.55:0/991683722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30ac09e4f0 con 0x7f30ac0d1d70 2024-05-20T22:08:07.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.038+0000 7f30a37fe640 1 -- 172.21.15.55:0/991683722 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f30ac153440 con 0x7f30ac0d1d70 2024-05-20T22:08:07.040 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:07.040 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:07.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.040+0000 7f30b1196640 1 -- 172.21.15.55:0/991683722 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f308c06be30 msgr2=0x7f308c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.040+0000 7f30b1196640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f308c06be30 0x7f308c06e340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f30ac09f9e0 tx=0x7f309c009830 comp rx=0 tx=0).stop 2024-05-20T22:08:07.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.040+0000 7f30b1196640 1 -- 172.21.15.55:0/991683722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 msgr2=0x7f30ac0ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.040+0000 7f30b1196640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 0x7f30ac0ca360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f30a402a2b0 tx=0x7f30a40046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.040+0000 7f30b1196640 1 -- 172.21.15.55:0/991683722 shutdown_connections 2024-05-20T22:08:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.040+0000 7f30b1196640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f308c06be30 0x7f308c06e340 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.040+0000 7f30b1196640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30ac0d1d70 0x7f30ac0ca360 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.040+0000 7f30b1196640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30ac06e520 0x7f30ac09b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.040+0000 7f30b1196640 1 --2- 172.21.15.55:0/991683722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f30ac06dfd0 0x7f30ac152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.040+0000 7f30b1196640 1 -- 172.21.15.55:0/991683722 >> 172.21.15.55:0/991683722 conn(0x7f30ac0b6580 msgr2=0x7f30ac0b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.041+0000 7f30b1196640 1 -- 172.21.15.55:0/991683722 shutdown_connections 2024-05-20T22:08:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.041+0000 7f30b1196640 1 -- 172.21.15.55:0/991683722 wait complete. 2024-05-20T22:08:07.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:07.054 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:07.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.171+0000 7fa2d6ca8640 1 Processor -- start 2024-05-20T22:08:07.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.171+0000 7fa2d6ca8640 1 -- start start 2024-05-20T22:08:07.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.172+0000 7fa2d6ca8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2d015e550 0x7fa2d015e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.172+0000 7fa2d6ca8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d0161140 0x7fa2d006c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.172+0000 7fa2d6ca8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2d0157870 con 0x7fa2d006bfe0 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.172+0000 7fa2d6ca8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d006c930 con 0x7fa2d015e550 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.172+0000 7fa2d6ca8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d0073310 con 0x7fa2d0161140 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.172+0000 7fa2d4a44640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2d015e550 0x7fa2d015e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.172+0000 7fa2d5245640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d0161140 0x7fa2d006c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.172+0000 7fa2cffff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa2d006bfe0 0x7fa2d0160a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:48840/0 (socket says 172.21.15.55:48840) 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.172+0000 7fa2d4a44640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2d015e550 0x7fa2d015e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:47202/0 (socket says 172.21.15.55:47202) 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.172+0000 7fa2cffff640 1 -- 172.21.15.55:0/3968203564 learned_addr learned my addr 172.21.15.55:0/3968203564 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.173+0000 7fa2cf7fe640 1 -- 172.21.15.55:0/3968203564 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 502954957 0 0) 0x7fa2d0157870 con 0x7fa2d006bfe0 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.173+0000 7fa2cf7fe640 1 -- 172.21.15.55:0/3968203564 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2c0003520 con 0x7fa2d006bfe0 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.173+0000 7fa2d5245640 1 -- 172.21.15.55:0/3968203564 >> v2:172.21.15.138:3300/0 conn(0x7fa2d015e550 msgr2=0x7fa2d015e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:07.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.173+0000 7fa2d5245640 1 --2- 172.21.15.55:0/3968203564 >> v2:172.21.15.138:3300/0 conn(0x7fa2d015e550 0x7fa2d015e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.173+0000 7fa2d5245640 1 -- 172.21.15.55:0/3968203564 >> v1:172.21.15.138:6789/0 conn(0x7fa2d006bfe0 legacy=0x7fa2d0160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.173+0000 7fa2d5245640 1 -- 172.21.15.55:0/3968203564 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d00735f0 con 0x7fa2d0161140 2024-05-20T22:08:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.173+0000 7fa2d4a44640 1 --2- 172.21.15.55:0/3968203564 >> v2:172.21.15.138:3300/0 conn(0x7fa2d015e550 0x7fa2d015e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.174+0000 7fa2d5245640 1 --2- 172.21.15.55:0/3968203564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d0161140 0x7fa2d006c3f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009d90 tx=0x7fa2c802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8936c72b31c2ebb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.174+0000 7fa2cf7fe640 1 -- 172.21.15.55:0/3968203564 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8007740 con 0x7fa2d0161140 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.174+0000 7fa2cf7fe640 1 -- 172.21.15.55:0/3968203564 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c80078e0 con 0x7fa2d0161140 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.174+0000 7fa2cf7fe640 1 -- 172.21.15.55:0/3968203564 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8007cb0 con 0x7fa2d0161140 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.174+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3968203564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d0161140 msgr2=0x7fa2d006c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.174+0000 7fa2d6ca8640 1 --2- 172.21.15.55:0/3968203564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d0161140 0x7fa2d006c3f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009d90 tx=0x7fa2c802a660 comp rx=0 tx=0).stop 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.174+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3968203564 shutdown_connections 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.174+0000 7fa2d6ca8640 1 --2- 172.21.15.55:0/3968203564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d0161140 0x7fa2d006c3f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.174+0000 7fa2d6ca8640 1 --2- 172.21.15.55:0/3968203564 >> v2:172.21.15.138:3300/0 conn(0x7fa2d015e550 0x7fa2d015e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.174+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3968203564 >> 172.21.15.55:0/3968203564 conn(0x7fa2d00b3980 msgr2=0x7fa2d00b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.175+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3968203564 shutdown_connections 2024-05-20T22:08:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.175+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3968203564 wait complete. 2024-05-20T22:08:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.176+0000 7fa2d6ca8640 1 Processor -- start 2024-05-20T22:08:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.176+0000 7fa2d6ca8640 1 -- start start 2024-05-20T22:08:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d6ca8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d006bfe0 0x7fa2d00d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d6ca8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d015e550 0x7fa2d00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d6ca8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d014a180 0x7fa2d009da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d6ca8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d006bac0 con 0x7fa2d006bfe0 2024-05-20T22:08:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d6ca8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d006b7e0 con 0x7fa2d015e550 2024-05-20T22:08:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d6ca8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d0072690 con 0x7fa2d014a180 2024-05-20T22:08:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d4a44640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d006bfe0 0x7fa2d00d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d4a44640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d006bfe0 0x7fa2d00d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52944/0 (socket says 172.21.15.55:52944) 2024-05-20T22:08:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d4a44640 1 -- 172.21.15.55:0/3224828887 learned_addr learned my addr 172.21.15.55:0/3224828887 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d5245640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d014a180 0x7fa2d009da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2cffff640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d015e550 0x7fa2d00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d4a44640 1 -- 172.21.15.55:0/3224828887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d014a180 msgr2=0x7fa2d009da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d4a44640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d014a180 0x7fa2d009da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d4a44640 1 -- 172.21.15.55:0/3224828887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d015e550 msgr2=0x7fa2d00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d4a44640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d015e550 0x7fa2d00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d4a44640 1 -- 172.21.15.55:0/3224828887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d00cef90 con 0x7fa2d006bfe0 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.178+0000 7fa2d5245640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d014a180 0x7fa2d009da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.179+0000 7fa2d4a44640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d006bfe0 0x7fa2d00d0030 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400e8e0 tx=0x7fa2c400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.179+0000 7fa2cd7fa640 1 -- 172.21.15.55:0/3224828887 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4017070 con 0x7fa2d006bfe0 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.179+0000 7fa2cd7fa640 1 -- 172.21.15.55:0/3224828887 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c400ca00 con 0x7fa2d006bfe0 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.179+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3224828887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d00cf270 con 0x7fa2d006bfe0 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.179+0000 7fa2cd7fa640 1 -- 172.21.15.55:0/3224828887 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400cdb0 con 0x7fa2d006bfe0 2024-05-20T22:08:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.179+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3224828887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d00cf830 con 0x7fa2d006bfe0 2024-05-20T22:08:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.180+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3224828887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d0069090 con 0x7fa2d006bfe0 2024-05-20T22:08:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.181+0000 7fa2cd7fa640 1 -- 172.21.15.55:0/3224828887 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2c401a070 con 0x7fa2d006bfe0 2024-05-20T22:08:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.181+0000 7fa2cd7fa640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2b406be30 0x7fa2b406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.181+0000 7fa2cffff640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2b406be30 0x7fa2b406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.181+0000 7fa2cd7fa640 1 -- 172.21.15.55:0/3224828887 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa2c408b420 con 0x7fa2d006bfe0 2024-05-20T22:08:07.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.182+0000 7fa2cffff640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2b406be30 0x7fa2b406e340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0154760 tx=0x7fa2bc000bc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.186+0000 7fa2cd7fa640 1 -- 172.21.15.55:0/3224828887 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa2c40593b0 con 0x7fa2d006bfe0 2024-05-20T22:08:07.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.369+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3224828887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa2d00b2570 con 0x7fa2d006bfe0 2024-05-20T22:08:07.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.370+0000 7fa2cd7fa640 1 -- 172.21.15.55:0/3224828887 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fa2c405d6b0 con 0x7fa2d006bfe0 2024-05-20T22:08:07.372 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:07.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.371+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3224828887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2b406be30 msgr2=0x7fa2b406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.371+0000 7fa2d6ca8640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2b406be30 0x7fa2b406e340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0154760 tx=0x7fa2bc000bc0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.371+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3224828887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d006bfe0 msgr2=0x7fa2d00d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.372+0000 7fa2d6ca8640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d006bfe0 0x7fa2d00d0030 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400e8e0 tx=0x7fa2c400eda0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.372+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3224828887 shutdown_connections 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.372+0000 7fa2d6ca8640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2b406be30 0x7fa2b406e340 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.372+0000 7fa2d6ca8640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2d014a180 0x7fa2d009da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.372+0000 7fa2d6ca8640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d015e550 0x7fa2d00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.372+0000 7fa2d6ca8640 1 --2- 172.21.15.55:0/3224828887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2d006bfe0 0x7fa2d00d0030 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.372+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3224828887 >> 172.21.15.55:0/3224828887 conn(0x7fa2d00b3980 msgr2=0x7fa2d00b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.372+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3224828887 shutdown_connections 2024-05-20T22:08:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.372+0000 7fa2d6ca8640 1 -- 172.21.15.55:0/3224828887 wait complete. 2024-05-20T22:08:07.385 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.500+0000 7f124874c640 1 Processor -- start 2024-05-20T22:08:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.500+0000 7f124874c640 1 -- start start 2024-05-20T22:08:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.501+0000 7f124874c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f12400cf680 0x7f124015f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.501+0000 7f124874c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f124006cb70 0x7f12400b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.501+0000 7f124874c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f124006b6a0 con 0x7f12400b7fb0 2024-05-20T22:08:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.501+0000 7f124874c640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f124006b980 con 0x7f124006cb70 2024-05-20T22:08:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.501+0000 7f124874c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f124006bc60 con 0x7f12400cf680 2024-05-20T22:08:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.502+0000 7f1246ce9640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f124006cb70 0x7f12400b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.502+0000 7f1246ce9640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f124006cb70 0x7f12400b83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52950/0 (socket says 172.21.15.55:52950) 2024-05-20T22:08:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.502+0000 7f1246ce9640 1 -- 172.21.15.55:0/1772762662 learned_addr learned my addr 172.21.15.55:0/1772762662 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.502+0000 7f12464e8640 1 --2- 172.21.15.55:0/1772762662 >> v2:172.21.15.138:3300/0 conn(0x7f12400cf680 0x7f124015f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.502+0000 7f1246ce9640 1 -- 172.21.15.55:0/1772762662 >> v2:172.21.15.138:3300/0 conn(0x7f12400cf680 msgr2=0x7f124015f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:07.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.502+0000 7f1246ce9640 1 --2- 172.21.15.55:0/1772762662 >> v2:172.21.15.138:3300/0 conn(0x7f12400cf680 0x7f124015f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.502+0000 7f1246ce9640 1 -- 172.21.15.55:0/1772762662 >> v1:172.21.15.138:6789/0 conn(0x7f12400b7fb0 legacy=0x7f124006c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:07.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.502+0000 7f1246ce9640 1 -- 172.21.15.55:0/1772762662 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124006bf40 con 0x7f124006cb70 2024-05-20T22:08:07.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.502+0000 7f12464e8640 1 --2- 172.21.15.55:0/1772762662 >> v2:172.21.15.138:3300/0 conn(0x7f12400cf680 0x7f124015f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:07.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.503+0000 7f1246ce9640 1 --2- 172.21.15.55:0/1772762662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124006cb70 0x7f12400b83c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f12400b6f50 tx=0x7f123802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9080a7e11b1b09ce server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:07.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.503+0000 7f12454e6640 1 -- 172.21.15.55:0/1772762662 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238036070 con 0x7f124006cb70 2024-05-20T22:08:07.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.503+0000 7f12454e6640 1 -- 172.21.15.55:0/1772762662 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1238031600 con 0x7f124006cb70 2024-05-20T22:08:07.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.503+0000 7f12454e6640 1 -- 172.21.15.55:0/1772762662 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12380319d0 con 0x7f124006cb70 2024-05-20T22:08:07.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.504+0000 7f124874c640 1 -- 172.21.15.55:0/1772762662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124006cb70 msgr2=0x7f12400b83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.504+0000 7f124874c640 1 --2- 172.21.15.55:0/1772762662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124006cb70 0x7f12400b83c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f12400b6f50 tx=0x7f123802a660 comp rx=0 tx=0).stop 2024-05-20T22:08:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.504+0000 7f124874c640 1 -- 172.21.15.55:0/1772762662 shutdown_connections 2024-05-20T22:08:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.504+0000 7f124874c640 1 --2- 172.21.15.55:0/1772762662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124006cb70 0x7f12400b83c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.504+0000 7f124874c640 1 --2- 172.21.15.55:0/1772762662 >> v2:172.21.15.138:3300/0 conn(0x7f12400cf680 0x7f124015f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.504+0000 7f124874c640 1 -- 172.21.15.55:0/1772762662 >> 172.21.15.55:0/1772762662 conn(0x7f12400d5450 msgr2=0x7f12400b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.504+0000 7f124874c640 1 -- 172.21.15.55:0/1772762662 shutdown_connections 2024-05-20T22:08:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.504+0000 7f124874c640 1 -- 172.21.15.55:0/1772762662 wait complete. 2024-05-20T22:08:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.506+0000 7f124874c640 1 Processor -- start 2024-05-20T22:08:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.506+0000 7f124874c640 1 -- start start 2024-05-20T22:08:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f124874c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124006cb70 0x7f1240155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f124874c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12400b7fb0 0x7f1240153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f124874c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12400cf680 0x7f12401f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f124874c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f12400d3740 con 0x7f12400cf680 2024-05-20T22:08:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f124874c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12400d35c0 con 0x7f124006cb70 2024-05-20T22:08:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f124874c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f12400d38c0 con 0x7f12400b7fb0 2024-05-20T22:08:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f1246ce9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12400cf680 0x7f12401f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f1246ce9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12400cf680 0x7f12401f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52964/0 (socket says 172.21.15.55:52964) 2024-05-20T22:08:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f1246ce9640 1 -- 172.21.15.55:0/1803417193 learned_addr learned my addr 172.21.15.55:0/1803417193 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f12464e8640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124006cb70 0x7f1240155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f1245ce7640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12400b7fb0 0x7f1240153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f1246ce9640 1 -- 172.21.15.55:0/1803417193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12400b7fb0 msgr2=0x7f1240153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f1246ce9640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12400b7fb0 0x7f1240153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f1246ce9640 1 -- 172.21.15.55:0/1803417193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124006cb70 msgr2=0x7f1240155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f1246ce9640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124006cb70 0x7f1240155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.507+0000 7f1246ce9640 1 -- 172.21.15.55:0/1803417193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12401547e0 con 0x7f12400cf680 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.508+0000 7f1246ce9640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12400cf680 0x7f12401f51c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f123802ac50 tx=0x7f1238004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.508+0000 7f1236ffd640 1 -- 172.21.15.55:0/1803417193 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238036050 con 0x7f12400cf680 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.508+0000 7f1236ffd640 1 -- 172.21.15.55:0/1803417193 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1238042070 con 0x7f12400cf680 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.508+0000 7f124874c640 1 -- 172.21.15.55:0/1803417193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1240154ac0 con 0x7f12400cf680 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.508+0000 7f1236ffd640 1 -- 172.21.15.55:0/1803417193 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238009510 con 0x7f12400cf680 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.508+0000 7f124874c640 1 -- 172.21.15.55:0/1803417193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1240155080 con 0x7f12400cf680 2024-05-20T22:08:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.509+0000 7f124874c640 1 -- 172.21.15.55:0/1803417193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1240154550 con 0x7f12400cf680 2024-05-20T22:08:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.509+0000 7f1236ffd640 1 -- 172.21.15.55:0/1803417193 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1238007bd0 con 0x7f12400cf680 2024-05-20T22:08:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.510+0000 7f1236ffd640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f122406be60 0x7f122406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.510+0000 7f1236ffd640 1 -- 172.21.15.55:0/1803417193 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f12380ac150 con 0x7f12400cf680 2024-05-20T22:08:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.510+0000 7f12464e8640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f122406be60 0x7f122406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.510+0000 7f12464e8640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f122406be60 0x7f122406e370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f123c00a950 tx=0x7f123c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.515+0000 7f1236ffd640 1 -- 172.21.15.55:0/1803417193 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f123803a360 con 0x7f12400cf680 2024-05-20T22:08:07.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.697+0000 7f124874c640 1 -- 172.21.15.55:0/1803417193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f12400d2e90 con 0x7f12400cf680 2024-05-20T22:08:07.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.697+0000 7f1236ffd640 1 -- 172.21.15.55:0/1803417193 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f123807a7c0 con 0x7f12400cf680 2024-05-20T22:08:07.703 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:07.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 -- 172.21.15.55:0/1803417193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f122406be60 msgr2=0x7f122406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f122406be60 0x7f122406e370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f123c00a950 tx=0x7f123c008040 comp rx=0 tx=0).stop 2024-05-20T22:08:07.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 -- 172.21.15.55:0/1803417193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12400cf680 msgr2=0x7f12401f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12400cf680 0x7f12401f51c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f123802ac50 tx=0x7f1238004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:07.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 -- 172.21.15.55:0/1803417193 shutdown_connections 2024-05-20T22:08:07.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f122406be60 0x7f122406e370 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12400cf680 0x7f12401f51c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12400b7fb0 0x7f1240153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 --2- 172.21.15.55:0/1803417193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124006cb70 0x7f1240155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 -- 172.21.15.55:0/1803417193 >> 172.21.15.55:0/1803417193 conn(0x7f12400d5450 msgr2=0x7f12400b5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:07.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 -- 172.21.15.55:0/1803417193 shutdown_connections 2024-05-20T22:08:07.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.699+0000 7f124874c640 1 -- 172.21.15.55:0/1803417193 wait complete. 2024-05-20T22:08:07.712 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:07.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.828+0000 7fed8ffe0640 1 Processor -- start 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.828+0000 7fed8ffe0640 1 -- start start 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8ffe0640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fed880cfad0 0x7fed880b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8ffe0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed8806d0b0 0x7fed880d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8ffe0640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed880b37b0 con 0x7fed880b2470 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8ffe0640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fed880b2880 con 0x7fed880cfad0 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8ffe0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed8806d8b0 con 0x7fed8806d0b0 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8d57b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fed880cfad0 0x7fed880b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8dd7c640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fed880b2470 0x7fed880cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47396/0 (socket says 172.21.15.55:47396) 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8dd7c640 1 -- 172.21.15.55:0/2453770271 learned_addr learned my addr 172.21.15.55:0/2453770271 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8e57d640 1 --2- 172.21.15.55:0/2453770271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed8806d0b0 0x7fed880d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8d57b640 1 -- 172.21.15.55:0/2453770271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed8806d0b0 msgr2=0x7fed880d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8d57b640 1 --2- 172.21.15.55:0/2453770271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed8806d0b0 0x7fed880d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8d57b640 1 -- 172.21.15.55:0/2453770271 >> v1:172.21.15.55:6789/0 conn(0x7fed880b2470 legacy=0x7fed880cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8d57b640 1 -- 172.21.15.55:0/2453770271 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8806db90 con 0x7fed880cfad0 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.830+0000 7fed8cd7a640 1 -- 172.21.15.55:0/2453770271 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 355367719 0 0) 0x7fed78001490 con 0x7fed880b2470 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.831+0000 7fed8d57b640 1 --2- 172.21.15.55:0/2453770271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880cfad0 0x7fed880b1eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097a0 tx=0x7fed7c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c166f7a79881a3ca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.831+0000 7fed8cd7a640 1 -- 172.21.15.55:0/2453770271 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c037070 con 0x7fed880cfad0 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.831+0000 7fed8cd7a640 1 -- 172.21.15.55:0/2453770271 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c032440 con 0x7fed880cfad0 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.831+0000 7fed8cd7a640 1 -- 172.21.15.55:0/2453770271 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c032810 con 0x7fed880cfad0 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.832+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2453770271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880cfad0 msgr2=0x7fed880b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.832+0000 7fed8ffe0640 1 --2- 172.21.15.55:0/2453770271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880cfad0 0x7fed880b1eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097a0 tx=0x7fed7c02a180 comp rx=0 tx=0).stop 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.832+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2453770271 shutdown_connections 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.832+0000 7fed8ffe0640 1 --2- 172.21.15.55:0/2453770271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed8806d0b0 0x7fed880d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.832+0000 7fed8ffe0640 1 --2- 172.21.15.55:0/2453770271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880cfad0 0x7fed880b1eb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.832+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2453770271 >> 172.21.15.55:0/2453770271 conn(0x7fed8809be10 msgr2=0x7fed880d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.832+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2453770271 shutdown_connections 2024-05-20T22:08:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.832+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2453770271 wait complete. 2024-05-20T22:08:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.834+0000 7fed8ffe0640 1 Processor -- start 2024-05-20T22:08:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.834+0000 7fed8ffe0640 1 -- start start 2024-05-20T22:08:07.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8ffe0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed8806d0b0 0x7fed8809f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8ffe0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880b2470 0x7fed8809fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8ffe0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed880a0430 0x7fed8809c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8ffe0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fed8806d4c0 con 0x7fed880b2470 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8ffe0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed880b3a30 con 0x7fed880a0430 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8ffe0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed880b3f50 con 0x7fed8806d0b0 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8d57b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880b2470 0x7fed8809fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8d57b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880b2470 0x7fed8809fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52974/0 (socket says 172.21.15.55:52974) 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8d57b640 1 -- 172.21.15.55:0/2028798735 learned_addr learned my addr 172.21.15.55:0/2028798735 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8dd7c640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed8806d0b0 0x7fed8809f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.835+0000 7fed8e57d640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed880a0430 0x7fed8809c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed8d57b640 1 -- 172.21.15.55:0/2028798735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed8806d0b0 msgr2=0x7fed8809f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed8d57b640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed8806d0b0 0x7fed8809f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed8d57b640 1 -- 172.21.15.55:0/2028798735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed880a0430 msgr2=0x7fed8809c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed8d57b640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed880a0430 0x7fed8809c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed8d57b640 1 -- 172.21.15.55:0/2028798735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed880a0840 con 0x7fed880b2470 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed8d57b640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880b2470 0x7fed8809fef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0098c0 tx=0x7fed7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed867fc640 1 -- 172.21.15.55:0/2028798735 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c037040 con 0x7fed880b2470 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed867fc640 1 -- 172.21.15.55:0/2028798735 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c042070 con 0x7fed880b2470 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2028798735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8809ec30 con 0x7fed880b2470 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed867fc640 1 -- 172.21.15.55:0/2028798735 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c030bb0 con 0x7fed880b2470 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.836+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2028798735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8809f1b0 con 0x7fed880b2470 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.837+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2028798735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed88069090 con 0x7fed880b2470 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.838+0000 7fed867fc640 1 -- 172.21.15.55:0/2028798735 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed7c047460 con 0x7fed880b2470 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.838+0000 7fed867fc640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed7406be60 0x7fed7406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:07.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.839+0000 7fed867fc640 1 -- 172.21.15.55:0/2028798735 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fed7c0abd20 con 0x7fed880b2470 2024-05-20T22:08:07.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.839+0000 7fed8dd7c640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed7406be60 0x7fed7406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:07.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.839+0000 7fed8dd7c640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed7406be60 0x7fed7406e370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed780012e0 tx=0x7fed78009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:07.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:07.844+0000 7fed867fc640 1 -- 172.21.15.55:0/2028798735 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fed7c032e20 con 0x7fed880b2470 2024-05-20T22:08:08.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.028+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2028798735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fed880d35a0 con 0x7fed880b2470 2024-05-20T22:08:08.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.028+0000 7fed867fc640 1 -- 172.21.15.55:0/2028798735 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fed7c079c10 con 0x7fed880b2470 2024-05-20T22:08:08.031 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:08.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2028798735 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed7406be60 msgr2=0x7fed7406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed7406be60 0x7fed7406e370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed780012e0 tx=0x7fed78009040 comp rx=0 tx=0).stop 2024-05-20T22:08:08.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2028798735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880b2470 msgr2=0x7fed8809fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880b2470 0x7fed8809fef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0098c0 tx=0x7fed7c0046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2028798735 shutdown_connections 2024-05-20T22:08:08.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed7406be60 0x7fed7406e370 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed880a0430 0x7fed8809c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880b2470 0x7fed8809fef0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 --2- 172.21.15.55:0/2028798735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed8806d0b0 0x7fed8809f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2028798735 >> 172.21.15.55:0/2028798735 conn(0x7fed8809be10 msgr2=0x7fed8808bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:08.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2028798735 shutdown_connections 2024-05-20T22:08:08.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.030+0000 7fed8ffe0640 1 -- 172.21.15.55:0/2028798735 wait complete. 2024-05-20T22:08:08.043 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:08.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.159+0000 7f1926c79640 1 Processor -- start 2024-05-20T22:08:08.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.159+0000 7f1926c79640 1 -- start start 2024-05-20T22:08:08.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.160+0000 7f1926c79640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19200cf270 0x7f19200cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.160+0000 7f1926c79640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192008cd70 con 0x7f192008c960 2024-05-20T22:08:08.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.160+0000 7f1926c79640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19200cfbc0 con 0x7f19200cd520 2024-05-20T22:08:08.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.160+0000 7f1926c79640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f19200d1020 con 0x7f19200cf270 2024-05-20T22:08:08.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.160+0000 7f191ffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f192008c960 0x7f19200b19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47402/0 (socket says 172.21.15.55:47402) 2024-05-20T22:08:08.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.160+0000 7f191ffff640 1 -- 172.21.15.55:0/1631088302 learned_addr learned my addr 172.21.15.55:0/1631088302 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:08.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f1925216640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19200cf270 0x7f19200cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 231537730 0 0) 0x7f190c001110 con 0x7f192008c960 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1910003520 con 0x7f192008c960 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1243161733 0 0) 0x7f1914001110 con 0x7f19200cd520 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f190c001110 con 0x7f19200cd520 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2807448667 0 0) 0x7f1910003520 con 0x7f192008c960 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19200cf270 msgr2=0x7f19200cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19200cf270 0x7f19200cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 >> v1:172.21.15.138:6789/0 conn(0x7f19200cd520 legacy=0x7f19200d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19200d1300 con 0x7f192008c960 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f190c001840 con 0x7f192008c960 2024-05-20T22:08:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 >> v1:172.21.15.55:6789/0 conn(0x7f192008c960 legacy=0x7f19200b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1910003b70 0x7f1910003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1910004800 0x7f1910004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19100051c0 0x7f191000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f19200d1020 con 0x7f19100051c0 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f191000c140 con 0x7f1910004800 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f191000c340 con 0x7f1910003b70 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f1925216640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19200cf270 0x7f19200cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f1925216640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19100051c0 0x7f191000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f191ffff640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1910004800 0x7f1910004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.161+0000 7f1924a15640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1910003b70 0x7f1910003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.162+0000 7f1925216640 1 -- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1910003b70 msgr2=0x7f1910003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.162+0000 7f1925216640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1910003b70 0x7f1910003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.162+0000 7f1925216640 1 -- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1910004800 msgr2=0x7f1910004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.162+0000 7f1925216640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1910004800 0x7f1910004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.162+0000 7f1925216640 1 -- 172.21.15.55:0/1631088302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1910003520 con 0x7f19100051c0 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.162+0000 7f1924a15640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1910003b70 0x7f1910003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.162+0000 7f1925216640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19100051c0 0x7f191000b9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f19200b8390 tx=0x7f1910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dcd988ab296aa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.162+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918036070 con 0x7f19100051c0 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.162+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1918003c50 con 0x7f19100051c0 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.162+0000 7f191f7fe640 1 -- 172.21.15.55:0/1631088302 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918009d40 con 0x7f19100051c0 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.163+0000 7f1926c79640 1 -- 172.21.15.55:0/1631088302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19100051c0 msgr2=0x7f191000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.163+0000 7f1926c79640 1 --2- 172.21.15.55:0/1631088302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19100051c0 0x7f191000b9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f19200b8390 tx=0x7f1910005830 comp rx=0 tx=0).stop 2024-05-20T22:08:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.163+0000 7f1925216640 1 -- 172.21.15.55:0/1631088302 reap_dead start 2024-05-20T22:08:08.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.163+0000 7f1926c79640 1 -- 172.21.15.55:0/1631088302 shutdown_connections 2024-05-20T22:08:08.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.163+0000 7f1926c79640 1 -- 172.21.15.55:0/1631088302 >> 172.21.15.55:0/1631088302 conn(0x7f19200d2860 msgr2=0x7f19200d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:08.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.163+0000 7f1926c79640 1 -- 172.21.15.55:0/1631088302 shutdown_connections 2024-05-20T22:08:08.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.163+0000 7f1926c79640 1 -- 172.21.15.55:0/1631088302 wait complete. 2024-05-20T22:08:08.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.165+0000 7f1926c79640 1 Processor -- start 2024-05-20T22:08:08.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.165+0000 7f1926c79640 1 -- start start 2024-05-20T22:08:08.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f1926c79640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19100051c0 0x7f1920152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f1926c79640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1920158a70 0x7f192015e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f1926c79640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f192015b130 0x7f192015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f1926c79640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f19200b2cb0 con 0x7f1920158a70 2024-05-20T22:08:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f1926c79640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19200d18c0 con 0x7f19100051c0 2024-05-20T22:08:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f1926c79640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f19200cf680 con 0x7f192015b130 2024-05-20T22:08:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1920158a70 0x7f192015e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1920158a70 0x7f192015e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52986/0 (socket says 172.21.15.55:52986) 2024-05-20T22:08:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191ffff640 1 -- 172.21.15.55:0/297191275 learned_addr learned my addr 172.21.15.55:0/297191275 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f1925216640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f192015b130 0x7f192015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f1924a15640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19100051c0 0x7f1920152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191ffff640 1 -- 172.21.15.55:0/297191275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f192015b130 msgr2=0x7f192015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191ffff640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f192015b130 0x7f192015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191ffff640 1 -- 172.21.15.55:0/297191275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19100051c0 msgr2=0x7f1920152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191ffff640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19100051c0 0x7f1920152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191ffff640 1 -- 172.21.15.55:0/297191275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192015ba80 con 0x7f1920158a70 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191ffff640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1920158a70 0x7f192015e930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f190c007340 tx=0x7f190c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191d7fa640 1 -- 172.21.15.55:0/297191275 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c00f9a0 con 0x7f1920158a70 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191d7fa640 1 -- 172.21.15.55:0/297191275 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f190c018070 con 0x7f1920158a70 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f191d7fa640 1 -- 172.21.15.55:0/297191275 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c00fd80 con 0x7f1920158a70 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f1926c79640 1 -- 172.21.15.55:0/297191275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1920160290 con 0x7f1920158a70 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.167+0000 7f1926c79640 1 -- 172.21.15.55:0/297191275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f192015f4d0 con 0x7f1920158a70 2024-05-20T22:08:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.170+0000 7f1926c79640 1 -- 172.21.15.55:0/297191275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1920069090 con 0x7f1920158a70 2024-05-20T22:08:08.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.170+0000 7f191d7fa640 1 -- 172.21.15.55:0/297191275 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f190c01d460 con 0x7f1920158a70 2024-05-20T22:08:08.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.170+0000 7f191d7fa640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f190406be60 0x7f190406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.170+0000 7f191d7fa640 1 -- 172.21.15.55:0/297191275 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f190c08b340 con 0x7f1920158a70 2024-05-20T22:08:08.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.170+0000 7f1924a15640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f190406be60 0x7f190406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.170+0000 7f1924a15640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f190406be60 0x7f190406e370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1914007fd0 tx=0x7f1914009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:08.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.174+0000 7f191d7fa640 1 -- 172.21.15.55:0/297191275 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f192015ba80 con 0x7f1920158a70 2024-05-20T22:08:08.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.360+0000 7f1926c79640 1 -- 172.21.15.55:0/297191275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f19200b27c0 con 0x7f1920158a70 2024-05-20T22:08:08.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.360+0000 7f191d7fa640 1 -- 172.21.15.55:0/297191275 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f192008cd70 con 0x7f1920158a70 2024-05-20T22:08:08.361 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:08.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 -- 172.21.15.55:0/297191275 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f190406be60 msgr2=0x7f190406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f190406be60 0x7f190406e370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1914007fd0 tx=0x7f1914009040 comp rx=0 tx=0).stop 2024-05-20T22:08:08.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 -- 172.21.15.55:0/297191275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1920158a70 msgr2=0x7f192015e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1920158a70 0x7f192015e930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f190c007340 tx=0x7f190c007800 comp rx=0 tx=0).stop 2024-05-20T22:08:08.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 -- 172.21.15.55:0/297191275 shutdown_connections 2024-05-20T22:08:08.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f190406be60 0x7f190406e370 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f192015b130 0x7f192015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1920158a70 0x7f192015e930 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 --2- 172.21.15.55:0/297191275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19100051c0 0x7f1920152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 -- 172.21.15.55:0/297191275 >> 172.21.15.55:0/297191275 conn(0x7f19200d2860 msgr2=0x7f19200b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:08.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 -- 172.21.15.55:0/297191275 shutdown_connections 2024-05-20T22:08:08.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.362+0000 7f1926c79640 1 -- 172.21.15.55:0/297191275 wait complete. 2024-05-20T22:08:08.375 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:08:08.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.490+0000 7f3680fb7640 1 Processor -- start 2024-05-20T22:08:08.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.490+0000 7f3680fb7640 1 -- start start 2024-05-20T22:08:08.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.491+0000 7f3680fb7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f367c0b35e0 0x7f367c15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.491+0000 7f3680fb7640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f367c06b500 con 0x7f367c0b4080 2024-05-20T22:08:08.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.491+0000 7f3680fb7640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f367c0b3e10 con 0x7f367c0d3290 2024-05-20T22:08:08.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.491+0000 7f3680fb7640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f367c163960 con 0x7f367c0b35e0 2024-05-20T22:08:08.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.491+0000 7f3679d9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f367c0b35e0 0x7f367c15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.491+0000 7f367a59c640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f367c0b4080 0x7f367c15f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47408/0 (socket says 172.21.15.55:47408) 2024-05-20T22:08:08.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.491+0000 7f367a59c640 1 -- 172.21.15.55:0/4028512764 learned_addr learned my addr 172.21.15.55:0/4028512764 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:08.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.492+0000 7f3679d9b640 1 -- 172.21.15.55:0/4028512764 >> v1:172.21.15.138:6789/0 conn(0x7f367c0d3290 legacy=0x7f367c0d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:08.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.492+0000 7f3679d9b640 1 -- 172.21.15.55:0/4028512764 >> v1:172.21.15.55:6789/0 conn(0x7f367c0b4080 legacy=0x7f367c15f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.492+0000 7f367959a640 1 -- 172.21.15.55:0/4028512764 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 495911485 0 0) 0x7f3674001490 con 0x7f367c0b4080 2024-05-20T22:08:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.492+0000 7f3679d9b640 1 -- 172.21.15.55:0/4028512764 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c0b3b30 con 0x7f367c0b35e0 2024-05-20T22:08:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.492+0000 7f3679d9b640 1 --2- 172.21.15.55:0/4028512764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0b35e0 0x7f367c15f9f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36640097a0 tx=0x7f366402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f49031787b1a5b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.492+0000 7f367959a640 1 -- 172.21.15.55:0/4028512764 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664037070 con 0x7f367c0b35e0 2024-05-20T22:08:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.493+0000 7f367959a640 1 -- 172.21.15.55:0/4028512764 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3664032440 con 0x7f367c0b35e0 2024-05-20T22:08:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.493+0000 7f367959a640 1 -- 172.21.15.55:0/4028512764 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664032810 con 0x7f367c0b35e0 2024-05-20T22:08:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.493+0000 7f3680fb7640 1 -- 172.21.15.55:0/4028512764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0b35e0 msgr2=0x7f367c15f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.493+0000 7f3680fb7640 1 --2- 172.21.15.55:0/4028512764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0b35e0 0x7f367c15f9f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36640097a0 tx=0x7f366402a1e0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.493+0000 7f3680fb7640 1 -- 172.21.15.55:0/4028512764 shutdown_connections 2024-05-20T22:08:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.493+0000 7f3680fb7640 1 --2- 172.21.15.55:0/4028512764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0b35e0 0x7f367c15f9f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.493+0000 7f3680fb7640 1 -- 172.21.15.55:0/4028512764 >> 172.21.15.55:0/4028512764 conn(0x7f367c0d51b0 msgr2=0x7f367c0b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:08.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.493+0000 7f3680fb7640 1 -- 172.21.15.55:0/4028512764 shutdown_connections 2024-05-20T22:08:08.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.493+0000 7f3680fb7640 1 -- 172.21.15.55:0/4028512764 wait complete. 2024-05-20T22:08:08.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.495+0000 7f3680fb7640 1 Processor -- start 2024-05-20T22:08:08.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.495+0000 7f3680fb7640 1 -- start start 2024-05-20T22:08:08.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f3680fb7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f367c0b35e0 0x7f367c15b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f3680fb7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0b4080 0x7f367c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f3680fb7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0d3290 0x7f367c15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f3680fb7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f367c1635f0 con 0x7f367c0d3290 2024-05-20T22:08:08.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f3680fb7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f367c1604f0 con 0x7f367c0b4080 2024-05-20T22:08:08.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f3680fb7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f367c15ff30 con 0x7f367c0b35e0 2024-05-20T22:08:08.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f367ad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0d3290 0x7f367c15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f367ad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0d3290 0x7f367c15d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52998/0 (socket says 172.21.15.55:52998) 2024-05-20T22:08:08.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f367ad9d640 1 -- 172.21.15.55:0/1497580599 learned_addr learned my addr 172.21.15.55:0/1497580599 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f367a59c640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f367c0b35e0 0x7f367c15b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f3679d9b640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0b4080 0x7f367c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f367ad9d640 1 -- 172.21.15.55:0/1497580599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f367c0b35e0 msgr2=0x7f367c15b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f367ad9d640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f367c0b35e0 0x7f367c15b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f367ad9d640 1 -- 172.21.15.55:0/1497580599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0b4080 msgr2=0x7f367c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f367ad9d640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0b4080 0x7f367c0b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f367ad9d640 1 -- 172.21.15.55:0/1497580599 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c15ba70 con 0x7f367c0d3290 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f3679d9b640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0b4080 0x7f367c0b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.497+0000 7f367ad9d640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0d3290 0x7f367c15d9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f366c00b3e0 tx=0x7f366c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.498+0000 7f3672ffd640 1 -- 172.21.15.55:0/1497580599 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c015030 con 0x7f367c0d3290 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.498+0000 7f3672ffd640 1 -- 172.21.15.55:0/1497580599 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366c004020 con 0x7f367c0d3290 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.498+0000 7f3680fb7640 1 -- 172.21.15.55:0/1497580599 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367c15bef0 con 0x7f367c0d3290 2024-05-20T22:08:08.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.498+0000 7f3672ffd640 1 -- 172.21.15.55:0/1497580599 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c004490 con 0x7f367c0d3290 2024-05-20T22:08:08.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.498+0000 7f3680fb7640 1 -- 172.21.15.55:0/1497580599 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367c15c4b0 con 0x7f367c0d3290 2024-05-20T22:08:08.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.499+0000 7f3680fb7640 1 -- 172.21.15.55:0/1497580599 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f367c15bc40 con 0x7f367c0d3290 2024-05-20T22:08:08.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.499+0000 7f3672ffd640 1 -- 172.21.15.55:0/1497580599 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f366c018070 con 0x7f367c0d3290 2024-05-20T22:08:08.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.500+0000 7f3672ffd640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f365c06c000 0x7f365c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:08.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.500+0000 7f3672ffd640 1 -- 172.21.15.55:0/1497580599 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f366c08acd0 con 0x7f367c0d3290 2024-05-20T22:08:08.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.500+0000 7f367a59c640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f365c06c000 0x7f365c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:08.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.500+0000 7f367a59c640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f365c06c000 0x7f365c06e510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f36740012e0 tx=0x7f3674001630 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:08.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.505+0000 7f3672ffd640 1 -- 172.21.15.55:0/1497580599 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f366c059210 con 0x7f367c0d3290 2024-05-20T22:08:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.704+0000 7f3680fb7640 1 -- 172.21.15.55:0/1497580599 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f367c0d2ec0 con 0x7f367c0d3290 2024-05-20T22:08:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.704+0000 7f3672ffd640 1 -- 172.21.15.55:0/1497580599 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f366c058b10 con 0x7f367c0d3290 2024-05-20T22:08:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-20T22:08:08.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.706+0000 7f3680fb7640 1 -- 172.21.15.55:0/1497580599 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f365c06c000 msgr2=0x7f365c06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.706+0000 7f3680fb7640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f365c06c000 0x7f365c06e510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f36740012e0 tx=0x7f3674001630 comp rx=0 tx=0).stop 2024-05-20T22:08:08.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.706+0000 7f3680fb7640 1 -- 172.21.15.55:0/1497580599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0d3290 msgr2=0x7f367c15d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:08.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.706+0000 7f3680fb7640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0d3290 0x7f367c15d9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f366c00b3e0 tx=0x7f366c00b8a0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.706+0000 7f3680fb7640 1 -- 172.21.15.55:0/1497580599 shutdown_connections 2024-05-20T22:08:08.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.706+0000 7f3680fb7640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f365c06c000 0x7f365c06e510 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.706+0000 7f3680fb7640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f367c0d3290 0x7f367c15d9f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.706+0000 7f3680fb7640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0b4080 0x7f367c0b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.706+0000 7f3680fb7640 1 --2- 172.21.15.55:0/1497580599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f367c0b35e0 0x7f367c15b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:08.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.706+0000 7f3680fb7640 1 -- 172.21.15.55:0/1497580599 >> 172.21.15.55:0/1497580599 conn(0x7f367c0d51b0 msgr2=0x7f367c159580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:08.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.707+0000 7f3680fb7640 1 -- 172.21.15.55:0/1497580599 shutdown_connections 2024-05-20T22:08:08.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:08.707+0000 7f3680fb7640 1 -- 172.21.15.55:0/1497580599 wait complete. 2024-05-20T22:08:08.719 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi055.front.sepia.ceph.com... 2024-05-20T22:08:08.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:08.720 DEBUG:teuthology.orchestra.run.smithi055:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-20T22:08:08.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi055.stderr:ceph-fuse[44456]: fuse finished with error 0 and tester_r 0 2024-05-20T22:08:08.747 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:08:08.837 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:08:08.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:08.838 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-20T22:08:08.859 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:09.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.013+0000 7fd8d2545640 1 Processor -- start 2024-05-20T22:08:09.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.013+0000 7fd8d2545640 1 -- start start 2024-05-20T22:08:09.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.014+0000 7fd8d2545640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8cc0d1f40 0x7fd8cc0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.014+0000 7fd8d2545640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc06dfd0 0x7fd8cc0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.014+0000 7fd8d2545640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8cc0b2720 con 0x7fd8cc06e520 2024-05-20T22:08:09.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.014+0000 7fd8d2545640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fd8cc06bd90 con 0x7fd8cc0d1f40 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.014+0000 7fd8d2545640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc0cd330 con 0x7fd8cc06dfd0 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.014+0000 7fd8cb7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8cc0d1f40 0x7fd8cc0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.014+0000 7fd8cb7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8cc0d1f40 0x7fd8cc0ce000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57772/0 (socket says 172.21.15.55:57772) 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.014+0000 7fd8cb7fe640 1 -- 172.21.15.55:0/1486032975 learned_addr learned my addr 172.21.15.55:0/1486032975 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.015+0000 7fd8d0ae2640 1 --2- 172.21.15.55:0/1486032975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc06dfd0 0x7fd8cc0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.015+0000 7fd8cb7fe640 1 -- 172.21.15.55:0/1486032975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc06dfd0 msgr2=0x7fd8cc0ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.015+0000 7fd8cb7fe640 1 --2- 172.21.15.55:0/1486032975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc06dfd0 0x7fd8cc0ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.015+0000 7fd8cb7fe640 1 -- 172.21.15.55:0/1486032975 >> v1:172.21.15.138:6789/0 conn(0x7fd8cc06e520 legacy=0x7fd8cc0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.015+0000 7fd8cb7fe640 1 -- 172.21.15.55:0/1486032975 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc0cd560 con 0x7fd8cc0d1f40 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.015+0000 7fd8d0ae2640 1 --2- 172.21.15.55:0/1486032975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc06dfd0 0x7fd8cc0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.015+0000 7fd8cb7fe640 1 --2- 172.21.15.55:0/1486032975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc0d1f40 0x7fd8cc0ce000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009870 tx=0x7fd8c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab6fd4afc8554fb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8caffd640 1 -- 172.21.15.55:0/1486032975 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0037070 con 0x7fd8cc0d1f40 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8caffd640 1 -- 172.21.15.55:0/1486032975 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c0032440 con 0x7fd8cc0d1f40 2024-05-20T22:08:09.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8caffd640 1 -- 172.21.15.55:0/1486032975 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0032810 con 0x7fd8cc0d1f40 2024-05-20T22:08:09.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8d2545640 1 -- 172.21.15.55:0/1486032975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc0d1f40 msgr2=0x7fd8cc0ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8d2545640 1 --2- 172.21.15.55:0/1486032975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc0d1f40 0x7fd8cc0ce000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009870 tx=0x7fd8c002a270 comp rx=0 tx=0).stop 2024-05-20T22:08:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8d2545640 1 -- 172.21.15.55:0/1486032975 shutdown_connections 2024-05-20T22:08:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8d2545640 1 --2- 172.21.15.55:0/1486032975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc06dfd0 0x7fd8cc0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8d2545640 1 --2- 172.21.15.55:0/1486032975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc0d1f40 0x7fd8cc0ce000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8d2545640 1 -- 172.21.15.55:0/1486032975 >> 172.21.15.55:0/1486032975 conn(0x7fd8cc0b6580 msgr2=0x7fd8cc0b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8d2545640 1 -- 172.21.15.55:0/1486032975 shutdown_connections 2024-05-20T22:08:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.016+0000 7fd8d2545640 1 -- 172.21.15.55:0/1486032975 wait complete. 2024-05-20T22:08:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.018+0000 7fd8d2545640 1 Processor -- start 2024-05-20T22:08:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.018+0000 7fd8d2545640 1 -- start start 2024-05-20T22:08:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8d2545640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc06dfd0 0x7fd8cc0c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8d2545640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc06e520 0x7fd8cc14fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8d2545640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc15f910 0x7fd8cc0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8d2545640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc0b2160 con 0x7fd8cc06e520 2024-05-20T22:08:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8d2545640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc0cd800 con 0x7fd8cc06dfd0 2024-05-20T22:08:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8d2545640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc15d4f0 con 0x7fd8cc15f910 2024-05-20T22:08:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8cb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc06e520 0x7fd8cc14fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8cb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc06e520 0x7fd8cc14fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57774/0 (socket says 172.21.15.55:57774) 2024-05-20T22:08:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8cb7fe640 1 -- 172.21.15.55:0/716587002 learned_addr learned my addr 172.21.15.55:0/716587002 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8d0ae2640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc15f910 0x7fd8cc0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8cbfff640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc06dfd0 0x7fd8cc0c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8cb7fe640 1 -- 172.21.15.55:0/716587002 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc15f910 msgr2=0x7fd8cc0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8cb7fe640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc15f910 0x7fd8cc0c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8cb7fe640 1 -- 172.21.15.55:0/716587002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc06dfd0 msgr2=0x7fd8cc0c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8cb7fe640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc06dfd0 0x7fd8cc0c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8cb7fe640 1 -- 172.21.15.55:0/716587002 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc15f610 con 0x7fd8cc06e520 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8d0ae2640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc15f910 0x7fd8cc0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8cb7fe640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc06e520 0x7fd8cc14fcb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009870 tx=0x7fd8c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.020+0000 7fd8c8ff9640 1 -- 172.21.15.55:0/716587002 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0041020 con 0x7fd8cc06e520 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.021+0000 7fd8c8ff9640 1 -- 172.21.15.55:0/716587002 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c00306c0 con 0x7fd8cc06e520 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.021+0000 7fd8d2545640 1 -- 172.21.15.55:0/716587002 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc0cb590 con 0x7fd8cc06e520 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.021+0000 7fd8c8ff9640 1 -- 172.21.15.55:0/716587002 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0037040 con 0x7fd8cc06e520 2024-05-20T22:08:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.021+0000 7fd8d2545640 1 -- 172.21.15.55:0/716587002 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc0cb830 con 0x7fd8cc06e520 2024-05-20T22:08:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.021+0000 7fd8d2545640 1 -- 172.21.15.55:0/716587002 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc069090 con 0x7fd8cc06e520 2024-05-20T22:08:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.022+0000 7fd8c8ff9640 1 -- 172.21.15.55:0/716587002 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8c0044020 con 0x7fd8cc06e520 2024-05-20T22:08:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.022+0000 7fd8c8ff9640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd8b806be60 0x7fd8b806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.023+0000 7fd8c8ff9640 1 -- 172.21.15.55:0/716587002 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd8c00ab7a0 con 0x7fd8cc06e520 2024-05-20T22:08:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.023+0000 7fd8cbfff640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd8b806be60 0x7fd8b806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.023+0000 7fd8cbfff640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd8b806be60 0x7fd8b806e370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc007fd0 tx=0x7fd8bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.027+0000 7fd8c8ff9640 1 -- 172.21.15.55:0/716587002 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd8c004b070 con 0x7fd8cc06e520 2024-05-20T22:08:09.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.211+0000 7fd8d2545640 1 -- 172.21.15.55:0/716587002 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8cc0d1af0 con 0x7fd8cc06e520 2024-05-20T22:08:09.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.212+0000 7fd8c8ff9640 1 -- 172.21.15.55:0/716587002 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fd8c0079e10 con 0x7fd8cc06e520 2024-05-20T22:08:09.213 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:09.213 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.214+0000 7fd8d2545640 1 -- 172.21.15.55:0/716587002 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd8b806be60 msgr2=0x7fd8b806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.214+0000 7fd8d2545640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd8b806be60 0x7fd8b806e370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc007fd0 tx=0x7fd8bc009040 comp rx=0 tx=0).stop 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.214+0000 7fd8d2545640 1 -- 172.21.15.55:0/716587002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc06e520 msgr2=0x7fd8cc14fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.214+0000 7fd8d2545640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc06e520 0x7fd8cc14fcb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009870 tx=0x7fd8c00046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.214+0000 7fd8d2545640 1 -- 172.21.15.55:0/716587002 shutdown_connections 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.214+0000 7fd8d2545640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd8b806be60 0x7fd8b806e370 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.214+0000 7fd8d2545640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8cc15f910 0x7fd8cc0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.214+0000 7fd8d2545640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8cc06e520 0x7fd8cc14fcb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.214+0000 7fd8d2545640 1 --2- 172.21.15.55:0/716587002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc06dfd0 0x7fd8cc0c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.215+0000 7fd8d2545640 1 -- 172.21.15.55:0/716587002 >> 172.21.15.55:0/716587002 conn(0x7fd8cc0b6580 msgr2=0x7fd8cc06cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.215+0000 7fd8d2545640 1 -- 172.21.15.55:0/716587002 shutdown_connections 2024-05-20T22:08:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.215+0000 7fd8d2545640 1 -- 172.21.15.55:0/716587002 wait complete. 2024-05-20T22:08:09.227 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:09.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.341+0000 7fb256502640 1 Processor -- start 2024-05-20T22:08:09.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.341+0000 7fb256502640 1 -- start start 2024-05-20T22:08:09.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb256502640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb2500b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb256502640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb25015f100 0x7fb25006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb256502640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2500d3dd0 con 0x7fb25006c5c0 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb256502640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb2500d40b0 con 0x7fb25015f100 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb256502640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb2500d4390 con 0x7fb2500b3b30 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24ffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb2500b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24f7fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb25015f100 0x7fb25006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24ffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb2500b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:46954/0 (socket says 172.21.15.55:46954) 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb254a9f640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb25006c5c0 0x7fb250162b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:37656/0 (socket says 172.21.15.55:37656) 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24f7fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb25015f100 0x7fb25006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50636/0 (socket says 172.21.15.55:50636) 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24ffff640 1 -- 172.21.15.55:0/1019951497 learned_addr learned my addr 172.21.15.55:0/1019951497 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24ffff640 1 -- 172.21.15.55:0/1019951497 >> v2:172.21.15.138:3300/0 conn(0x7fb25015f100 msgr2=0x7fb25006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24ffff640 1 --2- 172.21.15.55:0/1019951497 >> v2:172.21.15.138:3300/0 conn(0x7fb25015f100 0x7fb25006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24ffff640 1 -- 172.21.15.55:0/1019951497 >> v1:172.21.15.138:6789/0 conn(0x7fb25006c5c0 legacy=0x7fb250162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24effd640 1 -- 172.21.15.55:0/1019951497 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2026536782 0 0) 0x7fb2500d3dd0 con 0x7fb25006c5c0 2024-05-20T22:08:09.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24ffff640 1 -- 172.21.15.55:0/1019951497 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25006b5a0 con 0x7fb2500b3b30 2024-05-20T22:08:09.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.343+0000 7fb24f7fe640 1 --2- 172.21.15.55:0/1019951497 >> v2:172.21.15.138:3300/0 conn(0x7fb25015f100 0x7fb25006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:09.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.344+0000 7fb24ffff640 1 --2- 172.21.15.55:0/1019951497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb2500b7fe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb240009870 tx=0x7fb24002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400abfb0833402cf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:09.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.344+0000 7fb24effd640 1 -- 172.21.15.55:0/1019951497 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240037070 con 0x7fb2500b3b30 2024-05-20T22:08:09.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.344+0000 7fb24effd640 1 -- 172.21.15.55:0/1019951497 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb240032440 con 0x7fb2500b3b30 2024-05-20T22:08:09.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.344+0000 7fb24effd640 1 -- 172.21.15.55:0/1019951497 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240032810 con 0x7fb2500b3b30 2024-05-20T22:08:09.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.345+0000 7fb256502640 1 -- 172.21.15.55:0/1019951497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 msgr2=0x7fb2500b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.345+0000 7fb256502640 1 --2- 172.21.15.55:0/1019951497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb2500b7fe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb240009870 tx=0x7fb24002a270 comp rx=0 tx=0).stop 2024-05-20T22:08:09.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.345+0000 7fb256502640 1 -- 172.21.15.55:0/1019951497 shutdown_connections 2024-05-20T22:08:09.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.345+0000 7fb256502640 1 --2- 172.21.15.55:0/1019951497 >> v2:172.21.15.138:3300/0 conn(0x7fb25015f100 0x7fb25006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.345+0000 7fb256502640 1 --2- 172.21.15.55:0/1019951497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb2500b7fe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.345+0000 7fb256502640 1 -- 172.21.15.55:0/1019951497 >> 172.21.15.55:0/1019951497 conn(0x7fb2500d51d0 msgr2=0x7fb2500b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.345+0000 7fb256502640 1 -- 172.21.15.55:0/1019951497 shutdown_connections 2024-05-20T22:08:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.345+0000 7fb256502640 1 -- 172.21.15.55:0/1019951497 wait complete. 2024-05-20T22:08:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.347+0000 7fb256502640 1 Processor -- start 2024-05-20T22:08:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.347+0000 7fb256502640 1 -- start start 2024-05-20T22:08:09.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.348+0000 7fb256502640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25006c5c0 0x7fb250161270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.348+0000 7fb256502640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb250155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.348+0000 7fb256502640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25015f100 0x7fb250155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.348+0000 7fb256502640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb25006c9d0 con 0x7fb25015f100 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.348+0000 7fb256502640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb25006bb60 con 0x7fb25006c5c0 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.348+0000 7fb256502640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb25006cb50 con 0x7fb2500b3b30 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb254a9f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25015f100 0x7fb250155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb254a9f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25015f100 0x7fb250155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57782/0 (socket says 172.21.15.55:57782) 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb254a9f640 1 -- 172.21.15.55:0/2057156724 learned_addr learned my addr 172.21.15.55:0/2057156724 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb24f7fe640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb250155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb24ffff640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25006c5c0 0x7fb250161270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb254a9f640 1 -- 172.21.15.55:0/2057156724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 msgr2=0x7fb250155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb254a9f640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb250155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb254a9f640 1 -- 172.21.15.55:0/2057156724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25006c5c0 msgr2=0x7fb250161270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb254a9f640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25006c5c0 0x7fb250161270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb254a9f640 1 -- 172.21.15.55:0/2057156724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2501563e0 con 0x7fb25015f100 2024-05-20T22:08:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb24f7fe640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb250155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb254a9f640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25015f100 0x7fb250155b90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c730 tx=0x7fb24800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb24cff9640 1 -- 172.21.15.55:0/2057156724 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248018070 con 0x7fb25015f100 2024-05-20T22:08:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb24cff9640 1 -- 172.21.15.55:0/2057156724 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb24800ce40 con 0x7fb25015f100 2024-05-20T22:08:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb256502640 1 -- 172.21.15.55:0/2057156724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2501566c0 con 0x7fb25015f100 2024-05-20T22:08:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.349+0000 7fb24cff9640 1 -- 172.21.15.55:0/2057156724 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24800a800 con 0x7fb25015f100 2024-05-20T22:08:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.350+0000 7fb256502640 1 -- 172.21.15.55:0/2057156724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb250156c80 con 0x7fb25015f100 2024-05-20T22:08:09.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.351+0000 7fb256502640 1 -- 172.21.15.55:0/2057156724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb250069090 con 0x7fb25015f100 2024-05-20T22:08:09.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.351+0000 7fb24cff9640 1 -- 172.21.15.55:0/2057156724 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb24801b070 con 0x7fb25015f100 2024-05-20T22:08:09.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.352+0000 7fb24cff9640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb23c06be30 0x7fb23c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.352+0000 7fb24ffff640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb23c06be30 0x7fb23c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.352+0000 7fb24cff9640 1 -- 172.21.15.55:0/2057156724 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb24808a600 con 0x7fb25015f100 2024-05-20T22:08:09.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.352+0000 7fb24ffff640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb23c06be30 0x7fb23c06e340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb240034000 tx=0x7fb24002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:09.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.357+0000 7fb24cff9640 1 -- 172.21.15.55:0/2057156724 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb248058620 con 0x7fb25015f100 2024-05-20T22:08:09.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.542+0000 7fb256502640 1 -- 172.21.15.55:0/2057156724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb250157280 con 0x7fb25015f100 2024-05-20T22:08:09.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.543+0000 7fb24cff9640 1 -- 172.21.15.55:0/2057156724 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fb24805c920 con 0x7fb25015f100 2024-05-20T22:08:09.544 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:09.545 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:09.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 -- 172.21.15.55:0/2057156724 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb23c06be30 msgr2=0x7fb23c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb23c06be30 0x7fb23c06e340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb240034000 tx=0x7fb24002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:09.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 -- 172.21.15.55:0/2057156724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25015f100 msgr2=0x7fb250155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25015f100 0x7fb250155b90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c730 tx=0x7fb24800cbf0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 -- 172.21.15.55:0/2057156724 shutdown_connections 2024-05-20T22:08:09.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb23c06be30 0x7fb23c06e340 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb25015f100 0x7fb250155b90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2500b3b30 0x7fb250155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 --2- 172.21.15.55:0/2057156724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25006c5c0 0x7fb250161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 -- 172.21.15.55:0/2057156724 >> 172.21.15.55:0/2057156724 conn(0x7fb2500d51d0 msgr2=0x7fb2500cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:09.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 -- 172.21.15.55:0/2057156724 shutdown_connections 2024-05-20T22:08:09.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.545+0000 7fb256502640 1 -- 172.21.15.55:0/2057156724 wait complete. 2024-05-20T22:08:09.558 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T22:08:09.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.676+0000 7f1f59c5a640 1 Processor -- start 2024-05-20T22:08:09.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.676+0000 7f1f59c5a640 1 -- start start 2024-05-20T22:08:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f59c5a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f540b40e0 0x7f1f540cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f59c5a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f541639f0 con 0x7f1f5415f1c0 2024-05-20T22:08:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f59c5a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f540d0400 con 0x7f1f5406c510 2024-05-20T22:08:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f59c5a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1f540d3b40 con 0x7f1f540b40e0 2024-05-20T22:08:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f4bfff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1f5415f1c0 0x7f1f54162bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46254/0 (socket says 172.21.15.55:46254) 2024-05-20T22:08:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f4bfff640 1 -- 172.21.15.55:0/3428297793 learned_addr learned my addr 172.21.15.55:0/3428297793 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f537fe640 1 --2- 172.21.15.55:0/3428297793 >> v2:172.21.15.138:3300/0 conn(0x7f1f540b40e0 0x7f1f540cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f52ffd640 1 -- 172.21.15.55:0/3428297793 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1218606421 0 0) 0x7f1f541639f0 con 0x7f1f5415f1c0 2024-05-20T22:08:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f52ffd640 1 -- 172.21.15.55:0/3428297793 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f40003520 con 0x7f1f5415f1c0 2024-05-20T22:08:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f537fe640 1 -- 172.21.15.55:0/3428297793 >> v1:172.21.15.138:6789/0 conn(0x7f1f5406c510 legacy=0x7f1f541632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f537fe640 1 -- 172.21.15.55:0/3428297793 >> v1:172.21.15.55:6789/0 conn(0x7f1f5415f1c0 legacy=0x7f1f54162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f52ffd640 1 -- 172.21.15.55:0/3428297793 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164018332 0 0) 0x7f1f540d0400 con 0x7f1f5406c510 2024-05-20T22:08:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.678+0000 7f1f537fe640 1 -- 172.21.15.55:0/3428297793 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f540d3d90 con 0x7f1f540b40e0 2024-05-20T22:08:09.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.679+0000 7f1f537fe640 1 --2- 172.21.15.55:0/3428297793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f540b40e0 0x7f1f540cf730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f440097a0 tx=0x7f1f4402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e469b7fc1fd8ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:09.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.680+0000 7f1f52ffd640 1 -- 172.21.15.55:0/3428297793 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44037070 con 0x7f1f540b40e0 2024-05-20T22:08:09.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.680+0000 7f1f52ffd640 1 -- 172.21.15.55:0/3428297793 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f44032440 con 0x7f1f540b40e0 2024-05-20T22:08:09.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.680+0000 7f1f52ffd640 1 -- 172.21.15.55:0/3428297793 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44032810 con 0x7f1f540b40e0 2024-05-20T22:08:09.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.680+0000 7f1f59c5a640 1 -- 172.21.15.55:0/3428297793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f540b40e0 msgr2=0x7f1f540cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.680+0000 7f1f59c5a640 1 --2- 172.21.15.55:0/3428297793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f540b40e0 0x7f1f540cf730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f440097a0 tx=0x7f1f4402a1a0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.680+0000 7f1f59c5a640 1 -- 172.21.15.55:0/3428297793 shutdown_connections 2024-05-20T22:08:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.681+0000 7f1f59c5a640 1 --2- 172.21.15.55:0/3428297793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f540b40e0 0x7f1f540cf730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.681+0000 7f1f59c5a640 1 -- 172.21.15.55:0/3428297793 >> 172.21.15.55:0/3428297793 conn(0x7f1f540b65e0 msgr2=0x7f1f540b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.681+0000 7f1f59c5a640 1 -- 172.21.15.55:0/3428297793 shutdown_connections 2024-05-20T22:08:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.681+0000 7f1f59c5a640 1 -- 172.21.15.55:0/3428297793 wait complete. 2024-05-20T22:08:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.682+0000 7f1f59c5a640 1 Processor -- start 2024-05-20T22:08:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.682+0000 7f1f59c5a640 1 -- start start 2024-05-20T22:08:09.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f59c5a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f540b40e0 0x7f1f54147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f59c5a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f5415f1c0 0x7f1f540b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f59c5a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f540b4c50 0x7f1f540c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f59c5a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1f540b3d10 con 0x7f1f540b40e0 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f59c5a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f540b3b90 con 0x7f1f540b4c50 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f59c5a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1f540b3640 con 0x7f1f5415f1c0 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f540b40e0 0x7f1f54147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f540b40e0 0x7f1f54147500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57796/0 (socket says 172.21.15.55:57796) 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f537fe640 1 -- 172.21.15.55:0/940308026 learned_addr learned my addr 172.21.15.55:0/940308026 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f53fff640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f540b4c50 0x7f1f540c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f4bfff640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f5415f1c0 0x7f1f540b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f537fe640 1 -- 172.21.15.55:0/940308026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f5415f1c0 msgr2=0x7f1f540b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f537fe640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f5415f1c0 0x7f1f540b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f537fe640 1 -- 172.21.15.55:0/940308026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f540b4c50 msgr2=0x7f1f540c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f537fe640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f540b4c50 0x7f1f540c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f537fe640 1 -- 172.21.15.55:0/940308026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f540c9db0 con 0x7f1f540b40e0 2024-05-20T22:08:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f537fe640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f540b40e0 0x7f1f54147500 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f44035040 tx=0x7f1f440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f50ff9640 1 -- 172.21.15.55:0/940308026 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44041020 con 0x7f1f540b40e0 2024-05-20T22:08:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f50ff9640 1 -- 172.21.15.55:0/940308026 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4403d6e0 con 0x7f1f540b40e0 2024-05-20T22:08:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.684+0000 7f1f50ff9640 1 -- 172.21.15.55:0/940308026 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44037040 con 0x7f1f540b40e0 2024-05-20T22:08:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.685+0000 7f1f59c5a640 1 -- 172.21.15.55:0/940308026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f54147a40 con 0x7f1f540b40e0 2024-05-20T22:08:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.685+0000 7f1f59c5a640 1 -- 172.21.15.55:0/940308026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f540ca590 con 0x7f1f540b40e0 2024-05-20T22:08:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.685+0000 7f1f59c5a640 1 -- 172.21.15.55:0/940308026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f54069090 con 0x7f1f540b40e0 2024-05-20T22:08:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.686+0000 7f1f50ff9640 1 -- 172.21.15.55:0/940308026 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f44044020 con 0x7f1f540b40e0 2024-05-20T22:08:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.687+0000 7f1f50ff9640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1f3406be30 0x7f1f3406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.687+0000 7f1f50ff9640 1 -- 172.21.15.55:0/940308026 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1f440ab370 con 0x7f1f540b40e0 2024-05-20T22:08:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.687+0000 7f1f4bfff640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1f3406be30 0x7f1f3406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.687+0000 7f1f4bfff640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1f3406be30 0x7f1f3406e340 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1f540b5350 tx=0x7f1f3c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:09.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.691+0000 7f1f50ff9640 1 -- 172.21.15.55:0/940308026 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1f4404b020 con 0x7f1f540b40e0 2024-05-20T22:08:09.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:09.914+0000 7f1f59c5a640 1 -- 172.21.15.55:0/940308026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1f540d2f10 con 0x7f1f540b40e0 2024-05-20T22:08:10.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.754+0000 7f1f50ff9640 1 -- 172.21.15.55:0/940308026 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f1f44079300 con 0x7f1f540b40e0 2024-05-20T22:08:10.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 -- 172.21.15.55:0/940308026 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1f3406be30 msgr2=0x7f1f3406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:10.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1f3406be30 0x7f1f3406e340 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1f540b5350 tx=0x7f1f3c009040 comp rx=0 tx=0).stop 2024-05-20T22:08:10.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 -- 172.21.15.55:0/940308026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f540b40e0 msgr2=0x7f1f54147500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:10.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f540b40e0 0x7f1f54147500 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f44035040 tx=0x7f1f440046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:10.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 -- 172.21.15.55:0/940308026 shutdown_connections 2024-05-20T22:08:10.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1f3406be30 0x7f1f3406e340 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:10.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f540b4c50 0x7f1f540c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:10.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f5415f1c0 0x7f1f540b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:10.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 --2- 172.21.15.55:0/940308026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f540b40e0 0x7f1f54147500 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:10.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 -- 172.21.15.55:0/940308026 >> 172.21.15.55:0/940308026 conn(0x7f1f540b65e0 msgr2=0x7f1f540b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:10.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 -- 172.21.15.55:0/940308026 shutdown_connections 2024-05-20T22:08:10.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.757+0000 7f1f59c5a640 1 -- 172.21.15.55:0/940308026 wait complete. 2024-05-20T22:08:10.772 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-20T22:08:10.772 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T22:08:10.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.897+0000 7fd790614640 1 Processor -- start 2024-05-20T22:08:10.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.897+0000 7fd790614640 1 -- start start 2024-05-20T22:08:10.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.898+0000 7fd790614640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd78806cb70 0x7fd788169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.898+0000 7fd790614640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd78806b6a0 con 0x7fd78815f080 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.898+0000 7fd790614640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd78806b980 con 0x7fd7880cf680 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.898+0000 7fd790614640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd78806bc60 con 0x7fd78806cb70 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78dbaf640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd78815f080 0x7fd78806c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46264/0 (socket says 172.21.15.55:46264) 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78dbaf640 1 -- 172.21.15.55:0/789086200 learned_addr learned my addr 172.21.15.55:0/789086200 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78ebb1640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd78806cb70 0x7fd788169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607005797 0 0) 0x7fd78806b6a0 con 0x7fd78815f080 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd774003520 con 0x7fd78815f080 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2270944194 0 0) 0x7fd78806b980 con 0x7fd7880cf680 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd78806b6a0 con 0x7fd7880cf680 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 313780736 0 0) 0x7fd774003520 con 0x7fd78815f080 2024-05-20T22:08:10.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd78806cb70 msgr2=0x7fd788169ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd78806cb70 0x7fd788169ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 >> v1:172.21.15.138:6789/0 conn(0x7fd7880cf680 legacy=0x7fd7880b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78806bf40 con 0x7fd78815f080 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fd77c0019f0 con 0x7fd78815f080 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 >> v1:172.21.15.55:6789/0 conn(0x7fd78815f080 legacy=0x7fd78806c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd774003b70 0x7fd774003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd78806cb70 0x7fd774004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 0x7fd774005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd78806bc60 con 0x7fd774004d40 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd77400bb80 con 0x7fd774003b70 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.899+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd77400bd80 con 0x7fd78806cb70 2024-05-20T22:08:10.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.900+0000 7fd78ebb1640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 0x7fd774005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.900+0000 7fd78e3b0640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd774003b70 0x7fd774003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.900+0000 7fd78dbaf640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd78806cb70 0x7fd774004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.900+0000 7fd78ebb1640 1 -- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd78806cb70 msgr2=0x7fd774004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.900+0000 7fd78ebb1640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd78806cb70 0x7fd774004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.900+0000 7fd78ebb1640 1 -- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd774003b70 msgr2=0x7fd774003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.900+0000 7fd78ebb1640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd774003b70 0x7fd774003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.900+0000 7fd78ebb1640 1 -- 172.21.15.55:0/789086200 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd774003520 con 0x7fd774004d40 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.901+0000 7fd78ebb1640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 0x7fd774005150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd780009ac0 tx=0x7fd774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c786b2341a237c2f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.901+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780036070 con 0x7fd774004d40 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.901+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd780031400 con 0x7fd774004d40 2024-05-20T22:08:10.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.901+0000 7fd78d3ae640 1 -- 172.21.15.55:0/789086200 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780009690 con 0x7fd774004d40 2024-05-20T22:08:10.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.901+0000 7fd790614640 1 -- 172.21.15.55:0/789086200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 msgr2=0x7fd774005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:10.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.901+0000 7fd790614640 1 --2- 172.21.15.55:0/789086200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 0x7fd774005150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd780009ac0 tx=0x7fd774005830 comp rx=0 tx=0).stop 2024-05-20T22:08:10.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.901+0000 7fd78ebb1640 1 -- 172.21.15.55:0/789086200 reap_dead start 2024-05-20T22:08:10.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.902+0000 7fd790614640 1 -- 172.21.15.55:0/789086200 shutdown_connections 2024-05-20T22:08:10.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.902+0000 7fd790614640 1 -- 172.21.15.55:0/789086200 >> 172.21.15.55:0/789086200 conn(0x7fd7880d5450 msgr2=0x7fd7880b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:10.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.902+0000 7fd790614640 1 -- 172.21.15.55:0/789086200 shutdown_connections 2024-05-20T22:08:10.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.902+0000 7fd790614640 1 -- 172.21.15.55:0/789086200 wait complete. 2024-05-20T22:08:10.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.903+0000 7fd790614640 1 Processor -- start 2024-05-20T22:08:10.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.903+0000 7fd790614640 1 -- start start 2024-05-20T22:08:10.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd790614640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 0x7fd788156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:10.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd790614640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7881f93f0 0x7fd7881610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:10.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd790614640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7880b9d40 0x7fd7880ba150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:10.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd790614640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd7880b42a0 con 0x7fd774004d40 2024-05-20T22:08:10.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd790614640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd7880b4120 con 0x7fd7881f93f0 2024-05-20T22:08:10.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd790614640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd788167ed0 con 0x7fd7880b9d40 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78e3b0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 0x7fd788156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78e3b0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 0x7fd788156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57822/0 (socket says 172.21.15.55:57822) 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78e3b0640 1 -- 172.21.15.55:0/1840441804 learned_addr learned my addr 172.21.15.55:0/1840441804 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78dbaf640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7881f93f0 0x7fd7881610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78ebb1640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7880b9d40 0x7fd7880ba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78e3b0640 1 -- 172.21.15.55:0/1840441804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7880b9d40 msgr2=0x7fd7880ba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78e3b0640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7880b9d40 0x7fd7880ba150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78e3b0640 1 -- 172.21.15.55:0/1840441804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7881f93f0 msgr2=0x7fd7881610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78e3b0640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7881f93f0 0x7fd7881610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78e3b0640 1 -- 172.21.15.55:0/1840441804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7881492b0 con 0x7fd774004d40 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78ebb1640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7880b9d40 0x7fd7880ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd78e3b0640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 0x7fd788156750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd78400f7a0 tx=0x7fd78400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:10.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.905+0000 7fd77affd640 1 -- 172.21.15.55:0/1840441804 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78400fe50 con 0x7fd774004d40 2024-05-20T22:08:10.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.906+0000 7fd77affd640 1 -- 172.21.15.55:0/1840441804 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd784018070 con 0x7fd774004d40 2024-05-20T22:08:10.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.906+0000 7fd790614640 1 -- 172.21.15.55:0/1840441804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7880ba690 con 0x7fd774004d40 2024-05-20T22:08:10.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.906+0000 7fd77affd640 1 -- 172.21.15.55:0/1840441804 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78400d730 con 0x7fd774004d40 2024-05-20T22:08:10.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.906+0000 7fd790614640 1 -- 172.21.15.55:0/1840441804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7880b8cc0 con 0x7fd774004d40 2024-05-20T22:08:10.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.907+0000 7fd790614640 1 -- 172.21.15.55:0/1840441804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd788069090 con 0x7fd774004d40 2024-05-20T22:08:10.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.907+0000 7fd77affd640 1 -- 172.21.15.55:0/1840441804 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd78400d8d0 con 0x7fd774004d40 2024-05-20T22:08:10.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.908+0000 7fd77affd640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd76c06be30 0x7fd76c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:10.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.908+0000 7fd77affd640 1 -- 172.21.15.55:0/1840441804 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd78408b1b0 con 0x7fd774004d40 2024-05-20T22:08:10.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.908+0000 7fd78dbaf640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd76c06be30 0x7fd76c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:10.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.908+0000 7fd78dbaf640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd76c06be30 0x7fd76c06e340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd78815bb10 tx=0x7fd77c006040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:10.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:10.912+0000 7fd77affd640 1 -- 172.21.15.55:0/1840441804 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd7880ba690 con 0x7fd774004d40 2024-05-20T22:08:11.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.134+0000 7fd790614640 1 -- 172.21.15.55:0/1840441804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd7880d2e90 con 0x7fd774004d40 2024-05-20T22:08:11.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.757+0000 7fd77affd640 1 -- 172.21.15.55:0/1840441804 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7fd78401a310 con 0x7fd774004d40 2024-05-20T22:08:11.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 -- 172.21.15.55:0/1840441804 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd76c06be30 msgr2=0x7fd76c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:11.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd76c06be30 0x7fd76c06e340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd78815bb10 tx=0x7fd77c006040 comp rx=0 tx=0).stop 2024-05-20T22:08:11.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 -- 172.21.15.55:0/1840441804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 msgr2=0x7fd788156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:11.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 0x7fd788156750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd78400f7a0 tx=0x7fd78400fc60 comp rx=0 tx=0).stop 2024-05-20T22:08:11.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 -- 172.21.15.55:0/1840441804 shutdown_connections 2024-05-20T22:08:11.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd76c06be30 0x7fd76c06e340 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:11.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7880b9d40 0x7fd7880ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:11.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7881f93f0 0x7fd7881610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:11.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 --2- 172.21.15.55:0/1840441804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd774004d40 0x7fd788156750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:11.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 -- 172.21.15.55:0/1840441804 >> 172.21.15.55:0/1840441804 conn(0x7fd7880d5450 msgr2=0x7fd7880b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:11.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 -- 172.21.15.55:0/1840441804 shutdown_connections 2024-05-20T22:08:11.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.760+0000 7fd790614640 1 -- 172.21.15.55:0/1840441804 wait complete. 2024-05-20T22:08:11.775 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T22:08:11.775 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:08:11.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.895+0000 7ff74f909640 1 Processor -- start 2024-05-20T22:08:11.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.895+0000 7ff74f909640 1 -- start start 2024-05-20T22:08:11.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.896+0000 7ff74f909640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7480cece0 0x7ff74815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:11.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.896+0000 7ff74f909640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480b28b0 0x7ff7480d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:11.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.896+0000 7ff74f909640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7480d2040 con 0x7ff7480d1c30 2024-05-20T22:08:11.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.896+0000 7ff74f909640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff7480b2cc0 con 0x7ff7480cece0 2024-05-20T22:08:11.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.896+0000 7ff74f909640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff74806bd90 con 0x7ff7480b28b0 2024-05-20T22:08:11.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff74cea4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480b28b0 0x7ff7480d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:11.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff74d6a5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7480cece0 0x7ff74815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:11.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff74cea4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480b28b0 0x7ff7480d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47026/0 (socket says 172.21.15.55:47026) 2024-05-20T22:08:11.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff74cea4640 1 -- 172.21.15.55:0/2552389954 learned_addr learned my addr 172.21.15.55:0/2552389954 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:11.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff74d6a5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7480cece0 0x7ff74815cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50694/0 (socket says 172.21.15.55:50694) 2024-05-20T22:08:11.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff74dea6640 1 --1- 172.21.15.55:0/2552389954 >> v1:172.21.15.138:6789/0 conn(0x7ff7480d1c30 0x7ff7480cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:37700/0 (socket says 172.21.15.55:37700) 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff73ffff640 1 -- 172.21.15.55:0/2552389954 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122958742 0 0) 0x7ff740001490 con 0x7ff7480d1c30 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff74cea4640 1 -- 172.21.15.55:0/2552389954 >> v2:172.21.15.138:3300/0 conn(0x7ff7480cece0 msgr2=0x7ff74815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff74cea4640 1 --2- 172.21.15.55:0/2552389954 >> v2:172.21.15.138:3300/0 conn(0x7ff7480cece0 0x7ff74815cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff74cea4640 1 -- 172.21.15.55:0/2552389954 >> v1:172.21.15.138:6789/0 conn(0x7ff7480d1c30 legacy=0x7ff7480cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.897+0000 7ff74cea4640 1 -- 172.21.15.55:0/2552389954 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7480b2ee0 con 0x7ff7480b28b0 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.898+0000 7ff74cea4640 1 --2- 172.21.15.55:0/2552389954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480b28b0 0x7ff7480d16f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff738009870 tx=0x7ff73802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd7e7fef22451ca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.898+0000 7ff73ffff640 1 -- 172.21.15.55:0/2552389954 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738037070 con 0x7ff7480b28b0 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.898+0000 7ff73ffff640 1 -- 172.21.15.55:0/2552389954 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff738032440 con 0x7ff7480b28b0 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.898+0000 7ff73ffff640 1 -- 172.21.15.55:0/2552389954 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738032810 con 0x7ff7480b28b0 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.899+0000 7ff74f909640 1 -- 172.21.15.55:0/2552389954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480b28b0 msgr2=0x7ff7480d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.899+0000 7ff74f909640 1 --2- 172.21.15.55:0/2552389954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480b28b0 0x7ff7480d16f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff738009870 tx=0x7ff73802a270 comp rx=0 tx=0).stop 2024-05-20T22:08:11.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.899+0000 7ff74f909640 1 -- 172.21.15.55:0/2552389954 shutdown_connections 2024-05-20T22:08:11.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.899+0000 7ff74f909640 1 --2- 172.21.15.55:0/2552389954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480b28b0 0x7ff7480d16f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:11.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.899+0000 7ff74f909640 1 --2- 172.21.15.55:0/2552389954 >> v2:172.21.15.138:3300/0 conn(0x7ff7480cece0 0x7ff74815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:11.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.899+0000 7ff74f909640 1 -- 172.21.15.55:0/2552389954 >> 172.21.15.55:0/2552389954 conn(0x7ff7480b6420 msgr2=0x7ff7480b6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:11.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.899+0000 7ff74f909640 1 -- 172.21.15.55:0/2552389954 shutdown_connections 2024-05-20T22:08:11.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.899+0000 7ff74f909640 1 -- 172.21.15.55:0/2552389954 wait complete. 2024-05-20T22:08:11.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.901+0000 7ff74f909640 1 Processor -- start 2024-05-20T22:08:11.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.901+0000 7ff74f909640 1 -- start start 2024-05-20T22:08:11.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74f909640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7480b28b0 0x7ff74814ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:11.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74f909640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7480cece0 0x7ff74815d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:11.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74f909640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480d1c30 0x7ff74815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:11.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74f909640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff74806c580 con 0x7ff7480b28b0 2024-05-20T22:08:11.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74f909640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff74806c2a0 con 0x7ff7480cece0 2024-05-20T22:08:11.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74f909640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff74809a850 con 0x7ff7480d1c30 2024-05-20T22:08:11.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74d6a5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7480b28b0 0x7ff74814ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:11.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74d6a5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7480b28b0 0x7ff74814ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57836/0 (socket says 172.21.15.55:57836) 2024-05-20T22:08:11.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74d6a5640 1 -- 172.21.15.55:0/2776095216 learned_addr learned my addr 172.21.15.55:0/2776095216 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74cea4640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7480cece0 0x7ff74815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74dea6640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480d1c30 0x7ff74815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74d6a5640 1 -- 172.21.15.55:0/2776095216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480d1c30 msgr2=0x7ff74815dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.902+0000 7ff74d6a5640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480d1c30 0x7ff74815dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.903+0000 7ff74d6a5640 1 -- 172.21.15.55:0/2776095216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7480cece0 msgr2=0x7ff74815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.903+0000 7ff74d6a5640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7480cece0 0x7ff74815d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.903+0000 7ff74d6a5640 1 -- 172.21.15.55:0/2776095216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7480c8250 con 0x7ff7480b28b0 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.903+0000 7ff74dea6640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480d1c30 0x7ff74815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.903+0000 7ff74d6a5640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7480b28b0 0x7ff74814ab70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff74400e870 tx=0x7ff74400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.903+0000 7ff73dffb640 1 -- 172.21.15.55:0/2776095216 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744017070 con 0x7ff7480b28b0 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.903+0000 7ff73dffb640 1 -- 172.21.15.55:0/2776095216 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74400c940 con 0x7ff7480b28b0 2024-05-20T22:08:11.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.903+0000 7ff74f909640 1 -- 172.21.15.55:0/2776095216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7480c8e10 con 0x7ff7480b28b0 2024-05-20T22:08:11.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.903+0000 7ff73dffb640 1 -- 172.21.15.55:0/2776095216 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74400cd30 con 0x7ff7480b28b0 2024-05-20T22:08:11.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.903+0000 7ff74f909640 1 -- 172.21.15.55:0/2776095216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7480c93d0 con 0x7ff7480b28b0 2024-05-20T22:08:11.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.904+0000 7ff74f909640 1 -- 172.21.15.55:0/2776095216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff748069090 con 0x7ff7480b28b0 2024-05-20T22:08:11.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.905+0000 7ff73dffb640 1 -- 172.21.15.55:0/2776095216 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff74401a070 con 0x7ff7480b28b0 2024-05-20T22:08:11.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.905+0000 7ff73dffb640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff72c06be60 0x7ff72c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:11.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.905+0000 7ff73dffb640 1 -- 172.21.15.55:0/2776095216 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff74408b480 con 0x7ff7480b28b0 2024-05-20T22:08:11.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.906+0000 7ff74cea4640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff72c06be60 0x7ff72c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:11.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.906+0000 7ff74cea4640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff72c06be60 0x7ff72c06e370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff738034000 tx=0x7ff73802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:11.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:11.910+0000 7ff73dffb640 1 -- 172.21.15.55:0/2776095216 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff744059380 con 0x7ff7480b28b0 2024-05-20T22:08:12.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.134+0000 7ff74f909640 1 -- 172.21.15.55:0/2776095216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7480c4e80 con 0x7ff7480b28b0 2024-05-20T22:08:12.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.134+0000 7ff73dffb640 1 -- 172.21.15.55:0/2776095216 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7ff74405d680 con 0x7ff7480b28b0 2024-05-20T22:08:12.135 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:12.135 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:07:53.028952+0000","modified":"2024-05-20T22:08:11.135754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-05-20T22:08:12.136 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-20T22:08:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 -- 172.21.15.55:0/2776095216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff72c06be60 msgr2=0x7ff72c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff72c06be60 0x7ff72c06e370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff738034000 tx=0x7ff73802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 -- 172.21.15.55:0/2776095216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7480b28b0 msgr2=0x7ff74814ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7480b28b0 0x7ff74814ab70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff74400e870 tx=0x7ff74400ed30 comp rx=0 tx=0).stop 2024-05-20T22:08:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 -- 172.21.15.55:0/2776095216 shutdown_connections 2024-05-20T22:08:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff72c06be60 0x7ff72c06e370 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7480d1c30 0x7ff74815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7480cece0 0x7ff74815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 --2- 172.21.15.55:0/2776095216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7480b28b0 0x7ff74814ab70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 -- 172.21.15.55:0/2776095216 >> 172.21.15.55:0/2776095216 conn(0x7ff7480b6420 msgr2=0x7ff7480700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 -- 172.21.15.55:0/2776095216 shutdown_connections 2024-05-20T22:08:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.137+0000 7ff74f909640 1 -- 172.21.15.55:0/2776095216 wait complete. 2024-05-20T22:08:12.149 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:12.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.263+0000 7fded82fe640 1 Processor -- start 2024-05-20T22:08:12.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.263+0000 7fded82fe640 1 -- start start 2024-05-20T22:08:12.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.264+0000 7fded82fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fded00cd840 0x7fded00b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.264+0000 7fded82fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fded00d2e10 0x7fded00d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.264+0000 7fded82fe640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded015da90 con 0x7fded00b5830 2024-05-20T22:08:12.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.264+0000 7fded82fe640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fded00b6990 con 0x7fded00d2e10 2024-05-20T22:08:12.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.264+0000 7fded82fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fded00b5d80 con 0x7fded00cd840 2024-05-20T22:08:12.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.264+0000 7fded689b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fded00d2e10 0x7fded00d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.264+0000 7fded5899640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fded00b5830 0x7fded00d2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46270/0 (socket says 172.21.15.55:46270) 2024-05-20T22:08:12.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.264+0000 7fded689b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fded00d2e10 0x7fded00d1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57852/0 (socket says 172.21.15.55:57852) 2024-05-20T22:08:12.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.264+0000 7fded5899640 1 -- 172.21.15.55:0/3234454820 learned_addr learned my addr 172.21.15.55:0/3234454820 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:12.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.265+0000 7fded609a640 1 --2- 172.21.15.55:0/3234454820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fded00cd840 0x7fded00b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.265+0000 7fded689b640 1 -- 172.21.15.55:0/3234454820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fded00cd840 msgr2=0x7fded00b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:12.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.265+0000 7fded689b640 1 --2- 172.21.15.55:0/3234454820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fded00cd840 0x7fded00b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.265+0000 7fded689b640 1 -- 172.21.15.55:0/3234454820 >> v1:172.21.15.55:6789/0 conn(0x7fded00b5830 legacy=0x7fded00d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.265+0000 7fded689b640 1 -- 172.21.15.55:0/3234454820 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded00b6060 con 0x7fded00d2e10 2024-05-20T22:08:12.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.265+0000 7fded5098640 1 -- 172.21.15.55:0/3234454820 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2918487622 0 0) 0x7fded015da90 con 0x7fded00b5830 2024-05-20T22:08:12.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.265+0000 7fded689b640 1 --2- 172.21.15.55:0/3234454820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 0x7fded00d1890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fded00b6b10 tx=0x7fdec802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdf16d3572d9120 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.266+0000 7fded5098640 1 -- 172.21.15.55:0/3234454820 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8031600 con 0x7fded00d2e10 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.266+0000 7fded5098640 1 -- 172.21.15.55:0/3234454820 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec80317a0 con 0x7fded00d2e10 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.266+0000 7fded5098640 1 -- 172.21.15.55:0/3234454820 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8031b70 con 0x7fded00d2e10 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.266+0000 7fded82fe640 1 -- 172.21.15.55:0/3234454820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 msgr2=0x7fded00d1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.266+0000 7fded82fe640 1 --2- 172.21.15.55:0/3234454820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 0x7fded00d1890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fded00b6b10 tx=0x7fdec802a5d0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.266+0000 7fded82fe640 1 -- 172.21.15.55:0/3234454820 shutdown_connections 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.266+0000 7fded82fe640 1 --2- 172.21.15.55:0/3234454820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 0x7fded00d1890 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.266+0000 7fded82fe640 1 --2- 172.21.15.55:0/3234454820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fded00cd840 0x7fded00b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.266+0000 7fded82fe640 1 -- 172.21.15.55:0/3234454820 >> 172.21.15.55:0/3234454820 conn(0x7fded008bff0 msgr2=0x7fded008c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.267+0000 7fded82fe640 1 -- 172.21.15.55:0/3234454820 shutdown_connections 2024-05-20T22:08:12.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.267+0000 7fded82fe640 1 -- 172.21.15.55:0/3234454820 wait complete. 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.268+0000 7fded82fe640 1 Processor -- start 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.268+0000 7fded82fe640 1 -- start start 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded82fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fded00b5830 0x7fded0156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded82fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fded00cd840 0x7fded01510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded82fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 0x7fded00b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded82fe640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fded00b8160 con 0x7fded00d2e10 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded82fe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fded00b6620 con 0x7fded00b5830 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded82fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fded00b6e20 con 0x7fded00cd840 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded689b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 0x7fded00b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded689b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 0x7fded00b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57862/0 (socket says 172.21.15.55:57862) 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded689b640 1 -- 172.21.15.55:0/123047319 learned_addr learned my addr 172.21.15.55:0/123047319 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:12.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded5899640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fded00cd840 0x7fded01510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded609a640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fded00b5830 0x7fded0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded689b640 1 -- 172.21.15.55:0/123047319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fded00cd840 msgr2=0x7fded01510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded689b640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fded00cd840 0x7fded01510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded689b640 1 -- 172.21.15.55:0/123047319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fded00b5830 msgr2=0x7fded0156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded689b640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fded00b5830 0x7fded0156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded689b640 1 -- 172.21.15.55:0/123047319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded0154900 con 0x7fded00d2e10 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fded689b640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 0x7fded00b06e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009e50 tx=0x7fdec8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.270+0000 7fdec6ffd640 1 -- 172.21.15.55:0/123047319 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec803f070 con 0x7fded00d2e10 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.271+0000 7fdec6ffd640 1 -- 172.21.15.55:0/123047319 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec8030b90 con 0x7fded00d2e10 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.271+0000 7fded82fe640 1 -- 172.21.15.55:0/123047319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded0159d50 con 0x7fded00d2e10 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.271+0000 7fdec6ffd640 1 -- 172.21.15.55:0/123047319 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8009410 con 0x7fded00d2e10 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.271+0000 7fded82fe640 1 -- 172.21.15.55:0/123047319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded0152870 con 0x7fded00d2e10 2024-05-20T22:08:12.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.272+0000 7fded82fe640 1 -- 172.21.15.55:0/123047319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded0153cc0 con 0x7fded00d2e10 2024-05-20T22:08:12.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.272+0000 7fdec6ffd640 1 -- 172.21.15.55:0/123047319 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdec8044460 con 0x7fded00d2e10 2024-05-20T22:08:12.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.273+0000 7fdec6ffd640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdeb406be60 0x7fdeb406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.273+0000 7fdec6ffd640 1 -- 172.21.15.55:0/123047319 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdec80ac5c0 con 0x7fded00d2e10 2024-05-20T22:08:12.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.273+0000 7fded609a640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdeb406be60 0x7fdeb406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.273+0000 7fded609a640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdeb406be60 0x7fdeb406e370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fded008b940 tx=0x7fdecc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:12.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.278+0000 7fdec6ffd640 1 -- 172.21.15.55:0/123047319 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdec807a670 con 0x7fded00d2e10 2024-05-20T22:08:12.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.462+0000 7fded82fe640 1 -- 172.21.15.55:0/123047319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fded0152ad0 con 0x7fded00d2e10 2024-05-20T22:08:12.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.463+0000 7fdec6ffd640 1 -- 172.21.15.55:0/123047319 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fdec807e970 con 0x7fded00d2e10 2024-05-20T22:08:12.464 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:12.464 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:12.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 -- 172.21.15.55:0/123047319 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdeb406be60 msgr2=0x7fdeb406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdeb406be60 0x7fdeb406e370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fded008b940 tx=0x7fdecc008040 comp rx=0 tx=0).stop 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 -- 172.21.15.55:0/123047319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 msgr2=0x7fded00b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 0x7fded00b06e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009e50 tx=0x7fdec8004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 -- 172.21.15.55:0/123047319 shutdown_connections 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdeb406be60 0x7fdeb406e370 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fded00d2e10 0x7fded00b06e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fded00cd840 0x7fded01510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 --2- 172.21.15.55:0/123047319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fded00b5830 0x7fded0156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 -- 172.21.15.55:0/123047319 >> 172.21.15.55:0/123047319 conn(0x7fded008bff0 msgr2=0x7fded00707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 -- 172.21.15.55:0/123047319 shutdown_connections 2024-05-20T22:08:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.465+0000 7fded82fe640 1 -- 172.21.15.55:0/123047319 wait complete. 2024-05-20T22:08:12.478 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T22:08:12.478 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:08:12.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.593+0000 7ff3449e3640 1 Processor -- start 2024-05-20T22:08:12.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.593+0000 7ff3449e3640 1 -- start start 2024-05-20T22:08:12.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff3449e3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff3400b7fe0 0x7ff340169f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff3449e3640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3400b83f0 con 0x7ff3400b3b30 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff3449e3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3400d3dd0 con 0x7ff34015f100 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff3449e3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff34006cea0 con 0x7ff3400b7fe0 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33e59c640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff3400b3b30 0x7ff34006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46278/0 (socket says 172.21.15.55:46278) 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33e59c640 1 -- 172.21.15.55:0/4278321797 learned_addr learned my addr 172.21.15.55:0/4278321797 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33ed9d640 1 --2- 172.21.15.55:0/4278321797 >> v2:172.21.15.138:3300/0 conn(0x7ff3400b7fe0 0x7ff340169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2346041367 0 0) 0x7ff3400b83f0 con 0x7ff3400b3b30 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff328003520 con 0x7ff3400b3b30 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 650589284 0 0) 0x7ff328003520 con 0x7ff3400b3b30 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 >> v2:172.21.15.138:3300/0 conn(0x7ff3400b7fe0 msgr2=0x7ff340169f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 --2- 172.21.15.55:0/4278321797 >> v2:172.21.15.138:3300/0 conn(0x7ff3400b7fe0 0x7ff340169f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 >> v1:172.21.15.138:6789/0 conn(0x7ff34015f100 legacy=0x7ff34006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3400d40b0 con 0x7ff3400b3b30 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7ff330001a10 con 0x7ff3400b3b30 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 >> v1:172.21.15.55:6789/0 conn(0x7ff3400b3b30 legacy=0x7ff34006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff328003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff328004800 0x7ff328004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3280051c0 0x7ff32800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff34006cea0 con 0x7ff3280039b0 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff32800c160 con 0x7ff3280051c0 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.595+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff32800c360 con 0x7ff328004800 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.596+0000 7ff33e59c640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff328003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.596+0000 7ff33dd9b640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff328004800 0x7ff328004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.596+0000 7ff33ed9d640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3280051c0 0x7ff32800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.596+0000 7ff33e59c640 1 -- 172.21.15.55:0/4278321797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff328004800 msgr2=0x7ff328004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.596+0000 7ff33e59c640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff328004800 0x7ff328004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.596+0000 7ff33e59c640 1 -- 172.21.15.55:0/4278321797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3280051c0 msgr2=0x7ff32800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.596+0000 7ff33e59c640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3280051c0 0x7ff32800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.596+0000 7ff33e59c640 1 -- 172.21.15.55:0/4278321797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff328003520 con 0x7ff3280039b0 2024-05-20T22:08:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.596+0000 7ff33ed9d640 1 -- 172.21.15.55:0/4278321797 reap_dead start 2024-05-20T22:08:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.597+0000 7ff33e59c640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff328003e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff3400b85c0 tx=0x7ff328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28ea73f3cd6fa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.597+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33000a4a0 con 0x7ff3280039b0 2024-05-20T22:08:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.597+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33002f070 con 0x7ff3280039b0 2024-05-20T22:08:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.597+0000 7ff33d59a640 1 -- 172.21.15.55:0/4278321797 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33002bb50 con 0x7ff3280039b0 2024-05-20T22:08:12.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.597+0000 7ff3449e3640 1 -- 172.21.15.55:0/4278321797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 msgr2=0x7ff328003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.597+0000 7ff3449e3640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff328003e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff3400b85c0 tx=0x7ff328005830 comp rx=0 tx=0).stop 2024-05-20T22:08:12.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.597+0000 7ff3449e3640 1 -- 172.21.15.55:0/4278321797 shutdown_connections 2024-05-20T22:08:12.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.597+0000 7ff3449e3640 1 --2- 172.21.15.55:0/4278321797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff328003e20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.597+0000 7ff3449e3640 1 -- 172.21.15.55:0/4278321797 >> 172.21.15.55:0/4278321797 conn(0x7ff3400d51d0 msgr2=0x7ff3400b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:12.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.598+0000 7ff3449e3640 1 -- 172.21.15.55:0/4278321797 shutdown_connections 2024-05-20T22:08:12.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.598+0000 7ff3449e3640 1 -- 172.21.15.55:0/4278321797 wait complete. 2024-05-20T22:08:12.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.599+0000 7ff3449e3640 1 Processor -- start 2024-05-20T22:08:12.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.600+0000 7ff3449e3640 1 -- start start 2024-05-20T22:08:12.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff3449e3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff340158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff3449e3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff340155ba0 0x7ff340155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff3449e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3401f8d30 0x7ff3401f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff3449e3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff34006b880 con 0x7ff3280039b0 2024-05-20T22:08:12.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff3449e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff34006b5a0 con 0x7ff3401f8d30 2024-05-20T22:08:12.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff3449e3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff34006bb60 con 0x7ff340155ba0 2024-05-20T22:08:12.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff340158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff340158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57882/0 (socket says 172.21.15.55:57882) 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33e59c640 1 -- 172.21.15.55:0/1212383375 learned_addr learned my addr 172.21.15.55:0/1212383375 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33ed9d640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3401f8d30 0x7ff3401f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33dd9b640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff340155ba0 0x7ff340155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33e59c640 1 -- 172.21.15.55:0/1212383375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff340155ba0 msgr2=0x7ff340155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33e59c640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff340155ba0 0x7ff340155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33e59c640 1 -- 172.21.15.55:0/1212383375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3401f8d30 msgr2=0x7ff3401f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33e59c640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3401f8d30 0x7ff3401f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33e59c640 1 -- 172.21.15.55:0/1212383375 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3401f9680 con 0x7ff3280039b0 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33ed9d640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3401f8d30 0x7ff3401f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.601+0000 7ff33dd9b640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff340155ba0 0x7ff340155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.602+0000 7ff33e59c640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff340158640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff33000a880 tx=0x7ff33002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.602+0000 7ff32effd640 1 -- 172.21.15.55:0/1212383375 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33002bb00 con 0x7ff3280039b0 2024-05-20T22:08:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.602+0000 7ff32effd640 1 -- 172.21.15.55:0/1212383375 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33002bca0 con 0x7ff3280039b0 2024-05-20T22:08:12.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.602+0000 7ff3449e3640 1 -- 172.21.15.55:0/1212383375 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3401f6560 con 0x7ff3280039b0 2024-05-20T22:08:12.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.602+0000 7ff32effd640 1 -- 172.21.15.55:0/1212383375 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330034670 con 0x7ff3280039b0 2024-05-20T22:08:12.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.602+0000 7ff3449e3640 1 -- 172.21.15.55:0/1212383375 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3401550c0 con 0x7ff3280039b0 2024-05-20T22:08:12.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.603+0000 7ff3449e3640 1 -- 172.21.15.55:0/1212383375 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff340069090 con 0x7ff3280039b0 2024-05-20T22:08:12.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.604+0000 7ff32effd640 1 -- 172.21.15.55:0/1212383375 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff330034810 con 0x7ff3280039b0 2024-05-20T22:08:12.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.604+0000 7ff32effd640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff32006be60 0x7ff32006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.604+0000 7ff32effd640 1 -- 172.21.15.55:0/1212383375 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff3300ab900 con 0x7ff3280039b0 2024-05-20T22:08:12.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.604+0000 7ff33dd9b640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff32006be60 0x7ff32006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.604+0000 7ff33dd9b640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff32006be60 0x7ff32006e370 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff340161c60 tx=0x7ff334009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:12.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.609+0000 7ff32effd640 1 -- 172.21.15.55:0/1212383375 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff3401f9680 con 0x7ff3280039b0 2024-05-20T22:08:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.788+0000 7ff3449e3640 1 -- 172.21.15.55:0/1212383375 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3400d2ee0 con 0x7ff32006be60 2024-05-20T22:08:12.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.797+0000 7ff32effd640 1 -- 172.21.15.55:0/1212383375 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332562 (secure 0 0 0) 0x7ff3400d2ee0 con 0x7ff32006be60 2024-05-20T22:08:12.801 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:12.803 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:08:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 -- 172.21.15.55:0/1212383375 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff32006be60 msgr2=0x7ff32006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff32006be60 0x7ff32006e370 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff340161c60 tx=0x7ff334009040 comp rx=0 tx=0).stop 2024-05-20T22:08:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 -- 172.21.15.55:0/1212383375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 msgr2=0x7ff340158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff340158640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff33000a880 tx=0x7ff33002d070 comp rx=0 tx=0).stop 2024-05-20T22:08:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 -- 172.21.15.55:0/1212383375 shutdown_connections 2024-05-20T22:08:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff32006be60 0x7ff32006e370 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3401f8d30 0x7ff3401f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff340155ba0 0x7ff340155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 --2- 172.21.15.55:0/1212383375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3280039b0 0x7ff340158640 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 -- 172.21.15.55:0/1212383375 >> 172.21.15.55:0/1212383375 conn(0x7ff3400d51d0 msgr2=0x7ff34015e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 -- 172.21.15.55:0/1212383375 shutdown_connections 2024-05-20T22:08:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.800+0000 7ff3449e3640 1 -- 172.21.15.55:0/1212383375 wait complete. 2024-05-20T22:08:12.813 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-05-20T22:08:12.331063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236020,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738700,"statfs":{"total":773094113280,"available":772852428800,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002443"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581341+0000","last_change":"2024-05-20T22:07:52.664054+0000","last_active":"2024-05-20T22:07:59.581341+0000","last_peered":"2024-05-20T22:07:59.581341+0000","last_clean":"2024-05-20T22:07:59.581341+0000","last_became_active":"2024-05-20T22:07:52.663784+0000","last_became_peered":"2024-05-20T22:07:52.663784+0000","last_unstale":"2024-05-20T22:07:59.581341+0000","last_undegraded":"2024-05-20T22:07:59.581341+0000","last_fullsized":"2024-05-20T22:07:59.581341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:37.756571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092411+0000","last_change":"2024-05-20T22:07:53.678918+0000","last_active":"2024-05-20T22:08:00.092411+0000","last_peered":"2024-05-20T22:08:00.092411+0000","last_clean":"2024-05-20T22:08:00.092411+0000","last_became_active":"2024-05-20T22:07:53.678467+0000","last_became_peered":"2024-05-20T22:07:53.678467+0000","last_unstale":"2024-05-20T22:08:00.092411+0000","last_undegraded":"2024-05-20T22:08:00.092411+0000","last_fullsized":"2024-05-20T22:08:00.092411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092012+0000","last_change":"2024-05-20T22:07:57.732174+0000","last_active":"2024-05-20T22:08:00.092012+0000","last_peered":"2024-05-20T22:08:00.092012+0000","last_clean":"2024-05-20T22:08:00.092012+0000","last_became_active":"2024-05-20T22:07:57.731124+0000","last_became_peered":"2024-05-20T22:07:57.731124+0000","last_unstale":"2024-05-20T22:08:00.092012+0000","last_undegraded":"2024-05-20T22:08:00.092012+0000","last_fullsized":"2024-05-20T22:08:00.092012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:11.233995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092785+0000","last_change":"2024-05-20T22:07:52.664457+0000","last_active":"2024-05-20T22:08:00.092785+0000","last_peered":"2024-05-20T22:08:00.092785+0000","last_clean":"2024-05-20T22:08:00.092785+0000","last_became_active":"2024-05-20T22:07:52.664121+0000","last_became_peered":"2024-05-20T22:07:52.664121+0000","last_unstale":"2024-05-20T22:08:00.092785+0000","last_undegraded":"2024-05-20T22:08:00.092785+0000","last_fullsized":"2024-05-20T22:08:00.092785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581131+0000","last_change":"2024-05-20T22:07:53.669570+0000","last_active":"2024-05-20T22:07:59.581131+0000","last_peered":"2024-05-20T22:07:59.581131+0000","last_clean":"2024-05-20T22:07:59.581131+0000","last_became_active":"2024-05-20T22:07:53.668913+0000","last_became_peered":"2024-05-20T22:07:53.668913+0000","last_unstale":"2024-05-20T22:07:59.581131+0000","last_undegraded":"2024-05-20T22:07:59.581131+0000","last_fullsized":"2024-05-20T22:07:59.581131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092313+0000","last_change":"2024-05-20T22:07:57.728254+0000","last_active":"2024-05-20T22:08:00.092313+0000","last_peered":"2024-05-20T22:08:00.092313+0000","last_clean":"2024-05-20T22:08:00.092313+0000","last_became_active":"2024-05-20T22:07:57.727787+0000","last_became_peered":"2024-05-20T22:07:57.727787+0000","last_unstale":"2024-05-20T22:08:00.092313+0000","last_undegraded":"2024-05-20T22:08:00.092313+0000","last_fullsized":"2024-05-20T22:08:00.092313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:43.588870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706646+0000","last_change":"2024-05-20T22:07:52.657626+0000","last_active":"2024-05-20T22:07:58.706646+0000","last_peered":"2024-05-20T22:07:58.706646+0000","last_clean":"2024-05-20T22:07:58.706646+0000","last_became_active":"2024-05-20T22:07:52.657298+0000","last_became_peered":"2024-05-20T22:07:52.657298+0000","last_unstale":"2024-05-20T22:07:58.706646+0000","last_undegraded":"2024-05-20T22:07:58.706646+0000","last_fullsized":"2024-05-20T22:07:58.706646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:41.518373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092705+0000","last_change":"2024-05-20T22:07:53.669706+0000","last_active":"2024-05-20T22:08:00.092705+0000","last_peered":"2024-05-20T22:08:00.092705+0000","last_clean":"2024-05-20T22:08:00.092705+0000","last_became_active":"2024-05-20T22:07:53.669332+0000","last_became_peered":"2024-05-20T22:07:53.669332+0000","last_unstale":"2024-05-20T22:08:00.092705+0000","last_undegraded":"2024-05-20T22:08:00.092705+0000","last_fullsized":"2024-05-20T22:08:00.092705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092885+0000","last_change":"2024-05-20T22:07:57.734186+0000","last_active":"2024-05-20T22:08:00.092885+0000","last_peered":"2024-05-20T22:08:00.092885+0000","last_clean":"2024-05-20T22:08:00.092885+0000","last_became_active":"2024-05-20T22:07:57.733618+0000","last_became_peered":"2024-05-20T22:07:57.733618+0000","last_unstale":"2024-05-20T22:08:00.092885+0000","last_undegraded":"2024-05-20T22:08:00.092885+0000","last_fullsized":"2024-05-20T22:08:00.092885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:46.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706028+0000","last_change":"2024-05-20T22:07:52.660033+0000","last_active":"2024-05-20T22:07:58.706028+0000","last_peered":"2024-05-20T22:07:58.706028+0000","last_clean":"2024-05-20T22:07:58.706028+0000","last_became_active":"2024-05-20T22:07:52.659733+0000","last_became_peered":"2024-05-20T22:07:52.659733+0000","last_unstale":"2024-05-20T22:07:58.706028+0000","last_undegraded":"2024-05-20T22:07:58.706028+0000","last_fullsized":"2024-05-20T22:07:58.706028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:41.160812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093319+0000","last_change":"2024-05-20T22:07:53.669432+0000","last_active":"2024-05-20T22:08:00.093319+0000","last_peered":"2024-05-20T22:08:00.093319+0000","last_clean":"2024-05-20T22:08:00.093319+0000","last_became_active":"2024-05-20T22:07:53.668235+0000","last_became_peered":"2024-05-20T22:07:53.668235+0000","last_unstale":"2024-05-20T22:08:00.093319+0000","last_undegraded":"2024-05-20T22:08:00.093319+0000","last_fullsized":"2024-05-20T22:08:00.093319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:42.044571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092545+0000","last_change":"2024-05-20T22:07:57.734065+0000","last_active":"2024-05-20T22:08:00.092545+0000","last_peered":"2024-05-20T22:08:00.092545+0000","last_clean":"2024-05-20T22:08:00.092545+0000","last_became_active":"2024-05-20T22:07:57.733537+0000","last_became_peered":"2024-05-20T22:07:57.733537+0000","last_unstale":"2024-05-20T22:08:00.092545+0000","last_undegraded":"2024-05-20T22:08:00.092545+0000","last_fullsized":"2024-05-20T22:08:00.092545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:16.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705724+0000","last_change":"2024-05-20T22:07:53.675279+0000","last_active":"2024-05-20T22:07:58.705724+0000","last_peered":"2024-05-20T22:07:58.705724+0000","last_clean":"2024-05-20T22:07:58.705724+0000","last_became_active":"2024-05-20T22:07:53.674924+0000","last_became_peered":"2024-05-20T22:07:53.674924+0000","last_unstale":"2024-05-20T22:07:58.705724+0000","last_undegraded":"2024-05-20T22:07:58.705724+0000","last_fullsized":"2024-05-20T22:07:58.705724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705726+0000","last_change":"2024-05-20T22:07:52.655127+0000","last_active":"2024-05-20T22:07:58.705726+0000","last_peered":"2024-05-20T22:07:58.705726+0000","last_clean":"2024-05-20T22:07:58.705726+0000","last_became_active":"2024-05-20T22:07:52.654807+0000","last_became_peered":"2024-05-20T22:07:52.654807+0000","last_unstale":"2024-05-20T22:07:58.705726+0000","last_undegraded":"2024-05-20T22:07:58.705726+0000","last_fullsized":"2024-05-20T22:07:58.705726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093607+0000","last_change":"2024-05-20T22:07:57.732155+0000","last_active":"2024-05-20T22:08:00.093607+0000","last_peered":"2024-05-20T22:08:00.093607+0000","last_clean":"2024-05-20T22:08:00.093607+0000","last_became_active":"2024-05-20T22:07:57.731370+0000","last_became_peered":"2024-05-20T22:07:57.731370+0000","last_unstale":"2024-05-20T22:08:00.093607+0000","last_undegraded":"2024-05-20T22:08:00.093607+0000","last_fullsized":"2024-05-20T22:08:00.093607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:30.698775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092498+0000","last_change":"2024-05-20T22:07:53.677185+0000","last_active":"2024-05-20T22:08:00.092498+0000","last_peered":"2024-05-20T22:08:00.092498+0000","last_clean":"2024-05-20T22:08:00.092498+0000","last_became_active":"2024-05-20T22:07:53.676661+0000","last_became_peered":"2024-05-20T22:07:53.676661+0000","last_unstale":"2024-05-20T22:08:00.092498+0000","last_undegraded":"2024-05-20T22:08:00.092498+0000","last_fullsized":"2024-05-20T22:08:00.092498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093251+0000","last_change":"2024-05-20T22:07:52.668583+0000","last_active":"2024-05-20T22:08:00.093251+0000","last_peered":"2024-05-20T22:08:00.093251+0000","last_clean":"2024-05-20T22:08:00.093251+0000","last_became_active":"2024-05-20T22:07:52.667850+0000","last_became_peered":"2024-05-20T22:07:52.667850+0000","last_unstale":"2024-05-20T22:08:00.093251+0000","last_undegraded":"2024-05-20T22:08:00.093251+0000","last_fullsized":"2024-05-20T22:08:00.093251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092843+0000","last_change":"2024-05-20T22:07:57.730407+0000","last_active":"2024-05-20T22:08:00.092843+0000","last_peered":"2024-05-20T22:08:00.092843+0000","last_clean":"2024-05-20T22:08:00.092843+0000","last_became_active":"2024-05-20T22:07:57.729918+0000","last_became_peered":"2024-05-20T22:07:57.729918+0000","last_unstale":"2024-05-20T22:08:00.092843+0000","last_undegraded":"2024-05-20T22:08:00.092843+0000","last_fullsized":"2024-05-20T22:08:00.092843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:44.596866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092522+0000","last_change":"2024-05-20T22:07:53.675389+0000","last_active":"2024-05-20T22:08:00.092522+0000","last_peered":"2024-05-20T22:08:00.092522+0000","last_clean":"2024-05-20T22:08:00.092522+0000","last_became_active":"2024-05-20T22:07:53.675027+0000","last_became_peered":"2024-05-20T22:07:53.675027+0000","last_unstale":"2024-05-20T22:08:00.092522+0000","last_undegraded":"2024-05-20T22:08:00.092522+0000","last_fullsized":"2024-05-20T22:08:00.092522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092234+0000","last_change":"2024-05-20T22:07:52.662236+0000","last_active":"2024-05-20T22:08:00.092234+0000","last_peered":"2024-05-20T22:08:00.092234+0000","last_clean":"2024-05-20T22:08:00.092234+0000","last_became_active":"2024-05-20T22:07:52.660352+0000","last_became_peered":"2024-05-20T22:07:52.660352+0000","last_unstale":"2024-05-20T22:08:00.092234+0000","last_undegraded":"2024-05-20T22:08:00.092234+0000","last_fullsized":"2024-05-20T22:08:00.092234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:41.518373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093269+0000","last_change":"2024-05-20T22:07:57.736022+0000","last_active":"2024-05-20T22:08:00.093269+0000","last_peered":"2024-05-20T22:08:00.093269+0000","last_clean":"2024-05-20T22:08:00.093269+0000","last_became_active":"2024-05-20T22:07:57.735570+0000","last_became_peered":"2024-05-20T22:07:57.735570+0000","last_unstale":"2024-05-20T22:08:00.093269+0000","last_undegraded":"2024-05-20T22:08:00.093269+0000","last_fullsized":"2024-05-20T22:08:00.093269+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:53.866913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092952+0000","last_change":"2024-05-20T22:07:53.662746+0000","last_active":"2024-05-20T22:08:00.092952+0000","last_peered":"2024-05-20T22:08:00.092952+0000","last_clean":"2024-05-20T22:08:00.092952+0000","last_became_active":"2024-05-20T22:07:53.662458+0000","last_became_peered":"2024-05-20T22:07:53.662458+0000","last_unstale":"2024-05-20T22:08:00.092952+0000","last_undegraded":"2024-05-20T22:08:00.092952+0000","last_fullsized":"2024-05-20T22:08:00.092952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:18.996690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705082+0000","last_change":"2024-05-20T22:07:52.656158+0000","last_active":"2024-05-20T22:07:58.705082+0000","last_peered":"2024-05-20T22:07:58.705082+0000","last_clean":"2024-05-20T22:07:58.705082+0000","last_became_active":"2024-05-20T22:07:52.655448+0000","last_became_peered":"2024-05-20T22:07:52.655448+0000","last_unstale":"2024-05-20T22:07:58.705082+0000","last_undegraded":"2024-05-20T22:07:58.705082+0000","last_fullsized":"2024-05-20T22:07:58.705082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093755+0000","last_change":"2024-05-20T22:07:57.734374+0000","last_active":"2024-05-20T22:08:00.093755+0000","last_peered":"2024-05-20T22:08:00.093755+0000","last_clean":"2024-05-20T22:08:00.093755+0000","last_became_active":"2024-05-20T22:07:57.733737+0000","last_became_peered":"2024-05-20T22:07:57.733737+0000","last_unstale":"2024-05-20T22:08:00.093755+0000","last_undegraded":"2024-05-20T22:08:00.093755+0000","last_fullsized":"2024-05-20T22:08:00.093755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581291+0000","last_change":"2024-05-20T22:07:53.670700+0000","last_active":"2024-05-20T22:07:59.581291+0000","last_peered":"2024-05-20T22:07:59.581291+0000","last_clean":"2024-05-20T22:07:59.581291+0000","last_became_active":"2024-05-20T22:07:53.670361+0000","last_became_peered":"2024-05-20T22:07:53.670361+0000","last_unstale":"2024-05-20T22:07:59.581291+0000","last_undegraded":"2024-05-20T22:07:59.581291+0000","last_fullsized":"2024-05-20T22:07:59.581291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706857+0000","last_change":"2024-05-20T22:07:52.658539+0000","last_active":"2024-05-20T22:07:58.706857+0000","last_peered":"2024-05-20T22:07:58.706857+0000","last_clean":"2024-05-20T22:07:58.706857+0000","last_became_active":"2024-05-20T22:07:52.658232+0000","last_became_peered":"2024-05-20T22:07:52.658232+0000","last_unstale":"2024-05-20T22:07:58.706857+0000","last_undegraded":"2024-05-20T22:07:58.706857+0000","last_fullsized":"2024-05-20T22:07:58.706857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092305+0000","last_change":"2024-05-20T22:07:57.726798+0000","last_active":"2024-05-20T22:08:00.092305+0000","last_peered":"2024-05-20T22:08:00.092305+0000","last_clean":"2024-05-20T22:08:00.092305+0000","last_became_active":"2024-05-20T22:07:57.726321+0000","last_became_peered":"2024-05-20T22:07:57.726321+0000","last_unstale":"2024-05-20T22:08:00.092305+0000","last_undegraded":"2024-05-20T22:08:00.092305+0000","last_fullsized":"2024-05-20T22:08:00.092305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:00:00.091936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093912+0000","last_change":"2024-05-20T22:07:53.673830+0000","last_active":"2024-05-20T22:08:00.093912+0000","last_peered":"2024-05-20T22:08:00.093912+0000","last_clean":"2024-05-20T22:08:00.093912+0000","last_became_active":"2024-05-20T22:07:53.673509+0000","last_became_peered":"2024-05-20T22:07:53.673509+0000","last_unstale":"2024-05-20T22:08:00.093912+0000","last_undegraded":"2024-05-20T22:08:00.093912+0000","last_fullsized":"2024-05-20T22:08:00.093912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092565+0000","last_change":"2024-05-20T22:07:52.661954+0000","last_active":"2024-05-20T22:08:00.092565+0000","last_peered":"2024-05-20T22:08:00.092565+0000","last_clean":"2024-05-20T22:08:00.092565+0000","last_became_active":"2024-05-20T22:07:52.661304+0000","last_became_peered":"2024-05-20T22:07:52.661304+0000","last_unstale":"2024-05-20T22:08:00.092565+0000","last_undegraded":"2024-05-20T22:08:00.092565+0000","last_fullsized":"2024-05-20T22:08:00.092565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.091968+0000","last_change":"2024-05-20T22:07:57.731395+0000","last_active":"2024-05-20T22:08:00.091968+0000","last_peered":"2024-05-20T22:08:00.091968+0000","last_clean":"2024-05-20T22:08:00.091968+0000","last_became_active":"2024-05-20T22:07:57.730839+0000","last_became_peered":"2024-05-20T22:07:57.730839+0000","last_unstale":"2024-05-20T22:08:00.091968+0000","last_undegraded":"2024-05-20T22:08:00.091968+0000","last_fullsized":"2024-05-20T22:08:00.091968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:44:34.494247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092791+0000","last_change":"2024-05-20T22:07:53.673439+0000","last_active":"2024-05-20T22:08:00.092791+0000","last_peered":"2024-05-20T22:08:00.092791+0000","last_clean":"2024-05-20T22:08:00.092791+0000","last_became_active":"2024-05-20T22:07:53.672746+0000","last_became_peered":"2024-05-20T22:07:53.672746+0000","last_unstale":"2024-05-20T22:08:00.092791+0000","last_undegraded":"2024-05-20T22:08:00.092791+0000","last_fullsized":"2024-05-20T22:08:00.092791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706775+0000","last_change":"2024-05-20T22:07:52.662817+0000","last_active":"2024-05-20T22:07:58.706775+0000","last_peered":"2024-05-20T22:07:58.706775+0000","last_clean":"2024-05-20T22:07:58.706775+0000","last_became_active":"2024-05-20T22:07:52.662463+0000","last_became_peered":"2024-05-20T22:07:52.662463+0000","last_unstale":"2024-05-20T22:07:58.706775+0000","last_undegraded":"2024-05-20T22:07:58.706775+0000","last_fullsized":"2024-05-20T22:07:58.706775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:07.396536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580916+0000","last_change":"2024-05-20T22:07:57.730858+0000","last_active":"2024-05-20T22:07:59.580916+0000","last_peered":"2024-05-20T22:07:59.580916+0000","last_clean":"2024-05-20T22:07:59.580916+0000","last_became_active":"2024-05-20T22:07:57.730370+0000","last_became_peered":"2024-05-20T22:07:57.730370+0000","last_unstale":"2024-05-20T22:07:59.580916+0000","last_undegraded":"2024-05-20T22:07:59.580916+0000","last_fullsized":"2024-05-20T22:07:59.580916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093484+0000","last_change":"2024-05-20T22:07:53.673073+0000","last_active":"2024-05-20T22:08:00.093484+0000","last_peered":"2024-05-20T22:08:00.093484+0000","last_clean":"2024-05-20T22:08:00.093484+0000","last_became_active":"2024-05-20T22:07:53.672744+0000","last_became_peered":"2024-05-20T22:07:53.672744+0000","last_unstale":"2024-05-20T22:08:00.093484+0000","last_undegraded":"2024-05-20T22:08:00.093484+0000","last_fullsized":"2024-05-20T22:08:00.093484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:49.823373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093340+0000","last_change":"2024-05-20T22:07:52.668045+0000","last_active":"2024-05-20T22:08:00.093340+0000","last_peered":"2024-05-20T22:08:00.093340+0000","last_clean":"2024-05-20T22:08:00.093340+0000","last_became_active":"2024-05-20T22:07:52.666855+0000","last_became_peered":"2024-05-20T22:07:52.666855+0000","last_unstale":"2024-05-20T22:08:00.093340+0000","last_undegraded":"2024-05-20T22:08:00.093340+0000","last_fullsized":"2024-05-20T22:08:00.093340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092837+0000","last_change":"2024-05-20T22:07:57.740710+0000","last_active":"2024-05-20T22:08:00.092837+0000","last_peered":"2024-05-20T22:08:00.092837+0000","last_clean":"2024-05-20T22:08:00.092837+0000","last_became_active":"2024-05-20T22:07:57.737102+0000","last_became_peered":"2024-05-20T22:07:57.737102+0000","last_unstale":"2024-05-20T22:08:00.092837+0000","last_undegraded":"2024-05-20T22:08:00.092837+0000","last_fullsized":"2024-05-20T22:08:00.092837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092905+0000","last_change":"2024-05-20T22:07:53.667120+0000","last_active":"2024-05-20T22:08:00.092905+0000","last_peered":"2024-05-20T22:08:00.092905+0000","last_clean":"2024-05-20T22:08:00.092905+0000","last_became_active":"2024-05-20T22:07:53.666809+0000","last_became_peered":"2024-05-20T22:07:53.666809+0000","last_unstale":"2024-05-20T22:08:00.092905+0000","last_undegraded":"2024-05-20T22:08:00.092905+0000","last_fullsized":"2024-05-20T22:08:00.092905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:08.402858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093723+0000","last_change":"2024-05-20T22:07:52.664415+0000","last_active":"2024-05-20T22:08:00.093723+0000","last_peered":"2024-05-20T22:08:00.093723+0000","last_clean":"2024-05-20T22:08:00.093723+0000","last_became_active":"2024-05-20T22:07:52.664085+0000","last_became_peered":"2024-05-20T22:07:52.664085+0000","last_unstale":"2024-05-20T22:08:00.093723+0000","last_undegraded":"2024-05-20T22:08:00.093723+0000","last_fullsized":"2024-05-20T22:08:00.093723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092731+0000","last_change":"2024-05-20T22:07:57.736468+0000","last_active":"2024-05-20T22:08:00.092731+0000","last_peered":"2024-05-20T22:08:00.092731+0000","last_clean":"2024-05-20T22:08:00.092731+0000","last_became_active":"2024-05-20T22:07:57.736003+0000","last_became_peered":"2024-05-20T22:07:57.736003+0000","last_unstale":"2024-05-20T22:08:00.092731+0000","last_undegraded":"2024-05-20T22:08:00.092731+0000","last_fullsized":"2024-05-20T22:08:00.092731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.091885+0000","last_change":"2024-05-20T22:07:53.674097+0000","last_active":"2024-05-20T22:08:00.091885+0000","last_peered":"2024-05-20T22:08:00.091885+0000","last_clean":"2024-05-20T22:08:00.091885+0000","last_became_active":"2024-05-20T22:07:53.673766+0000","last_became_peered":"2024-05-20T22:07:53.673766+0000","last_unstale":"2024-05-20T22:08:00.091885+0000","last_undegraded":"2024-05-20T22:08:00.091885+0000","last_fullsized":"2024-05-20T22:08:00.091885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706155+0000","last_change":"2024-05-20T22:07:52.662684+0000","last_active":"2024-05-20T22:07:58.706155+0000","last_peered":"2024-05-20T22:07:58.706155+0000","last_clean":"2024-05-20T22:07:58.706155+0000","last_became_active":"2024-05-20T22:07:52.662220+0000","last_became_peered":"2024-05-20T22:07:52.662220+0000","last_unstale":"2024-05-20T22:07:58.706155+0000","last_undegraded":"2024-05-20T22:07:58.706155+0000","last_fullsized":"2024-05-20T22:07:58.706155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093363+0000","last_change":"2024-05-20T22:07:57.740613+0000","last_active":"2024-05-20T22:08:00.093363+0000","last_peered":"2024-05-20T22:08:00.093363+0000","last_clean":"2024-05-20T22:08:00.093363+0000","last_became_active":"2024-05-20T22:07:57.740165+0000","last_became_peered":"2024-05-20T22:07:57.740165+0000","last_unstale":"2024-05-20T22:08:00.093363+0000","last_undegraded":"2024-05-20T22:08:00.093363+0000","last_fullsized":"2024-05-20T22:08:00.093363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:03:25.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580372+0000","last_change":"2024-05-20T22:07:53.656427+0000","last_active":"2024-05-20T22:07:59.580372+0000","last_peered":"2024-05-20T22:07:59.580372+0000","last_clean":"2024-05-20T22:07:59.580372+0000","last_became_active":"2024-05-20T22:07:53.656081+0000","last_became_peered":"2024-05-20T22:07:53.656081+0000","last_unstale":"2024-05-20T22:07:59.580372+0000","last_undegraded":"2024-05-20T22:07:59.580372+0000","last_fullsized":"2024-05-20T22:07:59.580372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:58:02.316451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705143+0000","last_change":"2024-05-20T22:07:52.656503+0000","last_active":"2024-05-20T22:07:58.705143+0000","last_peered":"2024-05-20T22:07:58.705143+0000","last_clean":"2024-05-20T22:07:58.705143+0000","last_became_active":"2024-05-20T22:07:52.654665+0000","last_became_peered":"2024-05-20T22:07:52.654665+0000","last_unstale":"2024-05-20T22:07:58.705143+0000","last_undegraded":"2024-05-20T22:07:58.705143+0000","last_fullsized":"2024-05-20T22:07:58.705143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581267+0000","last_change":"2024-05-20T22:07:57.731186+0000","last_active":"2024-05-20T22:07:59.581267+0000","last_peered":"2024-05-20T22:07:59.581267+0000","last_clean":"2024-05-20T22:07:59.581267+0000","last_became_active":"2024-05-20T22:07:57.730239+0000","last_became_peered":"2024-05-20T22:07:57.730239+0000","last_unstale":"2024-05-20T22:07:59.581267+0000","last_undegraded":"2024-05-20T22:07:59.581267+0000","last_fullsized":"2024-05-20T22:07:59.581267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:29.190558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093069+0000","last_change":"2024-05-20T22:07:53.667981+0000","last_active":"2024-05-20T22:08:00.093069+0000","last_peered":"2024-05-20T22:08:00.093069+0000","last_clean":"2024-05-20T22:08:00.093069+0000","last_became_active":"2024-05-20T22:07:53.667320+0000","last_became_peered":"2024-05-20T22:07:53.667320+0000","last_unstale":"2024-05-20T22:08:00.093069+0000","last_undegraded":"2024-05-20T22:08:00.093069+0000","last_fullsized":"2024-05-20T22:08:00.093069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093094+0000","last_change":"2024-05-20T22:07:52.657838+0000","last_active":"2024-05-20T22:08:00.093094+0000","last_peered":"2024-05-20T22:08:00.093094+0000","last_clean":"2024-05-20T22:08:00.093094+0000","last_became_active":"2024-05-20T22:07:52.657530+0000","last_became_peered":"2024-05-20T22:07:52.657530+0000","last_unstale":"2024-05-20T22:08:00.093094+0000","last_undegraded":"2024-05-20T22:08:00.093094+0000","last_fullsized":"2024-05-20T22:08:00.093094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706234+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:07:58.706234+0000","last_peered":"2024-05-20T22:07:58.706234+0000","last_clean":"2024-05-20T22:07:58.706234+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:07:58.706234+0000","last_undegraded":"2024-05-20T22:07:58.706234+0000","last_fullsized":"2024-05-20T22:07:58.706234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706485+0000","last_change":"2024-05-20T22:07:57.738452+0000","last_active":"2024-05-20T22:07:58.706485+0000","last_peered":"2024-05-20T22:07:58.706485+0000","last_clean":"2024-05-20T22:07:58.706485+0000","last_became_active":"2024-05-20T22:07:57.737967+0000","last_became_peered":"2024-05-20T22:07:57.737967+0000","last_unstale":"2024-05-20T22:07:58.706485+0000","last_undegraded":"2024-05-20T22:07:58.706485+0000","last_fullsized":"2024-05-20T22:07:58.706485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706425+0000","last_change":"2024-05-20T22:07:53.677833+0000","last_active":"2024-05-20T22:07:58.706425+0000","last_peered":"2024-05-20T22:07:58.706425+0000","last_clean":"2024-05-20T22:07:58.706425+0000","last_became_active":"2024-05-20T22:07:53.677513+0000","last_became_peered":"2024-05-20T22:07:53.677513+0000","last_unstale":"2024-05-20T22:07:58.706425+0000","last_undegraded":"2024-05-20T22:07:58.706425+0000","last_fullsized":"2024-05-20T22:07:58.706425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706468+0000","last_change":"2024-05-20T22:07:52.659290+0000","last_active":"2024-05-20T22:07:58.706468+0000","last_peered":"2024-05-20T22:07:58.706468+0000","last_clean":"2024-05-20T22:07:58.706468+0000","last_became_active":"2024-05-20T22:07:52.658939+0000","last_became_peered":"2024-05-20T22:07:52.658939+0000","last_unstale":"2024-05-20T22:07:58.706468+0000","last_undegraded":"2024-05-20T22:07:58.706468+0000","last_fullsized":"2024-05-20T22:07:58.706468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:30.832431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705627+0000","last_change":"2024-05-20T22:07:57.732095+0000","last_active":"2024-05-20T22:07:58.705627+0000","last_peered":"2024-05-20T22:07:58.705627+0000","last_clean":"2024-05-20T22:07:58.705627+0000","last_became_active":"2024-05-20T22:07:57.731276+0000","last_became_peered":"2024-05-20T22:07:57.731276+0000","last_unstale":"2024-05-20T22:07:58.705627+0000","last_undegraded":"2024-05-20T22:07:58.705627+0000","last_fullsized":"2024-05-20T22:07:58.705627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:30.698775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092825+0000","last_change":"2024-05-20T22:07:53.669593+0000","last_active":"2024-05-20T22:08:00.092825+0000","last_peered":"2024-05-20T22:08:00.092825+0000","last_clean":"2024-05-20T22:08:00.092825+0000","last_became_active":"2024-05-20T22:07:53.668982+0000","last_became_peered":"2024-05-20T22:07:53.668982+0000","last_unstale":"2024-05-20T22:08:00.092825+0000","last_undegraded":"2024-05-20T22:08:00.092825+0000","last_fullsized":"2024-05-20T22:08:00.092825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:26.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092921+0000","last_change":"2024-05-20T22:07:52.666183+0000","last_active":"2024-05-20T22:08:00.092921+0000","last_peered":"2024-05-20T22:08:00.092921+0000","last_clean":"2024-05-20T22:08:00.092921+0000","last_became_active":"2024-05-20T22:07:52.662670+0000","last_became_peered":"2024-05-20T22:07:52.662670+0000","last_unstale":"2024-05-20T22:08:00.092921+0000","last_undegraded":"2024-05-20T22:08:00.092921+0000","last_fullsized":"2024-05-20T22:08:00.092921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581046+0000","last_change":"2024-05-20T22:07:57.734443+0000","last_active":"2024-05-20T22:07:59.581046+0000","last_peered":"2024-05-20T22:07:59.581046+0000","last_clean":"2024-05-20T22:07:59.581046+0000","last_became_active":"2024-05-20T22:07:57.734021+0000","last_became_peered":"2024-05-20T22:07:57.734021+0000","last_unstale":"2024-05-20T22:07:59.581046+0000","last_undegraded":"2024-05-20T22:07:59.581046+0000","last_fullsized":"2024-05-20T22:07:59.581046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:30.960467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707194+0000","last_change":"2024-05-20T22:07:53.666815+0000","last_active":"2024-05-20T22:07:58.707194+0000","last_peered":"2024-05-20T22:07:58.707194+0000","last_clean":"2024-05-20T22:07:58.707194+0000","last_became_active":"2024-05-20T22:07:53.666480+0000","last_became_peered":"2024-05-20T22:07:53.666480+0000","last_unstale":"2024-05-20T22:07:58.707194+0000","last_undegraded":"2024-05-20T22:07:58.707194+0000","last_fullsized":"2024-05-20T22:07:58.707194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:49.823373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706498+0000","last_change":"2024-05-20T22:07:52.661745+0000","last_active":"2024-05-20T22:07:58.706498+0000","last_peered":"2024-05-20T22:07:58.706498+0000","last_clean":"2024-05-20T22:07:58.706498+0000","last_became_active":"2024-05-20T22:07:52.661304+0000","last_became_peered":"2024-05-20T22:07:52.661304+0000","last_unstale":"2024-05-20T22:07:58.706498+0000","last_undegraded":"2024-05-20T22:07:58.706498+0000","last_fullsized":"2024-05-20T22:07:58.706498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092249+0000","last_change":"2024-05-20T22:07:57.722732+0000","last_active":"2024-05-20T22:08:00.092249+0000","last_peered":"2024-05-20T22:08:00.092249+0000","last_clean":"2024-05-20T22:08:00.092249+0000","last_became_active":"2024-05-20T22:07:57.722287+0000","last_became_peered":"2024-05-20T22:07:57.722287+0000","last_unstale":"2024-05-20T22:08:00.092249+0000","last_undegraded":"2024-05-20T22:08:00.092249+0000","last_fullsized":"2024-05-20T22:08:00.092249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:48.169892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706818+0000","last_change":"2024-05-20T22:07:53.668146+0000","last_active":"2024-05-20T22:07:58.706818+0000","last_peered":"2024-05-20T22:07:58.706818+0000","last_clean":"2024-05-20T22:07:58.706818+0000","last_became_active":"2024-05-20T22:07:53.666969+0000","last_became_peered":"2024-05-20T22:07:53.666969+0000","last_unstale":"2024-05-20T22:07:58.706818+0000","last_undegraded":"2024-05-20T22:07:58.706818+0000","last_fullsized":"2024-05-20T22:07:58.706818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093917+0000","last_change":"2024-05-20T22:07:52.651502+0000","last_active":"2024-05-20T22:08:00.093917+0000","last_peered":"2024-05-20T22:08:00.093917+0000","last_clean":"2024-05-20T22:08:00.093917+0000","last_became_active":"2024-05-20T22:07:52.651166+0000","last_became_peered":"2024-05-20T22:07:52.651166+0000","last_unstale":"2024-05-20T22:08:00.093917+0000","last_undegraded":"2024-05-20T22:08:00.093917+0000","last_fullsized":"2024-05-20T22:08:00.093917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:41:50.914741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092214+0000","last_change":"2024-05-20T22:07:57.736957+0000","last_active":"2024-05-20T22:08:00.092214+0000","last_peered":"2024-05-20T22:08:00.092214+0000","last_clean":"2024-05-20T22:08:00.092214+0000","last_became_active":"2024-05-20T22:07:57.736512+0000","last_became_peered":"2024-05-20T22:07:57.736512+0000","last_unstale":"2024-05-20T22:08:00.092214+0000","last_undegraded":"2024-05-20T22:08:00.092214+0000","last_fullsized":"2024-05-20T22:08:00.092214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:46.233608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705356+0000","last_change":"2024-05-20T22:07:53.655287+0000","last_active":"2024-05-20T22:07:58.705356+0000","last_peered":"2024-05-20T22:07:58.705356+0000","last_clean":"2024-05-20T22:07:58.705356+0000","last_became_active":"2024-05-20T22:07:53.653689+0000","last_became_peered":"2024-05-20T22:07:53.653689+0000","last_unstale":"2024-05-20T22:07:58.705356+0000","last_undegraded":"2024-05-20T22:07:58.705356+0000","last_fullsized":"2024-05-20T22:07:58.705356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:26.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705945+0000","last_change":"2024-05-20T22:07:52.653862+0000","last_active":"2024-05-20T22:07:58.705945+0000","last_peered":"2024-05-20T22:07:58.705945+0000","last_clean":"2024-05-20T22:07:58.705945+0000","last_became_active":"2024-05-20T22:07:52.653527+0000","last_became_peered":"2024-05-20T22:07:52.653527+0000","last_unstale":"2024-05-20T22:07:58.705945+0000","last_undegraded":"2024-05-20T22:07:58.705945+0000","last_fullsized":"2024-05-20T22:07:58.705945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581093+0000","last_change":"2024-05-20T22:07:57.737164+0000","last_active":"2024-05-20T22:07:59.581093+0000","last_peered":"2024-05-20T22:07:59.581093+0000","last_clean":"2024-05-20T22:07:59.581093+0000","last_became_active":"2024-05-20T22:07:57.736733+0000","last_became_peered":"2024-05-20T22:07:57.736733+0000","last_unstale":"2024-05-20T22:07:59.581093+0000","last_undegraded":"2024-05-20T22:07:59.581093+0000","last_fullsized":"2024-05-20T22:07:59.581093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:53.866913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705408+0000","last_change":"2024-05-20T22:07:53.680648+0000","last_active":"2024-05-20T22:07:58.705408+0000","last_peered":"2024-05-20T22:07:58.705408+0000","last_clean":"2024-05-20T22:07:58.705408+0000","last_became_active":"2024-05-20T22:07:53.680316+0000","last_became_peered":"2024-05-20T22:07:53.680316+0000","last_unstale":"2024-05-20T22:07:58.705408+0000","last_undegraded":"2024-05-20T22:07:58.705408+0000","last_fullsized":"2024-05-20T22:07:58.705408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706034+0000","last_change":"2024-05-20T22:07:52.660545+0000","last_active":"2024-05-20T22:07:58.706034+0000","last_peered":"2024-05-20T22:07:58.706034+0000","last_clean":"2024-05-20T22:07:58.706034+0000","last_became_active":"2024-05-20T22:07:52.660191+0000","last_became_peered":"2024-05-20T22:07:52.660191+0000","last_unstale":"2024-05-20T22:07:58.706034+0000","last_undegraded":"2024-05-20T22:07:58.706034+0000","last_fullsized":"2024-05-20T22:07:58.706034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093896+0000","last_change":"2024-05-20T22:07:57.730561+0000","last_active":"2024-05-20T22:08:00.093896+0000","last_peered":"2024-05-20T22:08:00.093896+0000","last_clean":"2024-05-20T22:08:00.093896+0000","last_became_active":"2024-05-20T22:07:57.728162+0000","last_became_peered":"2024-05-20T22:07:57.728162+0000","last_unstale":"2024-05-20T22:08:00.093896+0000","last_undegraded":"2024-05-20T22:08:00.093896+0000","last_fullsized":"2024-05-20T22:08:00.093896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:18.032764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705624+0000","last_change":"2024-05-20T22:07:53.654238+0000","last_active":"2024-05-20T22:07:58.705624+0000","last_peered":"2024-05-20T22:07:58.705624+0000","last_clean":"2024-05-20T22:07:58.705624+0000","last_became_active":"2024-05-20T22:07:53.653629+0000","last_became_peered":"2024-05-20T22:07:53.653629+0000","last_unstale":"2024-05-20T22:07:58.705624+0000","last_undegraded":"2024-05-20T22:07:58.705624+0000","last_fullsized":"2024-05-20T22:07:58.705624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093667+0000","last_change":"2024-05-20T22:07:52.667566+0000","last_active":"2024-05-20T22:08:00.093667+0000","last_peered":"2024-05-20T22:08:00.093667+0000","last_clean":"2024-05-20T22:08:00.093667+0000","last_became_active":"2024-05-20T22:07:52.667214+0000","last_became_peered":"2024-05-20T22:07:52.667214+0000","last_unstale":"2024-05-20T22:08:00.093667+0000","last_undegraded":"2024-05-20T22:08:00.093667+0000","last_fullsized":"2024-05-20T22:08:00.093667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093690+0000","last_change":"2024-05-20T22:07:57.728380+0000","last_active":"2024-05-20T22:08:00.093690+0000","last_peered":"2024-05-20T22:08:00.093690+0000","last_clean":"2024-05-20T22:08:00.093690+0000","last_became_active":"2024-05-20T22:07:57.726788+0000","last_became_peered":"2024-05-20T22:07:57.726788+0000","last_unstale":"2024-05-20T22:08:00.093690+0000","last_undegraded":"2024-05-20T22:08:00.093690+0000","last_fullsized":"2024-05-20T22:08:00.093690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:46:04.066141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580651+0000","last_change":"2024-05-20T22:07:53.659126+0000","last_active":"2024-05-20T22:07:59.580651+0000","last_peered":"2024-05-20T22:07:59.580651+0000","last_clean":"2024-05-20T22:07:59.580651+0000","last_became_active":"2024-05-20T22:07:53.658751+0000","last_became_peered":"2024-05-20T22:07:53.658751+0000","last_unstale":"2024-05-20T22:07:59.580651+0000","last_undegraded":"2024-05-20T22:07:59.580651+0000","last_fullsized":"2024-05-20T22:07:59.580651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581140+0000","last_change":"2024-05-20T22:07:52.659607+0000","last_active":"2024-05-20T22:07:59.581140+0000","last_peered":"2024-05-20T22:07:59.581140+0000","last_clean":"2024-05-20T22:07:59.581140+0000","last_became_active":"2024-05-20T22:07:52.658939+0000","last_became_peered":"2024-05-20T22:07:52.658939+0000","last_unstale":"2024-05-20T22:07:59.581140+0000","last_undegraded":"2024-05-20T22:07:59.581140+0000","last_fullsized":"2024-05-20T22:07:59.581140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:12:51.889148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093500+0000","last_change":"2024-05-20T22:07:57.726495+0000","last_active":"2024-05-20T22:08:00.093500+0000","last_peered":"2024-05-20T22:08:00.093500+0000","last_clean":"2024-05-20T22:08:00.093500+0000","last_became_active":"2024-05-20T22:07:57.726012+0000","last_became_peered":"2024-05-20T22:07:57.726012+0000","last_unstale":"2024-05-20T22:08:00.093500+0000","last_undegraded":"2024-05-20T22:08:00.093500+0000","last_fullsized":"2024-05-20T22:08:00.093500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:44:34.494247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705411+0000","last_change":"2024-05-20T22:07:53.666422+0000","last_active":"2024-05-20T22:07:58.705411+0000","last_peered":"2024-05-20T22:07:58.705411+0000","last_clean":"2024-05-20T22:07:58.705411+0000","last_became_active":"2024-05-20T22:07:53.666109+0000","last_became_peered":"2024-05-20T22:07:53.666109+0000","last_unstale":"2024-05-20T22:07:58.705411+0000","last_undegraded":"2024-05-20T22:07:58.705411+0000","last_fullsized":"2024-05-20T22:07:58.705411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:03.694258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:08.748697+0000","last_change":"2024-05-20T22:07:52.661686+0000","last_active":"2024-05-20T22:08:08.748697+0000","last_peered":"2024-05-20T22:08:08.748697+0000","last_clean":"2024-05-20T22:08:08.748697+0000","last_became_active":"2024-05-20T22:07:52.661192+0000","last_became_peered":"2024-05-20T22:07:52.661192+0000","last_unstale":"2024-05-20T22:08:08.748697+0000","last_undegraded":"2024-05-20T22:08:08.748697+0000","last_fullsized":"2024-05-20T22:08:08.748697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093188+0000","last_change":"2024-05-20T22:07:57.733250+0000","last_active":"2024-05-20T22:08:00.093188+0000","last_peered":"2024-05-20T22:08:00.093188+0000","last_clean":"2024-05-20T22:08:00.093188+0000","last_became_active":"2024-05-20T22:07:57.732096+0000","last_became_peered":"2024-05-20T22:07:57.732096+0000","last_unstale":"2024-05-20T22:08:00.093188+0000","last_undegraded":"2024-05-20T22:08:00.093188+0000","last_fullsized":"2024-05-20T22:08:00.093188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:30.960467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093127+0000","last_change":"2024-05-20T22:07:53.664378+0000","last_active":"2024-05-20T22:08:00.093127+0000","last_peered":"2024-05-20T22:08:00.093127+0000","last_clean":"2024-05-20T22:08:00.093127+0000","last_became_active":"2024-05-20T22:07:53.664061+0000","last_became_peered":"2024-05-20T22:07:53.664061+0000","last_unstale":"2024-05-20T22:08:00.093127+0000","last_undegraded":"2024-05-20T22:08:00.093127+0000","last_fullsized":"2024-05-20T22:08:00.093127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:42.167135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092791+0000","last_change":"2024-05-20T22:07:52.665754+0000","last_active":"2024-05-20T22:08:00.092791+0000","last_peered":"2024-05-20T22:08:00.092791+0000","last_clean":"2024-05-20T22:08:00.092791+0000","last_became_active":"2024-05-20T22:07:52.663248+0000","last_became_peered":"2024-05-20T22:07:52.663248+0000","last_unstale":"2024-05-20T22:08:00.092791+0000","last_undegraded":"2024-05-20T22:08:00.092791+0000","last_fullsized":"2024-05-20T22:08:00.092791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092574+0000","last_change":"2024-05-20T22:07:57.732444+0000","last_active":"2024-05-20T22:08:00.092574+0000","last_peered":"2024-05-20T22:08:00.092574+0000","last_clean":"2024-05-20T22:08:00.092574+0000","last_became_active":"2024-05-20T22:07:57.731063+0000","last_became_peered":"2024-05-20T22:07:57.731063+0000","last_unstale":"2024-05-20T22:08:00.092574+0000","last_undegraded":"2024-05-20T22:08:00.092574+0000","last_fullsized":"2024-05-20T22:08:00.092574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:17.240623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581288+0000","last_change":"2024-05-20T22:07:53.669268+0000","last_active":"2024-05-20T22:07:59.581288+0000","last_peered":"2024-05-20T22:07:59.581288+0000","last_clean":"2024-05-20T22:07:59.581288+0000","last_became_active":"2024-05-20T22:07:53.666127+0000","last_became_peered":"2024-05-20T22:07:53.666127+0000","last_unstale":"2024-05-20T22:07:59.581288+0000","last_undegraded":"2024-05-20T22:07:59.581288+0000","last_fullsized":"2024-05-20T22:07:59.581288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093187+0000","last_change":"2024-05-20T22:07:52.660573+0000","last_active":"2024-05-20T22:08:00.093187+0000","last_peered":"2024-05-20T22:08:00.093187+0000","last_clean":"2024-05-20T22:08:00.093187+0000","last_became_active":"2024-05-20T22:07:52.660227+0000","last_became_peered":"2024-05-20T22:07:52.660227+0000","last_unstale":"2024-05-20T22:08:00.093187+0000","last_undegraded":"2024-05-20T22:08:00.093187+0000","last_fullsized":"2024-05-20T22:08:00.093187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092346+0000","last_change":"2024-05-20T22:07:57.726865+0000","last_active":"2024-05-20T22:08:00.092346+0000","last_peered":"2024-05-20T22:08:00.092346+0000","last_clean":"2024-05-20T22:08:00.092346+0000","last_became_active":"2024-05-20T22:07:57.726411+0000","last_became_peered":"2024-05-20T22:07:57.726411+0000","last_unstale":"2024-05-20T22:08:00.092346+0000","last_undegraded":"2024-05-20T22:08:00.092346+0000","last_fullsized":"2024-05-20T22:08:00.092346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:14.532804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093027+0000","last_change":"2024-05-20T22:07:53.674069+0000","last_active":"2024-05-20T22:08:00.093027+0000","last_peered":"2024-05-20T22:08:00.093027+0000","last_clean":"2024-05-20T22:08:00.093027+0000","last_became_active":"2024-05-20T22:07:53.673668+0000","last_became_peered":"2024-05-20T22:07:53.673668+0000","last_unstale":"2024-05-20T22:08:00.093027+0000","last_undegraded":"2024-05-20T22:08:00.093027+0000","last_fullsized":"2024-05-20T22:08:00.093027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:03.694258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706745+0000","last_change":"2024-05-20T22:07:52.659232+0000","last_active":"2024-05-20T22:07:58.706745+0000","last_peered":"2024-05-20T22:07:58.706745+0000","last_clean":"2024-05-20T22:07:58.706745+0000","last_became_active":"2024-05-20T22:07:52.657718+0000","last_became_peered":"2024-05-20T22:07:52.657718+0000","last_unstale":"2024-05-20T22:07:58.706745+0000","last_undegraded":"2024-05-20T22:07:58.706745+0000","last_fullsized":"2024-05-20T22:07:58.706745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092928+0000","last_change":"2024-05-20T22:07:57.730328+0000","last_active":"2024-05-20T22:08:00.092928+0000","last_peered":"2024-05-20T22:08:00.092928+0000","last_clean":"2024-05-20T22:08:00.092928+0000","last_became_active":"2024-05-20T22:07:57.729093+0000","last_became_peered":"2024-05-20T22:07:57.729093+0000","last_unstale":"2024-05-20T22:08:00.092928+0000","last_undegraded":"2024-05-20T22:08:00.092928+0000","last_fullsized":"2024-05-20T22:08:00.092928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:42:24.385164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581132+0000","last_change":"2024-05-20T22:07:57.738586+0000","last_active":"2024-05-20T22:07:59.581132+0000","last_peered":"2024-05-20T22:07:59.581132+0000","last_clean":"2024-05-20T22:07:59.581132+0000","last_became_active":"2024-05-20T22:07:57.736969+0000","last_became_peered":"2024-05-20T22:07:57.736969+0000","last_unstale":"2024-05-20T22:07:59.581132+0000","last_undegraded":"2024-05-20T22:07:59.581132+0000","last_fullsized":"2024-05-20T22:07:59.581132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:03:25.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092006+0000","last_change":"2024-05-20T22:07:53.677156+0000","last_active":"2024-05-20T22:08:00.092006+0000","last_peered":"2024-05-20T22:08:00.092006+0000","last_clean":"2024-05-20T22:08:00.092006+0000","last_became_active":"2024-05-20T22:07:53.676805+0000","last_became_peered":"2024-05-20T22:07:53.676805+0000","last_unstale":"2024-05-20T22:08:00.092006+0000","last_undegraded":"2024-05-20T22:08:00.092006+0000","last_fullsized":"2024-05-20T22:08:00.092006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092929+0000","last_change":"2024-05-20T22:07:52.659254+0000","last_active":"2024-05-20T22:08:00.092929+0000","last_peered":"2024-05-20T22:08:00.092929+0000","last_clean":"2024-05-20T22:08:00.092929+0000","last_became_active":"2024-05-20T22:07:52.658938+0000","last_became_peered":"2024-05-20T22:07:52.658938+0000","last_unstale":"2024-05-20T22:08:00.092929+0000","last_undegraded":"2024-05-20T22:08:00.092929+0000","last_fullsized":"2024-05-20T22:08:00.092929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706615+0000","last_change":"2024-05-20T22:07:57.731289+0000","last_active":"2024-05-20T22:07:58.706615+0000","last_peered":"2024-05-20T22:07:58.706615+0000","last_clean":"2024-05-20T22:07:58.706615+0000","last_became_active":"2024-05-20T22:07:57.730844+0000","last_became_peered":"2024-05-20T22:07:57.730844+0000","last_unstale":"2024-05-20T22:07:58.706615+0000","last_undegraded":"2024-05-20T22:07:58.706615+0000","last_fullsized":"2024-05-20T22:07:58.706615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092411+0000","last_change":"2024-05-20T22:07:53.676088+0000","last_active":"2024-05-20T22:08:00.092411+0000","last_peered":"2024-05-20T22:08:00.092411+0000","last_clean":"2024-05-20T22:08:00.092411+0000","last_became_active":"2024-05-20T22:07:53.675756+0000","last_became_peered":"2024-05-20T22:07:53.675756+0000","last_unstale":"2024-05-20T22:08:00.092411+0000","last_undegraded":"2024-05-20T22:08:00.092411+0000","last_fullsized":"2024-05-20T22:08:00.092411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093073+0000","last_change":"2024-05-20T22:07:52.660691+0000","last_active":"2024-05-20T22:08:00.093073+0000","last_peered":"2024-05-20T22:08:00.093073+0000","last_clean":"2024-05-20T22:08:00.093073+0000","last_became_active":"2024-05-20T22:07:52.660324+0000","last_became_peered":"2024-05-20T22:07:52.660324+0000","last_unstale":"2024-05-20T22:08:00.093073+0000","last_undegraded":"2024-05-20T22:08:00.093073+0000","last_fullsized":"2024-05-20T22:08:00.093073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:37.756571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706140+0000","last_change":"2024-05-20T22:07:52.660131+0000","last_active":"2024-05-20T22:07:58.706140+0000","last_peered":"2024-05-20T22:07:58.706140+0000","last_clean":"2024-05-20T22:07:58.706140+0000","last_became_active":"2024-05-20T22:07:52.659784+0000","last_became_peered":"2024-05-20T22:07:52.659784+0000","last_unstale":"2024-05-20T22:07:58.706140+0000","last_undegraded":"2024-05-20T22:07:58.706140+0000","last_fullsized":"2024-05-20T22:07:58.706140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581103+0000","last_change":"2024-05-20T22:07:53.659806+0000","last_active":"2024-05-20T22:07:59.581103+0000","last_peered":"2024-05-20T22:07:59.581103+0000","last_clean":"2024-05-20T22:07:59.581103+0000","last_became_active":"2024-05-20T22:07:53.659275+0000","last_became_peered":"2024-05-20T22:07:53.659275+0000","last_unstale":"2024-05-20T22:07:59.581103+0000","last_undegraded":"2024-05-20T22:07:59.581103+0000","last_fullsized":"2024-05-20T22:07:59.581103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706349+0000","last_change":"2024-05-20T22:07:57.732871+0000","last_active":"2024-05-20T22:07:58.706349+0000","last_peered":"2024-05-20T22:07:58.706349+0000","last_clean":"2024-05-20T22:07:58.706349+0000","last_became_active":"2024-05-20T22:07:57.732391+0000","last_became_peered":"2024-05-20T22:07:57.732391+0000","last_unstale":"2024-05-20T22:07:58.706349+0000","last_undegraded":"2024-05-20T22:07:58.706349+0000","last_fullsized":"2024-05-20T22:07:58.706349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:30.967143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092608+0000","last_change":"2024-05-20T22:07:52.665108+0000","last_active":"2024-05-20T22:08:00.092608+0000","last_peered":"2024-05-20T22:08:00.092608+0000","last_clean":"2024-05-20T22:08:00.092608+0000","last_became_active":"2024-05-20T22:07:52.664750+0000","last_became_peered":"2024-05-20T22:07:52.664750+0000","last_unstale":"2024-05-20T22:08:00.092608+0000","last_undegraded":"2024-05-20T22:08:00.092608+0000","last_fullsized":"2024-05-20T22:08:00.092608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:30.832431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092570+0000","last_change":"2024-05-20T22:07:53.671188+0000","last_active":"2024-05-20T22:08:00.092570+0000","last_peered":"2024-05-20T22:08:00.092570+0000","last_clean":"2024-05-20T22:08:00.092570+0000","last_became_active":"2024-05-20T22:07:53.670842+0000","last_became_peered":"2024-05-20T22:07:53.670842+0000","last_unstale":"2024-05-20T22:08:00.092570+0000","last_undegraded":"2024-05-20T22:08:00.092570+0000","last_fullsized":"2024-05-20T22:08:00.092570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:34.677214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093617+0000","last_change":"2024-05-20T22:07:57.725024+0000","last_active":"2024-05-20T22:08:00.093617+0000","last_peered":"2024-05-20T22:08:00.093617+0000","last_clean":"2024-05-20T22:08:00.093617+0000","last_became_active":"2024-05-20T22:07:57.724576+0000","last_became_peered":"2024-05-20T22:07:57.724576+0000","last_unstale":"2024-05-20T22:08:00.093617+0000","last_undegraded":"2024-05-20T22:08:00.093617+0000","last_fullsized":"2024-05-20T22:08:00.093617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:46:27.003055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705082+0000","last_change":"2024-05-20T22:07:53.680796+0000","last_active":"2024-05-20T22:07:58.705082+0000","last_peered":"2024-05-20T22:07:58.705082+0000","last_clean":"2024-05-20T22:07:58.705082+0000","last_became_active":"2024-05-20T22:07:53.680460+0000","last_became_peered":"2024-05-20T22:07:53.680460+0000","last_unstale":"2024-05-20T22:07:58.705082+0000","last_undegraded":"2024-05-20T22:07:58.705082+0000","last_fullsized":"2024-05-20T22:07:58.705082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705955+0000","last_change":"2024-05-20T22:07:53.664643+0000","last_active":"2024-05-20T22:07:58.705955+0000","last_peered":"2024-05-20T22:07:58.705955+0000","last_clean":"2024-05-20T22:07:58.705955+0000","last_became_active":"2024-05-20T22:07:53.664307+0000","last_became_peered":"2024-05-20T22:07:53.664307+0000","last_unstale":"2024-05-20T22:07:58.705955+0000","last_undegraded":"2024-05-20T22:07:58.705955+0000","last_fullsized":"2024-05-20T22:07:58.705955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:08.402858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093600+0000","last_change":"2024-05-20T22:07:53.669161+0000","last_active":"2024-05-20T22:08:00.093600+0000","last_peered":"2024-05-20T22:08:00.093600+0000","last_clean":"2024-05-20T22:08:00.093600+0000","last_became_active":"2024-05-20T22:07:53.668720+0000","last_became_peered":"2024-05-20T22:07:53.668720+0000","last_unstale":"2024-05-20T22:08:00.093600+0000","last_undegraded":"2024-05-20T22:08:00.093600+0000","last_fullsized":"2024-05-20T22:08:00.093600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:42.757306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580972+0000","last_change":"2024-05-20T22:07:53.667728+0000","last_active":"2024-05-20T22:07:59.580972+0000","last_peered":"2024-05-20T22:07:59.580972+0000","last_clean":"2024-05-20T22:07:59.580972+0000","last_became_active":"2024-05-20T22:07:53.665722+0000","last_became_peered":"2024-05-20T22:07:53.665722+0000","last_unstale":"2024-05-20T22:07:59.580972+0000","last_undegraded":"2024-05-20T22:07:59.580972+0000","last_fullsized":"2024-05-20T22:07:59.580972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:48.605408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093771+0000","last_change":"2024-05-20T22:07:53.665673+0000","last_active":"2024-05-20T22:08:00.093771+0000","last_peered":"2024-05-20T22:08:00.093771+0000","last_clean":"2024-05-20T22:08:00.093771+0000","last_became_active":"2024-05-20T22:07:53.665322+0000","last_became_peered":"2024-05-20T22:07:53.665322+0000","last_unstale":"2024-05-20T22:08:00.093771+0000","last_undegraded":"2024-05-20T22:08:00.093771+0000","last_fullsized":"2024-05-20T22:08:00.093771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092400+0000","last_change":"2024-05-20T22:07:53.673539+0000","last_active":"2024-05-20T22:08:00.092400+0000","last_peered":"2024-05-20T22:08:00.092400+0000","last_clean":"2024-05-20T22:08:00.092400+0000","last_became_active":"2024-05-20T22:07:53.673213+0000","last_became_peered":"2024-05-20T22:07:53.673213+0000","last_unstale":"2024-05-20T22:08:00.092400+0000","last_undegraded":"2024-05-20T22:08:00.092400+0000","last_fullsized":"2024-05-20T22:08:00.092400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706633+0000","last_change":"2024-05-20T22:07:53.676094+0000","last_active":"2024-05-20T22:07:58.706633+0000","last_peered":"2024-05-20T22:07:58.706633+0000","last_clean":"2024-05-20T22:07:58.706633+0000","last_became_active":"2024-05-20T22:07:53.675752+0000","last_became_peered":"2024-05-20T22:07:53.675752+0000","last_unstale":"2024-05-20T22:07:58.706633+0000","last_undegraded":"2024-05-20T22:07:58.706633+0000","last_fullsized":"2024-05-20T22:07:58.706633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093799+0000","last_change":"2024-05-20T22:07:53.665953+0000","last_active":"2024-05-20T22:08:00.093799+0000","last_peered":"2024-05-20T22:08:00.093799+0000","last_clean":"2024-05-20T22:08:00.093799+0000","last_became_active":"2024-05-20T22:07:53.665631+0000","last_became_peered":"2024-05-20T22:07:53.665631+0000","last_unstale":"2024-05-20T22:08:00.093799+0000","last_undegraded":"2024-05-20T22:08:00.093799+0000","last_fullsized":"2024-05-20T22:08:00.093799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092802+0000","last_change":"2024-05-20T22:07:53.669986+0000","last_active":"2024-05-20T22:08:00.092802+0000","last_peered":"2024-05-20T22:08:00.092802+0000","last_clean":"2024-05-20T22:08:00.092802+0000","last_became_active":"2024-05-20T22:07:53.669565+0000","last_became_peered":"2024-05-20T22:07:53.669565+0000","last_unstale":"2024-05-20T22:08:00.092802+0000","last_undegraded":"2024-05-20T22:08:00.092802+0000","last_fullsized":"2024-05-20T22:08:00.092802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092993+0000","last_change":"2024-05-20T22:07:53.667290+0000","last_active":"2024-05-20T22:08:00.092993+0000","last_peered":"2024-05-20T22:08:00.092993+0000","last_clean":"2024-05-20T22:08:00.092993+0000","last_became_active":"2024-05-20T22:07:53.666953+0000","last_became_peered":"2024-05-20T22:07:53.666953+0000","last_unstale":"2024-05-20T22:08:00.092993+0000","last_undegraded":"2024-05-20T22:08:00.092993+0000","last_fullsized":"2024-05-20T22:08:00.092993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:48.605408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093536+0000","last_change":"2024-05-20T22:07:53.673713+0000","last_active":"2024-05-20T22:08:00.093536+0000","last_peered":"2024-05-20T22:08:00.093536+0000","last_clean":"2024-05-20T22:08:00.093536+0000","last_became_active":"2024-05-20T22:07:53.673419+0000","last_became_peered":"2024-05-20T22:07:53.673419+0000","last_unstale":"2024-05-20T22:08:00.093536+0000","last_undegraded":"2024-05-20T22:08:00.093536+0000","last_fullsized":"2024-05-20T22:08:00.093536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580805+0000","last_change":"2024-05-20T22:07:53.657177+0000","last_active":"2024-05-20T22:07:59.580805+0000","last_peered":"2024-05-20T22:07:59.580805+0000","last_clean":"2024-05-20T22:07:59.580805+0000","last_became_active":"2024-05-20T22:07:53.656847+0000","last_became_peered":"2024-05-20T22:07:53.656847+0000","last_unstale":"2024-05-20T22:07:59.580805+0000","last_undegraded":"2024-05-20T22:07:59.580805+0000","last_fullsized":"2024-05-20T22:07:59.580805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706361+0000","last_change":"2024-05-20T22:07:53.675562+0000","last_active":"2024-05-20T22:07:58.706361+0000","last_peered":"2024-05-20T22:07:58.706361+0000","last_clean":"2024-05-20T22:07:58.706361+0000","last_became_active":"2024-05-20T22:07:53.673333+0000","last_became_peered":"2024-05-20T22:07:53.673333+0000","last_unstale":"2024-05-20T22:07:58.706361+0000","last_undegraded":"2024-05-20T22:07:58.706361+0000","last_fullsized":"2024-05-20T22:07:58.706361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706689+0000","last_change":"2024-05-20T22:07:53.664701+0000","last_active":"2024-05-20T22:07:58.706689+0000","last_peered":"2024-05-20T22:07:58.706689+0000","last_clean":"2024-05-20T22:07:58.706689+0000","last_became_active":"2024-05-20T22:07:53.663423+0000","last_became_peered":"2024-05-20T22:07:53.663423+0000","last_unstale":"2024-05-20T22:07:58.706689+0000","last_undegraded":"2024-05-20T22:07:58.706689+0000","last_fullsized":"2024-05-20T22:07:58.706689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706441+0000","last_change":"2024-05-20T22:07:53.677915+0000","last_active":"2024-05-20T22:07:58.706441+0000","last_peered":"2024-05-20T22:07:58.706441+0000","last_clean":"2024-05-20T22:07:58.706441+0000","last_became_active":"2024-05-20T22:07:53.677479+0000","last_became_peered":"2024-05-20T22:07:53.677479+0000","last_unstale":"2024-05-20T22:07:58.706441+0000","last_undegraded":"2024-05-20T22:07:58.706441+0000","last_fullsized":"2024-05-20T22:07:58.706441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:42.167135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092947+0000","last_change":"2024-05-20T22:07:53.674336+0000","last_active":"2024-05-20T22:08:00.092947+0000","last_peered":"2024-05-20T22:08:00.092947+0000","last_clean":"2024-05-20T22:08:00.092947+0000","last_became_active":"2024-05-20T22:07:53.673643+0000","last_became_peered":"2024-05-20T22:07:53.673643+0000","last_unstale":"2024-05-20T22:08:00.092947+0000","last_undegraded":"2024-05-20T22:08:00.092947+0000","last_fullsized":"2024-05-20T22:08:00.092947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:26.916927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092025+0000","last_change":"2024-05-20T22:07:53.680694+0000","last_active":"2024-05-20T22:08:00.092025+0000","last_peered":"2024-05-20T22:08:00.092025+0000","last_clean":"2024-05-20T22:08:00.092025+0000","last_became_active":"2024-05-20T22:07:53.680338+0000","last_became_peered":"2024-05-20T22:07:53.680338+0000","last_unstale":"2024-05-20T22:08:00.092025+0000","last_undegraded":"2024-05-20T22:08:00.092025+0000","last_fullsized":"2024-05-20T22:08:00.092025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092124+0000","last_change":"2024-05-20T22:07:53.668909+0000","last_active":"2024-05-20T22:08:00.092124+0000","last_peered":"2024-05-20T22:08:00.092124+0000","last_clean":"2024-05-20T22:08:00.092124+0000","last_became_active":"2024-05-20T22:07:53.668540+0000","last_became_peered":"2024-05-20T22:07:53.668540+0000","last_unstale":"2024-05-20T22:08:00.092124+0000","last_undegraded":"2024-05-20T22:08:00.092124+0000","last_fullsized":"2024-05-20T22:08:00.092124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580610+0000","last_change":"2024-05-20T22:07:53.671760+0000","last_active":"2024-05-20T22:07:59.580610+0000","last_peered":"2024-05-20T22:07:59.580610+0000","last_clean":"2024-05-20T22:07:59.580610+0000","last_became_active":"2024-05-20T22:07:53.671454+0000","last_became_peered":"2024-05-20T22:07:53.671454+0000","last_unstale":"2024-05-20T22:07:59.580610+0000","last_undegraded":"2024-05-20T22:07:59.580610+0000","last_fullsized":"2024-05-20T22:07:59.580610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:56.576260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092384+0000","last_change":"2024-05-20T22:07:53.678862+0000","last_active":"2024-05-20T22:08:00.092384+0000","last_peered":"2024-05-20T22:08:00.092384+0000","last_clean":"2024-05-20T22:08:00.092384+0000","last_became_active":"2024-05-20T22:07:53.678323+0000","last_became_peered":"2024-05-20T22:07:53.678323+0000","last_unstale":"2024-05-20T22:08:00.092384+0000","last_undegraded":"2024-05-20T22:08:00.092384+0000","last_fullsized":"2024-05-20T22:08:00.092384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093008+0000","last_change":"2024-05-20T22:07:53.677127+0000","last_active":"2024-05-20T22:08:00.093008+0000","last_peered":"2024-05-20T22:08:00.093008+0000","last_clean":"2024-05-20T22:08:00.093008+0000","last_became_active":"2024-05-20T22:07:53.676804+0000","last_became_peered":"2024-05-20T22:07:53.676804+0000","last_unstale":"2024-05-20T22:08:00.093008+0000","last_undegraded":"2024-05-20T22:08:00.093008+0000","last_fullsized":"2024-05-20T22:08:00.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:42.524695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706507+0000","last_change":"2024-05-20T22:07:53.677487+0000","last_active":"2024-05-20T22:07:58.706507+0000","last_peered":"2024-05-20T22:07:58.706507+0000","last_clean":"2024-05-20T22:07:58.706507+0000","last_became_active":"2024-05-20T22:07:53.677160+0000","last_became_peered":"2024-05-20T22:07:53.677160+0000","last_unstale":"2024-05-20T22:07:58.706507+0000","last_undegraded":"2024-05-20T22:07:58.706507+0000","last_fullsized":"2024-05-20T22:07:58.706507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706545+0000","last_change":"2024-05-20T22:07:53.675130+0000","last_active":"2024-05-20T22:07:58.706545+0000","last_peered":"2024-05-20T22:07:58.706545+0000","last_clean":"2024-05-20T22:07:58.706545+0000","last_became_active":"2024-05-20T22:07:53.674566+0000","last_became_peered":"2024-05-20T22:07:53.674566+0000","last_unstale":"2024-05-20T22:07:58.706545+0000","last_undegraded":"2024-05-20T22:07:58.706545+0000","last_fullsized":"2024-05-20T22:07:58.706545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707188+0000","last_change":"2024-05-20T22:07:53.668737+0000","last_active":"2024-05-20T22:07:58.707188+0000","last_peered":"2024-05-20T22:07:58.707188+0000","last_clean":"2024-05-20T22:07:58.707188+0000","last_became_active":"2024-05-20T22:07:53.668436+0000","last_became_peered":"2024-05-20T22:07:53.668436+0000","last_unstale":"2024-05-20T22:07:58.707188+0000","last_undegraded":"2024-05-20T22:07:58.707188+0000","last_fullsized":"2024-05-20T22:07:58.707188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:26.916927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092105+0000","last_change":"2024-05-20T22:07:53.667651+0000","last_active":"2024-05-20T22:08:00.092105+0000","last_peered":"2024-05-20T22:08:00.092105+0000","last_clean":"2024-05-20T22:08:00.092105+0000","last_became_active":"2024-05-20T22:07:53.667290+0000","last_became_peered":"2024-05-20T22:07:53.667290+0000","last_unstale":"2024-05-20T22:08:00.092105+0000","last_undegraded":"2024-05-20T22:08:00.092105+0000","last_fullsized":"2024-05-20T22:08:00.092105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705225+0000","last_change":"2024-05-20T22:07:53.655295+0000","last_active":"2024-05-20T22:07:58.705225+0000","last_peered":"2024-05-20T22:07:58.705225+0000","last_clean":"2024-05-20T22:07:58.705225+0000","last_became_active":"2024-05-20T22:07:53.654945+0000","last_became_peered":"2024-05-20T22:07:53.654945+0000","last_unstale":"2024-05-20T22:07:58.705225+0000","last_undegraded":"2024-05-20T22:07:58.705225+0000","last_fullsized":"2024-05-20T22:07:58.705225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:42.757306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093460+0000","last_change":"2024-05-20T22:07:53.681737+0000","last_active":"2024-05-20T22:08:00.093460+0000","last_peered":"2024-05-20T22:08:00.093460+0000","last_clean":"2024-05-20T22:08:00.093460+0000","last_became_active":"2024-05-20T22:07:53.681436+0000","last_became_peered":"2024-05-20T22:07:53.681436+0000","last_unstale":"2024-05-20T22:08:00.093460+0000","last_undegraded":"2024-05-20T22:08:00.093460+0000","last_fullsized":"2024-05-20T22:08:00.093460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705391+0000","last_change":"2024-05-20T22:07:53.662939+0000","last_active":"2024-05-20T22:07:58.705391+0000","last_peered":"2024-05-20T22:07:58.705391+0000","last_clean":"2024-05-20T22:07:58.705391+0000","last_became_active":"2024-05-20T22:07:53.662545+0000","last_became_peered":"2024-05-20T22:07:53.662545+0000","last_unstale":"2024-05-20T22:07:58.705391+0000","last_undegraded":"2024-05-20T22:07:58.705391+0000","last_fullsized":"2024-05-20T22:07:58.705391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092568+0000","last_change":"2024-05-20T22:07:53.677132+0000","last_active":"2024-05-20T22:08:00.092568+0000","last_peered":"2024-05-20T22:08:00.092568+0000","last_clean":"2024-05-20T22:08:00.092568+0000","last_became_active":"2024-05-20T22:07:53.676804+0000","last_became_peered":"2024-05-20T22:07:53.676804+0000","last_unstale":"2024-05-20T22:08:00.092568+0000","last_undegraded":"2024-05-20T22:08:00.092568+0000","last_fullsized":"2024-05-20T22:08:00.092568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580901+0000","last_change":"2024-05-20T22:07:53.669468+0000","last_active":"2024-05-20T22:07:59.580901+0000","last_peered":"2024-05-20T22:07:59.580901+0000","last_clean":"2024-05-20T22:07:59.580901+0000","last_became_active":"2024-05-20T22:07:53.669074+0000","last_became_peered":"2024-05-20T22:07:53.669074+0000","last_unstale":"2024-05-20T22:07:59.580901+0000","last_undegraded":"2024-05-20T22:07:59.580901+0000","last_fullsized":"2024-05-20T22:07:59.580901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:26.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092451+0000","last_change":"2024-05-20T22:07:53.680851+0000","last_active":"2024-05-20T22:08:00.092451+0000","last_peered":"2024-05-20T22:08:00.092451+0000","last_clean":"2024-05-20T22:08:00.092451+0000","last_became_active":"2024-05-20T22:07:53.680468+0000","last_became_peered":"2024-05-20T22:07:53.680468+0000","last_unstale":"2024-05-20T22:08:00.092451+0000","last_undegraded":"2024-05-20T22:08:00.092451+0000","last_fullsized":"2024-05-20T22:08:00.092451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707096+0000","last_change":"2024-05-20T22:07:53.675836+0000","last_active":"2024-05-20T22:07:58.707096+0000","last_peered":"2024-05-20T22:07:58.707096+0000","last_clean":"2024-05-20T22:07:58.707096+0000","last_became_active":"2024-05-20T22:07:53.675544+0000","last_became_peered":"2024-05-20T22:07:53.675544+0000","last_unstale":"2024-05-20T22:07:58.707096+0000","last_undegraded":"2024-05-20T22:07:58.707096+0000","last_fullsized":"2024-05-20T22:07:58.707096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:42.044571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:08:12.818 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:08:12.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.966+0000 7f0f2523d640 1 Processor -- start 2024-05-20T22:08:12.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.966+0000 7f0f2523d640 1 -- start start 2024-05-20T22:08:12.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f2523d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0f200d3390 0x7f0f2006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f2523d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0f200b8720 0x7f0f2015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f2523d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f2006ce50 con 0x7f0f200b3c90 2024-05-20T22:08:12.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f2523d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f0f2006b870 con 0x7f0f200d3390 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f2523d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0f2006bb50 con 0x7f0f200b8720 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f1ed9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0f200d3390 0x7f0f2006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f1e59c640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0f200b3c90 0x7f0f200b8010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46292/0 (socket says 172.21.15.55:46292) 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f1ed9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0f200d3390 0x7f0f2006c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57892/0 (socket says 172.21.15.55:57892) 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f1e59c640 1 -- 172.21.15.55:0/564168772 learned_addr learned my addr 172.21.15.55:0/564168772 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f1f59e640 1 --2- 172.21.15.55:0/564168772 >> v2:172.21.15.138:3300/0 conn(0x7f0f200b8720 0x7f0f2015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f1ed9d640 1 -- 172.21.15.55:0/564168772 >> v2:172.21.15.138:3300/0 conn(0x7f0f200b8720 msgr2=0x7f0f2015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f1ed9d640 1 --2- 172.21.15.55:0/564168772 >> v2:172.21.15.138:3300/0 conn(0x7f0f200b8720 0x7f0f2015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f1ed9d640 1 -- 172.21.15.55:0/564168772 >> v1:172.21.15.55:6789/0 conn(0x7f0f200b3c90 legacy=0x7f0f200b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f1ed9d640 1 -- 172.21.15.55:0/564168772 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2006be30 con 0x7f0f200d3390 2024-05-20T22:08:12.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.968+0000 7f0f1dd9b640 1 -- 172.21.15.55:0/564168772 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 912176343 0 0) 0x7f0f2006ce50 con 0x7f0f200b3c90 2024-05-20T22:08:12.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.969+0000 7f0f1ed9d640 1 --2- 172.21.15.55:0/564168772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200d3390 0x7f0f2006c910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f200d1390 tx=0x7f0f1002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c0905bdf8a3b83 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:12.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.969+0000 7f0f1dd9b640 1 -- 172.21.15.55:0/564168772 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10007740 con 0x7f0f200d3390 2024-05-20T22:08:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.969+0000 7f0f1dd9b640 1 -- 172.21.15.55:0/564168772 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f100078e0 con 0x7f0f200d3390 2024-05-20T22:08:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.969+0000 7f0f1dd9b640 1 -- 172.21.15.55:0/564168772 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10007cb0 con 0x7f0f200d3390 2024-05-20T22:08:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.970+0000 7f0f2523d640 1 -- 172.21.15.55:0/564168772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200d3390 msgr2=0x7f0f2006c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.970+0000 7f0f2523d640 1 --2- 172.21.15.55:0/564168772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200d3390 0x7f0f2006c910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f200d1390 tx=0x7f0f1002a680 comp rx=0 tx=0).stop 2024-05-20T22:08:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.970+0000 7f0f2523d640 1 -- 172.21.15.55:0/564168772 shutdown_connections 2024-05-20T22:08:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.970+0000 7f0f2523d640 1 --2- 172.21.15.55:0/564168772 >> v2:172.21.15.138:3300/0 conn(0x7f0f200b8720 0x7f0f2015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.970+0000 7f0f2523d640 1 --2- 172.21.15.55:0/564168772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200d3390 0x7f0f2006c910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.970+0000 7f0f2523d640 1 -- 172.21.15.55:0/564168772 >> 172.21.15.55:0/564168772 conn(0x7f0f200b65f0 msgr2=0x7f0f200b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.970+0000 7f0f2523d640 1 -- 172.21.15.55:0/564168772 shutdown_connections 2024-05-20T22:08:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.970+0000 7f0f2523d640 1 -- 172.21.15.55:0/564168772 wait complete. 2024-05-20T22:08:12.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.972+0000 7f0f2523d640 1 Processor -- start 2024-05-20T22:08:12.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.972+0000 7f0f2523d640 1 -- start start 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.973+0000 7f0f2523d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200b3c90 0x7f0f2015f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.973+0000 7f0f2523d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f200b8720 0x7f0f201e8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.973+0000 7f0f2523d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f200d3390 0x7f0f200b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.973+0000 7f0f2523d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2006b500 con 0x7f0f200b3c90 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.973+0000 7f0f2523d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2006c3f0 con 0x7f0f200b8720 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.973+0000 7f0f2523d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0f200b8b30 con 0x7f0f200d3390 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.973+0000 7f0f1ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200b3c90 0x7f0f2015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.973+0000 7f0f1ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200b3c90 0x7f0f2015f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57898/0 (socket says 172.21.15.55:57898) 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.973+0000 7f0f1ed9d640 1 -- 172.21.15.55:0/2016462108 learned_addr learned my addr 172.21.15.55:0/2016462108 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f1e59c640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f200b8720 0x7f0f201e8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f1f59e640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f200d3390 0x7f0f200b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f1ed9d640 1 -- 172.21.15.55:0/2016462108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f200d3390 msgr2=0x7f0f200b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f1ed9d640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f200d3390 0x7f0f200b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f1ed9d640 1 -- 172.21.15.55:0/2016462108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f200b8720 msgr2=0x7f0f201e8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f1ed9d640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f200b8720 0x7f0f201e8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f1ed9d640 1 -- 172.21.15.55:0/2016462108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f201e9540 con 0x7f0f200b3c90 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f1f59e640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f200d3390 0x7f0f200b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f1ed9d640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200b3c90 0x7f0f2015f940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0f1002ac70 tx=0x7f0f10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f177fe640 1 -- 172.21.15.55:0/2016462108 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10040070 con 0x7f0f200b3c90 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f177fe640 1 -- 172.21.15.55:0/2016462108 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f10031c40 con 0x7f0f200b3c90 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f2523d640 1 -- 172.21.15.55:0/2016462108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f201e9820 con 0x7f0f200b3c90 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f177fe640 1 -- 172.21.15.55:0/2016462108 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10045460 con 0x7f0f200b3c90 2024-05-20T22:08:12.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.974+0000 7f0f2523d640 1 -- 172.21.15.55:0/2016462108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f201e9de0 con 0x7f0f200b3c90 2024-05-20T22:08:12.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.975+0000 7f0f2523d640 1 -- 172.21.15.55:0/2016462108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f20069090 con 0x7f0f200b3c90 2024-05-20T22:08:12.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.976+0000 7f0f177fe640 1 -- 172.21.15.55:0/2016462108 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f10039040 con 0x7f0f200b3c90 2024-05-20T22:08:12.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.976+0000 7f0f177fe640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0f0006be30 0x7f0f0006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:12.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.976+0000 7f0f177fe640 1 -- 172.21.15.55:0/2016462108 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0f100acc20 con 0x7f0f200b3c90 2024-05-20T22:08:12.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.977+0000 7f0f1e59c640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0f0006be30 0x7f0f0006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:12.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.977+0000 7f0f1e59c640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0f0006be30 0x7f0f0006e340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f201ea570 tx=0x7f0f08007c00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:12.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:12.981+0000 7f0f177fe640 1 -- 172.21.15.55:0/2016462108 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0f1007b160 con 0x7f0f200b3c90 2024-05-20T22:08:13.155 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:08:13.158 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:08:13.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.163+0000 7f0f2523d640 1 -- 172.21.15.55:0/2016462108 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f201ea150 con 0x7f0f0006be30 2024-05-20T22:08:13.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.172+0000 7f0f177fe640 1 -- 172.21.15.55:0/2016462108 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332562 (secure 0 0 0) 0x7f0f201ea150 con 0x7f0f0006be30 2024-05-20T22:08:13.177 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:13.179 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:08:13.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 -- 172.21.15.55:0/2016462108 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0f0006be30 msgr2=0x7f0f0006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0f0006be30 0x7f0f0006e340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f201ea570 tx=0x7f0f08007c00 comp rx=0 tx=0).stop 2024-05-20T22:08:13.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 -- 172.21.15.55:0/2016462108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200b3c90 msgr2=0x7f0f2015f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200b3c90 0x7f0f2015f940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0f1002ac70 tx=0x7f0f10004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 -- 172.21.15.55:0/2016462108 shutdown_connections 2024-05-20T22:08:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0f0006be30 0x7f0f0006e340 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0f200d3390 0x7f0f200b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f200b8720 0x7f0f201e8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 --2- 172.21.15.55:0/2016462108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200b3c90 0x7f0f2015f940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 -- 172.21.15.55:0/2016462108 >> 172.21.15.55:0/2016462108 conn(0x7f0f200b65f0 msgr2=0x7f0f200b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 -- 172.21.15.55:0/2016462108 shutdown_connections 2024-05-20T22:08:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.176+0000 7f0f2523d640 1 -- 172.21.15.55:0/2016462108 wait complete. 2024-05-20T22:08:13.189 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-05-20T22:08:12.331063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236020,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738700,"statfs":{"total":773094113280,"available":772852428800,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002443"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581341+0000","last_change":"2024-05-20T22:07:52.664054+0000","last_active":"2024-05-20T22:07:59.581341+0000","last_peered":"2024-05-20T22:07:59.581341+0000","last_clean":"2024-05-20T22:07:59.581341+0000","last_became_active":"2024-05-20T22:07:52.663784+0000","last_became_peered":"2024-05-20T22:07:52.663784+0000","last_unstale":"2024-05-20T22:07:59.581341+0000","last_undegraded":"2024-05-20T22:07:59.581341+0000","last_fullsized":"2024-05-20T22:07:59.581341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:37.756571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092411+0000","last_change":"2024-05-20T22:07:53.678918+0000","last_active":"2024-05-20T22:08:00.092411+0000","last_peered":"2024-05-20T22:08:00.092411+0000","last_clean":"2024-05-20T22:08:00.092411+0000","last_became_active":"2024-05-20T22:07:53.678467+0000","last_became_peered":"2024-05-20T22:07:53.678467+0000","last_unstale":"2024-05-20T22:08:00.092411+0000","last_undegraded":"2024-05-20T22:08:00.092411+0000","last_fullsized":"2024-05-20T22:08:00.092411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092012+0000","last_change":"2024-05-20T22:07:57.732174+0000","last_active":"2024-05-20T22:08:00.092012+0000","last_peered":"2024-05-20T22:08:00.092012+0000","last_clean":"2024-05-20T22:08:00.092012+0000","last_became_active":"2024-05-20T22:07:57.731124+0000","last_became_peered":"2024-05-20T22:07:57.731124+0000","last_unstale":"2024-05-20T22:08:00.092012+0000","last_undegraded":"2024-05-20T22:08:00.092012+0000","last_fullsized":"2024-05-20T22:08:00.092012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:11.233995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092785+0000","last_change":"2024-05-20T22:07:52.664457+0000","last_active":"2024-05-20T22:08:00.092785+0000","last_peered":"2024-05-20T22:08:00.092785+0000","last_clean":"2024-05-20T22:08:00.092785+0000","last_became_active":"2024-05-20T22:07:52.664121+0000","last_became_peered":"2024-05-20T22:07:52.664121+0000","last_unstale":"2024-05-20T22:08:00.092785+0000","last_undegraded":"2024-05-20T22:08:00.092785+0000","last_fullsized":"2024-05-20T22:08:00.092785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581131+0000","last_change":"2024-05-20T22:07:53.669570+0000","last_active":"2024-05-20T22:07:59.581131+0000","last_peered":"2024-05-20T22:07:59.581131+0000","last_clean":"2024-05-20T22:07:59.581131+0000","last_became_active":"2024-05-20T22:07:53.668913+0000","last_became_peered":"2024-05-20T22:07:53.668913+0000","last_unstale":"2024-05-20T22:07:59.581131+0000","last_undegraded":"2024-05-20T22:07:59.581131+0000","last_fullsized":"2024-05-20T22:07:59.581131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092313+0000","last_change":"2024-05-20T22:07:57.728254+0000","last_active":"2024-05-20T22:08:00.092313+0000","last_peered":"2024-05-20T22:08:00.092313+0000","last_clean":"2024-05-20T22:08:00.092313+0000","last_became_active":"2024-05-20T22:07:57.727787+0000","last_became_peered":"2024-05-20T22:07:57.727787+0000","last_unstale":"2024-05-20T22:08:00.092313+0000","last_undegraded":"2024-05-20T22:08:00.092313+0000","last_fullsized":"2024-05-20T22:08:00.092313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:43.588870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706646+0000","last_change":"2024-05-20T22:07:52.657626+0000","last_active":"2024-05-20T22:07:58.706646+0000","last_peered":"2024-05-20T22:07:58.706646+0000","last_clean":"2024-05-20T22:07:58.706646+0000","last_became_active":"2024-05-20T22:07:52.657298+0000","last_became_peered":"2024-05-20T22:07:52.657298+0000","last_unstale":"2024-05-20T22:07:58.706646+0000","last_undegraded":"2024-05-20T22:07:58.706646+0000","last_fullsized":"2024-05-20T22:07:58.706646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:41.518373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092705+0000","last_change":"2024-05-20T22:07:53.669706+0000","last_active":"2024-05-20T22:08:00.092705+0000","last_peered":"2024-05-20T22:08:00.092705+0000","last_clean":"2024-05-20T22:08:00.092705+0000","last_became_active":"2024-05-20T22:07:53.669332+0000","last_became_peered":"2024-05-20T22:07:53.669332+0000","last_unstale":"2024-05-20T22:08:00.092705+0000","last_undegraded":"2024-05-20T22:08:00.092705+0000","last_fullsized":"2024-05-20T22:08:00.092705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092885+0000","last_change":"2024-05-20T22:07:57.734186+0000","last_active":"2024-05-20T22:08:00.092885+0000","last_peered":"2024-05-20T22:08:00.092885+0000","last_clean":"2024-05-20T22:08:00.092885+0000","last_became_active":"2024-05-20T22:07:57.733618+0000","last_became_peered":"2024-05-20T22:07:57.733618+0000","last_unstale":"2024-05-20T22:08:00.092885+0000","last_undegraded":"2024-05-20T22:08:00.092885+0000","last_fullsized":"2024-05-20T22:08:00.092885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:46.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706028+0000","last_change":"2024-05-20T22:07:52.660033+0000","last_active":"2024-05-20T22:07:58.706028+0000","last_peered":"2024-05-20T22:07:58.706028+0000","last_clean":"2024-05-20T22:07:58.706028+0000","last_became_active":"2024-05-20T22:07:52.659733+0000","last_became_peered":"2024-05-20T22:07:52.659733+0000","last_unstale":"2024-05-20T22:07:58.706028+0000","last_undegraded":"2024-05-20T22:07:58.706028+0000","last_fullsized":"2024-05-20T22:07:58.706028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:41.160812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093319+0000","last_change":"2024-05-20T22:07:53.669432+0000","last_active":"2024-05-20T22:08:00.093319+0000","last_peered":"2024-05-20T22:08:00.093319+0000","last_clean":"2024-05-20T22:08:00.093319+0000","last_became_active":"2024-05-20T22:07:53.668235+0000","last_became_peered":"2024-05-20T22:07:53.668235+0000","last_unstale":"2024-05-20T22:08:00.093319+0000","last_undegraded":"2024-05-20T22:08:00.093319+0000","last_fullsized":"2024-05-20T22:08:00.093319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:42.044571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092545+0000","last_change":"2024-05-20T22:07:57.734065+0000","last_active":"2024-05-20T22:08:00.092545+0000","last_peered":"2024-05-20T22:08:00.092545+0000","last_clean":"2024-05-20T22:08:00.092545+0000","last_became_active":"2024-05-20T22:07:57.733537+0000","last_became_peered":"2024-05-20T22:07:57.733537+0000","last_unstale":"2024-05-20T22:08:00.092545+0000","last_undegraded":"2024-05-20T22:08:00.092545+0000","last_fullsized":"2024-05-20T22:08:00.092545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:16.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705724+0000","last_change":"2024-05-20T22:07:53.675279+0000","last_active":"2024-05-20T22:07:58.705724+0000","last_peered":"2024-05-20T22:07:58.705724+0000","last_clean":"2024-05-20T22:07:58.705724+0000","last_became_active":"2024-05-20T22:07:53.674924+0000","last_became_peered":"2024-05-20T22:07:53.674924+0000","last_unstale":"2024-05-20T22:07:58.705724+0000","last_undegraded":"2024-05-20T22:07:58.705724+0000","last_fullsized":"2024-05-20T22:07:58.705724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705726+0000","last_change":"2024-05-20T22:07:52.655127+0000","last_active":"2024-05-20T22:07:58.705726+0000","last_peered":"2024-05-20T22:07:58.705726+0000","last_clean":"2024-05-20T22:07:58.705726+0000","last_became_active":"2024-05-20T22:07:52.654807+0000","last_became_peered":"2024-05-20T22:07:52.654807+0000","last_unstale":"2024-05-20T22:07:58.705726+0000","last_undegraded":"2024-05-20T22:07:58.705726+0000","last_fullsized":"2024-05-20T22:07:58.705726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093607+0000","last_change":"2024-05-20T22:07:57.732155+0000","last_active":"2024-05-20T22:08:00.093607+0000","last_peered":"2024-05-20T22:08:00.093607+0000","last_clean":"2024-05-20T22:08:00.093607+0000","last_became_active":"2024-05-20T22:07:57.731370+0000","last_became_peered":"2024-05-20T22:07:57.731370+0000","last_unstale":"2024-05-20T22:08:00.093607+0000","last_undegraded":"2024-05-20T22:08:00.093607+0000","last_fullsized":"2024-05-20T22:08:00.093607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:30.698775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092498+0000","last_change":"2024-05-20T22:07:53.677185+0000","last_active":"2024-05-20T22:08:00.092498+0000","last_peered":"2024-05-20T22:08:00.092498+0000","last_clean":"2024-05-20T22:08:00.092498+0000","last_became_active":"2024-05-20T22:07:53.676661+0000","last_became_peered":"2024-05-20T22:07:53.676661+0000","last_unstale":"2024-05-20T22:08:00.092498+0000","last_undegraded":"2024-05-20T22:08:00.092498+0000","last_fullsized":"2024-05-20T22:08:00.092498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093251+0000","last_change":"2024-05-20T22:07:52.668583+0000","last_active":"2024-05-20T22:08:00.093251+0000","last_peered":"2024-05-20T22:08:00.093251+0000","last_clean":"2024-05-20T22:08:00.093251+0000","last_became_active":"2024-05-20T22:07:52.667850+0000","last_became_peered":"2024-05-20T22:07:52.667850+0000","last_unstale":"2024-05-20T22:08:00.093251+0000","last_undegraded":"2024-05-20T22:08:00.093251+0000","last_fullsized":"2024-05-20T22:08:00.093251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092843+0000","last_change":"2024-05-20T22:07:57.730407+0000","last_active":"2024-05-20T22:08:00.092843+0000","last_peered":"2024-05-20T22:08:00.092843+0000","last_clean":"2024-05-20T22:08:00.092843+0000","last_became_active":"2024-05-20T22:07:57.729918+0000","last_became_peered":"2024-05-20T22:07:57.729918+0000","last_unstale":"2024-05-20T22:08:00.092843+0000","last_undegraded":"2024-05-20T22:08:00.092843+0000","last_fullsized":"2024-05-20T22:08:00.092843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:44.596866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092522+0000","last_change":"2024-05-20T22:07:53.675389+0000","last_active":"2024-05-20T22:08:00.092522+0000","last_peered":"2024-05-20T22:08:00.092522+0000","last_clean":"2024-05-20T22:08:00.092522+0000","last_became_active":"2024-05-20T22:07:53.675027+0000","last_became_peered":"2024-05-20T22:07:53.675027+0000","last_unstale":"2024-05-20T22:08:00.092522+0000","last_undegraded":"2024-05-20T22:08:00.092522+0000","last_fullsized":"2024-05-20T22:08:00.092522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092234+0000","last_change":"2024-05-20T22:07:52.662236+0000","last_active":"2024-05-20T22:08:00.092234+0000","last_peered":"2024-05-20T22:08:00.092234+0000","last_clean":"2024-05-20T22:08:00.092234+0000","last_became_active":"2024-05-20T22:07:52.660352+0000","last_became_peered":"2024-05-20T22:07:52.660352+0000","last_unstale":"2024-05-20T22:08:00.092234+0000","last_undegraded":"2024-05-20T22:08:00.092234+0000","last_fullsized":"2024-05-20T22:08:00.092234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:41.518373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093269+0000","last_change":"2024-05-20T22:07:57.736022+0000","last_active":"2024-05-20T22:08:00.093269+0000","last_peered":"2024-05-20T22:08:00.093269+0000","last_clean":"2024-05-20T22:08:00.093269+0000","last_became_active":"2024-05-20T22:07:57.735570+0000","last_became_peered":"2024-05-20T22:07:57.735570+0000","last_unstale":"2024-05-20T22:08:00.093269+0000","last_undegraded":"2024-05-20T22:08:00.093269+0000","last_fullsized":"2024-05-20T22:08:00.093269+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:53.866913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092952+0000","last_change":"2024-05-20T22:07:53.662746+0000","last_active":"2024-05-20T22:08:00.092952+0000","last_peered":"2024-05-20T22:08:00.092952+0000","last_clean":"2024-05-20T22:08:00.092952+0000","last_became_active":"2024-05-20T22:07:53.662458+0000","last_became_peered":"2024-05-20T22:07:53.662458+0000","last_unstale":"2024-05-20T22:08:00.092952+0000","last_undegraded":"2024-05-20T22:08:00.092952+0000","last_fullsized":"2024-05-20T22:08:00.092952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:18.996690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705082+0000","last_change":"2024-05-20T22:07:52.656158+0000","last_active":"2024-05-20T22:07:58.705082+0000","last_peered":"2024-05-20T22:07:58.705082+0000","last_clean":"2024-05-20T22:07:58.705082+0000","last_became_active":"2024-05-20T22:07:52.655448+0000","last_became_peered":"2024-05-20T22:07:52.655448+0000","last_unstale":"2024-05-20T22:07:58.705082+0000","last_undegraded":"2024-05-20T22:07:58.705082+0000","last_fullsized":"2024-05-20T22:07:58.705082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093755+0000","last_change":"2024-05-20T22:07:57.734374+0000","last_active":"2024-05-20T22:08:00.093755+0000","last_peered":"2024-05-20T22:08:00.093755+0000","last_clean":"2024-05-20T22:08:00.093755+0000","last_became_active":"2024-05-20T22:07:57.733737+0000","last_became_peered":"2024-05-20T22:07:57.733737+0000","last_unstale":"2024-05-20T22:08:00.093755+0000","last_undegraded":"2024-05-20T22:08:00.093755+0000","last_fullsized":"2024-05-20T22:08:00.093755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581291+0000","last_change":"2024-05-20T22:07:53.670700+0000","last_active":"2024-05-20T22:07:59.581291+0000","last_peered":"2024-05-20T22:07:59.581291+0000","last_clean":"2024-05-20T22:07:59.581291+0000","last_became_active":"2024-05-20T22:07:53.670361+0000","last_became_peered":"2024-05-20T22:07:53.670361+0000","last_unstale":"2024-05-20T22:07:59.581291+0000","last_undegraded":"2024-05-20T22:07:59.581291+0000","last_fullsized":"2024-05-20T22:07:59.581291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706857+0000","last_change":"2024-05-20T22:07:52.658539+0000","last_active":"2024-05-20T22:07:58.706857+0000","last_peered":"2024-05-20T22:07:58.706857+0000","last_clean":"2024-05-20T22:07:58.706857+0000","last_became_active":"2024-05-20T22:07:52.658232+0000","last_became_peered":"2024-05-20T22:07:52.658232+0000","last_unstale":"2024-05-20T22:07:58.706857+0000","last_undegraded":"2024-05-20T22:07:58.706857+0000","last_fullsized":"2024-05-20T22:07:58.706857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092305+0000","last_change":"2024-05-20T22:07:57.726798+0000","last_active":"2024-05-20T22:08:00.092305+0000","last_peered":"2024-05-20T22:08:00.092305+0000","last_clean":"2024-05-20T22:08:00.092305+0000","last_became_active":"2024-05-20T22:07:57.726321+0000","last_became_peered":"2024-05-20T22:07:57.726321+0000","last_unstale":"2024-05-20T22:08:00.092305+0000","last_undegraded":"2024-05-20T22:08:00.092305+0000","last_fullsized":"2024-05-20T22:08:00.092305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:00:00.091936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093912+0000","last_change":"2024-05-20T22:07:53.673830+0000","last_active":"2024-05-20T22:08:00.093912+0000","last_peered":"2024-05-20T22:08:00.093912+0000","last_clean":"2024-05-20T22:08:00.093912+0000","last_became_active":"2024-05-20T22:07:53.673509+0000","last_became_peered":"2024-05-20T22:07:53.673509+0000","last_unstale":"2024-05-20T22:08:00.093912+0000","last_undegraded":"2024-05-20T22:08:00.093912+0000","last_fullsized":"2024-05-20T22:08:00.093912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092565+0000","last_change":"2024-05-20T22:07:52.661954+0000","last_active":"2024-05-20T22:08:00.092565+0000","last_peered":"2024-05-20T22:08:00.092565+0000","last_clean":"2024-05-20T22:08:00.092565+0000","last_became_active":"2024-05-20T22:07:52.661304+0000","last_became_peered":"2024-05-20T22:07:52.661304+0000","last_unstale":"2024-05-20T22:08:00.092565+0000","last_undegraded":"2024-05-20T22:08:00.092565+0000","last_fullsized":"2024-05-20T22:08:00.092565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.091968+0000","last_change":"2024-05-20T22:07:57.731395+0000","last_active":"2024-05-20T22:08:00.091968+0000","last_peered":"2024-05-20T22:08:00.091968+0000","last_clean":"2024-05-20T22:08:00.091968+0000","last_became_active":"2024-05-20T22:07:57.730839+0000","last_became_peered":"2024-05-20T22:07:57.730839+0000","last_unstale":"2024-05-20T22:08:00.091968+0000","last_undegraded":"2024-05-20T22:08:00.091968+0000","last_fullsized":"2024-05-20T22:08:00.091968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:44:34.494247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092791+0000","last_change":"2024-05-20T22:07:53.673439+0000","last_active":"2024-05-20T22:08:00.092791+0000","last_peered":"2024-05-20T22:08:00.092791+0000","last_clean":"2024-05-20T22:08:00.092791+0000","last_became_active":"2024-05-20T22:07:53.672746+0000","last_became_peered":"2024-05-20T22:07:53.672746+0000","last_unstale":"2024-05-20T22:08:00.092791+0000","last_undegraded":"2024-05-20T22:08:00.092791+0000","last_fullsized":"2024-05-20T22:08:00.092791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706775+0000","last_change":"2024-05-20T22:07:52.662817+0000","last_active":"2024-05-20T22:07:58.706775+0000","last_peered":"2024-05-20T22:07:58.706775+0000","last_clean":"2024-05-20T22:07:58.706775+0000","last_became_active":"2024-05-20T22:07:52.662463+0000","last_became_peered":"2024-05-20T22:07:52.662463+0000","last_unstale":"2024-05-20T22:07:58.706775+0000","last_undegraded":"2024-05-20T22:07:58.706775+0000","last_fullsized":"2024-05-20T22:07:58.706775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:07.396536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580916+0000","last_change":"2024-05-20T22:07:57.730858+0000","last_active":"2024-05-20T22:07:59.580916+0000","last_peered":"2024-05-20T22:07:59.580916+0000","last_clean":"2024-05-20T22:07:59.580916+0000","last_became_active":"2024-05-20T22:07:57.730370+0000","last_became_peered":"2024-05-20T22:07:57.730370+0000","last_unstale":"2024-05-20T22:07:59.580916+0000","last_undegraded":"2024-05-20T22:07:59.580916+0000","last_fullsized":"2024-05-20T22:07:59.580916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093484+0000","last_change":"2024-05-20T22:07:53.673073+0000","last_active":"2024-05-20T22:08:00.093484+0000","last_peered":"2024-05-20T22:08:00.093484+0000","last_clean":"2024-05-20T22:08:00.093484+0000","last_became_active":"2024-05-20T22:07:53.672744+0000","last_became_peered":"2024-05-20T22:07:53.672744+0000","last_unstale":"2024-05-20T22:08:00.093484+0000","last_undegraded":"2024-05-20T22:08:00.093484+0000","last_fullsized":"2024-05-20T22:08:00.093484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:49.823373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093340+0000","last_change":"2024-05-20T22:07:52.668045+0000","last_active":"2024-05-20T22:08:00.093340+0000","last_peered":"2024-05-20T22:08:00.093340+0000","last_clean":"2024-05-20T22:08:00.093340+0000","last_became_active":"2024-05-20T22:07:52.666855+0000","last_became_peered":"2024-05-20T22:07:52.666855+0000","last_unstale":"2024-05-20T22:08:00.093340+0000","last_undegraded":"2024-05-20T22:08:00.093340+0000","last_fullsized":"2024-05-20T22:08:00.093340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092837+0000","last_change":"2024-05-20T22:07:57.740710+0000","last_active":"2024-05-20T22:08:00.092837+0000","last_peered":"2024-05-20T22:08:00.092837+0000","last_clean":"2024-05-20T22:08:00.092837+0000","last_became_active":"2024-05-20T22:07:57.737102+0000","last_became_peered":"2024-05-20T22:07:57.737102+0000","last_unstale":"2024-05-20T22:08:00.092837+0000","last_undegraded":"2024-05-20T22:08:00.092837+0000","last_fullsized":"2024-05-20T22:08:00.092837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092905+0000","last_change":"2024-05-20T22:07:53.667120+0000","last_active":"2024-05-20T22:08:00.092905+0000","last_peered":"2024-05-20T22:08:00.092905+0000","last_clean":"2024-05-20T22:08:00.092905+0000","last_became_active":"2024-05-20T22:07:53.666809+0000","last_became_peered":"2024-05-20T22:07:53.666809+0000","last_unstale":"2024-05-20T22:08:00.092905+0000","last_undegraded":"2024-05-20T22:08:00.092905+0000","last_fullsized":"2024-05-20T22:08:00.092905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:08.402858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093723+0000","last_change":"2024-05-20T22:07:52.664415+0000","last_active":"2024-05-20T22:08:00.093723+0000","last_peered":"2024-05-20T22:08:00.093723+0000","last_clean":"2024-05-20T22:08:00.093723+0000","last_became_active":"2024-05-20T22:07:52.664085+0000","last_became_peered":"2024-05-20T22:07:52.664085+0000","last_unstale":"2024-05-20T22:08:00.093723+0000","last_undegraded":"2024-05-20T22:08:00.093723+0000","last_fullsized":"2024-05-20T22:08:00.093723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092731+0000","last_change":"2024-05-20T22:07:57.736468+0000","last_active":"2024-05-20T22:08:00.092731+0000","last_peered":"2024-05-20T22:08:00.092731+0000","last_clean":"2024-05-20T22:08:00.092731+0000","last_became_active":"2024-05-20T22:07:57.736003+0000","last_became_peered":"2024-05-20T22:07:57.736003+0000","last_unstale":"2024-05-20T22:08:00.092731+0000","last_undegraded":"2024-05-20T22:08:00.092731+0000","last_fullsized":"2024-05-20T22:08:00.092731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.091885+0000","last_change":"2024-05-20T22:07:53.674097+0000","last_active":"2024-05-20T22:08:00.091885+0000","last_peered":"2024-05-20T22:08:00.091885+0000","last_clean":"2024-05-20T22:08:00.091885+0000","last_became_active":"2024-05-20T22:07:53.673766+0000","last_became_peered":"2024-05-20T22:07:53.673766+0000","last_unstale":"2024-05-20T22:08:00.091885+0000","last_undegraded":"2024-05-20T22:08:00.091885+0000","last_fullsized":"2024-05-20T22:08:00.091885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706155+0000","last_change":"2024-05-20T22:07:52.662684+0000","last_active":"2024-05-20T22:07:58.706155+0000","last_peered":"2024-05-20T22:07:58.706155+0000","last_clean":"2024-05-20T22:07:58.706155+0000","last_became_active":"2024-05-20T22:07:52.662220+0000","last_became_peered":"2024-05-20T22:07:52.662220+0000","last_unstale":"2024-05-20T22:07:58.706155+0000","last_undegraded":"2024-05-20T22:07:58.706155+0000","last_fullsized":"2024-05-20T22:07:58.706155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093363+0000","last_change":"2024-05-20T22:07:57.740613+0000","last_active":"2024-05-20T22:08:00.093363+0000","last_peered":"2024-05-20T22:08:00.093363+0000","last_clean":"2024-05-20T22:08:00.093363+0000","last_became_active":"2024-05-20T22:07:57.740165+0000","last_became_peered":"2024-05-20T22:07:57.740165+0000","last_unstale":"2024-05-20T22:08:00.093363+0000","last_undegraded":"2024-05-20T22:08:00.093363+0000","last_fullsized":"2024-05-20T22:08:00.093363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:03:25.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580372+0000","last_change":"2024-05-20T22:07:53.656427+0000","last_active":"2024-05-20T22:07:59.580372+0000","last_peered":"2024-05-20T22:07:59.580372+0000","last_clean":"2024-05-20T22:07:59.580372+0000","last_became_active":"2024-05-20T22:07:53.656081+0000","last_became_peered":"2024-05-20T22:07:53.656081+0000","last_unstale":"2024-05-20T22:07:59.580372+0000","last_undegraded":"2024-05-20T22:07:59.580372+0000","last_fullsized":"2024-05-20T22:07:59.580372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:58:02.316451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705143+0000","last_change":"2024-05-20T22:07:52.656503+0000","last_active":"2024-05-20T22:07:58.705143+0000","last_peered":"2024-05-20T22:07:58.705143+0000","last_clean":"2024-05-20T22:07:58.705143+0000","last_became_active":"2024-05-20T22:07:52.654665+0000","last_became_peered":"2024-05-20T22:07:52.654665+0000","last_unstale":"2024-05-20T22:07:58.705143+0000","last_undegraded":"2024-05-20T22:07:58.705143+0000","last_fullsized":"2024-05-20T22:07:58.705143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581267+0000","last_change":"2024-05-20T22:07:57.731186+0000","last_active":"2024-05-20T22:07:59.581267+0000","last_peered":"2024-05-20T22:07:59.581267+0000","last_clean":"2024-05-20T22:07:59.581267+0000","last_became_active":"2024-05-20T22:07:57.730239+0000","last_became_peered":"2024-05-20T22:07:57.730239+0000","last_unstale":"2024-05-20T22:07:59.581267+0000","last_undegraded":"2024-05-20T22:07:59.581267+0000","last_fullsized":"2024-05-20T22:07:59.581267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:29.190558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093069+0000","last_change":"2024-05-20T22:07:53.667981+0000","last_active":"2024-05-20T22:08:00.093069+0000","last_peered":"2024-05-20T22:08:00.093069+0000","last_clean":"2024-05-20T22:08:00.093069+0000","last_became_active":"2024-05-20T22:07:53.667320+0000","last_became_peered":"2024-05-20T22:07:53.667320+0000","last_unstale":"2024-05-20T22:08:00.093069+0000","last_undegraded":"2024-05-20T22:08:00.093069+0000","last_fullsized":"2024-05-20T22:08:00.093069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093094+0000","last_change":"2024-05-20T22:07:52.657838+0000","last_active":"2024-05-20T22:08:00.093094+0000","last_peered":"2024-05-20T22:08:00.093094+0000","last_clean":"2024-05-20T22:08:00.093094+0000","last_became_active":"2024-05-20T22:07:52.657530+0000","last_became_peered":"2024-05-20T22:07:52.657530+0000","last_unstale":"2024-05-20T22:08:00.093094+0000","last_undegraded":"2024-05-20T22:08:00.093094+0000","last_fullsized":"2024-05-20T22:08:00.093094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706234+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:07:58.706234+0000","last_peered":"2024-05-20T22:07:58.706234+0000","last_clean":"2024-05-20T22:07:58.706234+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:07:58.706234+0000","last_undegraded":"2024-05-20T22:07:58.706234+0000","last_fullsized":"2024-05-20T22:07:58.706234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706485+0000","last_change":"2024-05-20T22:07:57.738452+0000","last_active":"2024-05-20T22:07:58.706485+0000","last_peered":"2024-05-20T22:07:58.706485+0000","last_clean":"2024-05-20T22:07:58.706485+0000","last_became_active":"2024-05-20T22:07:57.737967+0000","last_became_peered":"2024-05-20T22:07:57.737967+0000","last_unstale":"2024-05-20T22:07:58.706485+0000","last_undegraded":"2024-05-20T22:07:58.706485+0000","last_fullsized":"2024-05-20T22:07:58.706485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706425+0000","last_change":"2024-05-20T22:07:53.677833+0000","last_active":"2024-05-20T22:07:58.706425+0000","last_peered":"2024-05-20T22:07:58.706425+0000","last_clean":"2024-05-20T22:07:58.706425+0000","last_became_active":"2024-05-20T22:07:53.677513+0000","last_became_peered":"2024-05-20T22:07:53.677513+0000","last_unstale":"2024-05-20T22:07:58.706425+0000","last_undegraded":"2024-05-20T22:07:58.706425+0000","last_fullsized":"2024-05-20T22:07:58.706425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706468+0000","last_change":"2024-05-20T22:07:52.659290+0000","last_active":"2024-05-20T22:07:58.706468+0000","last_peered":"2024-05-20T22:07:58.706468+0000","last_clean":"2024-05-20T22:07:58.706468+0000","last_became_active":"2024-05-20T22:07:52.658939+0000","last_became_peered":"2024-05-20T22:07:52.658939+0000","last_unstale":"2024-05-20T22:07:58.706468+0000","last_undegraded":"2024-05-20T22:07:58.706468+0000","last_fullsized":"2024-05-20T22:07:58.706468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:30.832431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705627+0000","last_change":"2024-05-20T22:07:57.732095+0000","last_active":"2024-05-20T22:07:58.705627+0000","last_peered":"2024-05-20T22:07:58.705627+0000","last_clean":"2024-05-20T22:07:58.705627+0000","last_became_active":"2024-05-20T22:07:57.731276+0000","last_became_peered":"2024-05-20T22:07:57.731276+0000","last_unstale":"2024-05-20T22:07:58.705627+0000","last_undegraded":"2024-05-20T22:07:58.705627+0000","last_fullsized":"2024-05-20T22:07:58.705627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:30.698775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092825+0000","last_change":"2024-05-20T22:07:53.669593+0000","last_active":"2024-05-20T22:08:00.092825+0000","last_peered":"2024-05-20T22:08:00.092825+0000","last_clean":"2024-05-20T22:08:00.092825+0000","last_became_active":"2024-05-20T22:07:53.668982+0000","last_became_peered":"2024-05-20T22:07:53.668982+0000","last_unstale":"2024-05-20T22:08:00.092825+0000","last_undegraded":"2024-05-20T22:08:00.092825+0000","last_fullsized":"2024-05-20T22:08:00.092825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:26.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092921+0000","last_change":"2024-05-20T22:07:52.666183+0000","last_active":"2024-05-20T22:08:00.092921+0000","last_peered":"2024-05-20T22:08:00.092921+0000","last_clean":"2024-05-20T22:08:00.092921+0000","last_became_active":"2024-05-20T22:07:52.662670+0000","last_became_peered":"2024-05-20T22:07:52.662670+0000","last_unstale":"2024-05-20T22:08:00.092921+0000","last_undegraded":"2024-05-20T22:08:00.092921+0000","last_fullsized":"2024-05-20T22:08:00.092921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581046+0000","last_change":"2024-05-20T22:07:57.734443+0000","last_active":"2024-05-20T22:07:59.581046+0000","last_peered":"2024-05-20T22:07:59.581046+0000","last_clean":"2024-05-20T22:07:59.581046+0000","last_became_active":"2024-05-20T22:07:57.734021+0000","last_became_peered":"2024-05-20T22:07:57.734021+0000","last_unstale":"2024-05-20T22:07:59.581046+0000","last_undegraded":"2024-05-20T22:07:59.581046+0000","last_fullsized":"2024-05-20T22:07:59.581046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:30.960467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707194+0000","last_change":"2024-05-20T22:07:53.666815+0000","last_active":"2024-05-20T22:07:58.707194+0000","last_peered":"2024-05-20T22:07:58.707194+0000","last_clean":"2024-05-20T22:07:58.707194+0000","last_became_active":"2024-05-20T22:07:53.666480+0000","last_became_peered":"2024-05-20T22:07:53.666480+0000","last_unstale":"2024-05-20T22:07:58.707194+0000","last_undegraded":"2024-05-20T22:07:58.707194+0000","last_fullsized":"2024-05-20T22:07:58.707194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:49.823373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706498+0000","last_change":"2024-05-20T22:07:52.661745+0000","last_active":"2024-05-20T22:07:58.706498+0000","last_peered":"2024-05-20T22:07:58.706498+0000","last_clean":"2024-05-20T22:07:58.706498+0000","last_became_active":"2024-05-20T22:07:52.661304+0000","last_became_peered":"2024-05-20T22:07:52.661304+0000","last_unstale":"2024-05-20T22:07:58.706498+0000","last_undegraded":"2024-05-20T22:07:58.706498+0000","last_fullsized":"2024-05-20T22:07:58.706498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092249+0000","last_change":"2024-05-20T22:07:57.722732+0000","last_active":"2024-05-20T22:08:00.092249+0000","last_peered":"2024-05-20T22:08:00.092249+0000","last_clean":"2024-05-20T22:08:00.092249+0000","last_became_active":"2024-05-20T22:07:57.722287+0000","last_became_peered":"2024-05-20T22:07:57.722287+0000","last_unstale":"2024-05-20T22:08:00.092249+0000","last_undegraded":"2024-05-20T22:08:00.092249+0000","last_fullsized":"2024-05-20T22:08:00.092249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:48.169892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706818+0000","last_change":"2024-05-20T22:07:53.668146+0000","last_active":"2024-05-20T22:07:58.706818+0000","last_peered":"2024-05-20T22:07:58.706818+0000","last_clean":"2024-05-20T22:07:58.706818+0000","last_became_active":"2024-05-20T22:07:53.666969+0000","last_became_peered":"2024-05-20T22:07:53.666969+0000","last_unstale":"2024-05-20T22:07:58.706818+0000","last_undegraded":"2024-05-20T22:07:58.706818+0000","last_fullsized":"2024-05-20T22:07:58.706818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093917+0000","last_change":"2024-05-20T22:07:52.651502+0000","last_active":"2024-05-20T22:08:00.093917+0000","last_peered":"2024-05-20T22:08:00.093917+0000","last_clean":"2024-05-20T22:08:00.093917+0000","last_became_active":"2024-05-20T22:07:52.651166+0000","last_became_peered":"2024-05-20T22:07:52.651166+0000","last_unstale":"2024-05-20T22:08:00.093917+0000","last_undegraded":"2024-05-20T22:08:00.093917+0000","last_fullsized":"2024-05-20T22:08:00.093917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:41:50.914741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092214+0000","last_change":"2024-05-20T22:07:57.736957+0000","last_active":"2024-05-20T22:08:00.092214+0000","last_peered":"2024-05-20T22:08:00.092214+0000","last_clean":"2024-05-20T22:08:00.092214+0000","last_became_active":"2024-05-20T22:07:57.736512+0000","last_became_peered":"2024-05-20T22:07:57.736512+0000","last_unstale":"2024-05-20T22:08:00.092214+0000","last_undegraded":"2024-05-20T22:08:00.092214+0000","last_fullsized":"2024-05-20T22:08:00.092214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:46.233608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705356+0000","last_change":"2024-05-20T22:07:53.655287+0000","last_active":"2024-05-20T22:07:58.705356+0000","last_peered":"2024-05-20T22:07:58.705356+0000","last_clean":"2024-05-20T22:07:58.705356+0000","last_became_active":"2024-05-20T22:07:53.653689+0000","last_became_peered":"2024-05-20T22:07:53.653689+0000","last_unstale":"2024-05-20T22:07:58.705356+0000","last_undegraded":"2024-05-20T22:07:58.705356+0000","last_fullsized":"2024-05-20T22:07:58.705356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:26.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705945+0000","last_change":"2024-05-20T22:07:52.653862+0000","last_active":"2024-05-20T22:07:58.705945+0000","last_peered":"2024-05-20T22:07:58.705945+0000","last_clean":"2024-05-20T22:07:58.705945+0000","last_became_active":"2024-05-20T22:07:52.653527+0000","last_became_peered":"2024-05-20T22:07:52.653527+0000","last_unstale":"2024-05-20T22:07:58.705945+0000","last_undegraded":"2024-05-20T22:07:58.705945+0000","last_fullsized":"2024-05-20T22:07:58.705945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581093+0000","last_change":"2024-05-20T22:07:57.737164+0000","last_active":"2024-05-20T22:07:59.581093+0000","last_peered":"2024-05-20T22:07:59.581093+0000","last_clean":"2024-05-20T22:07:59.581093+0000","last_became_active":"2024-05-20T22:07:57.736733+0000","last_became_peered":"2024-05-20T22:07:57.736733+0000","last_unstale":"2024-05-20T22:07:59.581093+0000","last_undegraded":"2024-05-20T22:07:59.581093+0000","last_fullsized":"2024-05-20T22:07:59.581093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:53.866913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705408+0000","last_change":"2024-05-20T22:07:53.680648+0000","last_active":"2024-05-20T22:07:58.705408+0000","last_peered":"2024-05-20T22:07:58.705408+0000","last_clean":"2024-05-20T22:07:58.705408+0000","last_became_active":"2024-05-20T22:07:53.680316+0000","last_became_peered":"2024-05-20T22:07:53.680316+0000","last_unstale":"2024-05-20T22:07:58.705408+0000","last_undegraded":"2024-05-20T22:07:58.705408+0000","last_fullsized":"2024-05-20T22:07:58.705408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706034+0000","last_change":"2024-05-20T22:07:52.660545+0000","last_active":"2024-05-20T22:07:58.706034+0000","last_peered":"2024-05-20T22:07:58.706034+0000","last_clean":"2024-05-20T22:07:58.706034+0000","last_became_active":"2024-05-20T22:07:52.660191+0000","last_became_peered":"2024-05-20T22:07:52.660191+0000","last_unstale":"2024-05-20T22:07:58.706034+0000","last_undegraded":"2024-05-20T22:07:58.706034+0000","last_fullsized":"2024-05-20T22:07:58.706034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093896+0000","last_change":"2024-05-20T22:07:57.730561+0000","last_active":"2024-05-20T22:08:00.093896+0000","last_peered":"2024-05-20T22:08:00.093896+0000","last_clean":"2024-05-20T22:08:00.093896+0000","last_became_active":"2024-05-20T22:07:57.728162+0000","last_became_peered":"2024-05-20T22:07:57.728162+0000","last_unstale":"2024-05-20T22:08:00.093896+0000","last_undegraded":"2024-05-20T22:08:00.093896+0000","last_fullsized":"2024-05-20T22:08:00.093896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:18.032764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705624+0000","last_change":"2024-05-20T22:07:53.654238+0000","last_active":"2024-05-20T22:07:58.705624+0000","last_peered":"2024-05-20T22:07:58.705624+0000","last_clean":"2024-05-20T22:07:58.705624+0000","last_became_active":"2024-05-20T22:07:53.653629+0000","last_became_peered":"2024-05-20T22:07:53.653629+0000","last_unstale":"2024-05-20T22:07:58.705624+0000","last_undegraded":"2024-05-20T22:07:58.705624+0000","last_fullsized":"2024-05-20T22:07:58.705624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093667+0000","last_change":"2024-05-20T22:07:52.667566+0000","last_active":"2024-05-20T22:08:00.093667+0000","last_peered":"2024-05-20T22:08:00.093667+0000","last_clean":"2024-05-20T22:08:00.093667+0000","last_became_active":"2024-05-20T22:07:52.667214+0000","last_became_peered":"2024-05-20T22:07:52.667214+0000","last_unstale":"2024-05-20T22:08:00.093667+0000","last_undegraded":"2024-05-20T22:08:00.093667+0000","last_fullsized":"2024-05-20T22:08:00.093667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093690+0000","last_change":"2024-05-20T22:07:57.728380+0000","last_active":"2024-05-20T22:08:00.093690+0000","last_peered":"2024-05-20T22:08:00.093690+0000","last_clean":"2024-05-20T22:08:00.093690+0000","last_became_active":"2024-05-20T22:07:57.726788+0000","last_became_peered":"2024-05-20T22:07:57.726788+0000","last_unstale":"2024-05-20T22:08:00.093690+0000","last_undegraded":"2024-05-20T22:08:00.093690+0000","last_fullsized":"2024-05-20T22:08:00.093690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:46:04.066141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580651+0000","last_change":"2024-05-20T22:07:53.659126+0000","last_active":"2024-05-20T22:07:59.580651+0000","last_peered":"2024-05-20T22:07:59.580651+0000","last_clean":"2024-05-20T22:07:59.580651+0000","last_became_active":"2024-05-20T22:07:53.658751+0000","last_became_peered":"2024-05-20T22:07:53.658751+0000","last_unstale":"2024-05-20T22:07:59.580651+0000","last_undegraded":"2024-05-20T22:07:59.580651+0000","last_fullsized":"2024-05-20T22:07:59.580651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581140+0000","last_change":"2024-05-20T22:07:52.659607+0000","last_active":"2024-05-20T22:07:59.581140+0000","last_peered":"2024-05-20T22:07:59.581140+0000","last_clean":"2024-05-20T22:07:59.581140+0000","last_became_active":"2024-05-20T22:07:52.658939+0000","last_became_peered":"2024-05-20T22:07:52.658939+0000","last_unstale":"2024-05-20T22:07:59.581140+0000","last_undegraded":"2024-05-20T22:07:59.581140+0000","last_fullsized":"2024-05-20T22:07:59.581140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:12:51.889148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093500+0000","last_change":"2024-05-20T22:07:57.726495+0000","last_active":"2024-05-20T22:08:00.093500+0000","last_peered":"2024-05-20T22:08:00.093500+0000","last_clean":"2024-05-20T22:08:00.093500+0000","last_became_active":"2024-05-20T22:07:57.726012+0000","last_became_peered":"2024-05-20T22:07:57.726012+0000","last_unstale":"2024-05-20T22:08:00.093500+0000","last_undegraded":"2024-05-20T22:08:00.093500+0000","last_fullsized":"2024-05-20T22:08:00.093500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:44:34.494247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705411+0000","last_change":"2024-05-20T22:07:53.666422+0000","last_active":"2024-05-20T22:07:58.705411+0000","last_peered":"2024-05-20T22:07:58.705411+0000","last_clean":"2024-05-20T22:07:58.705411+0000","last_became_active":"2024-05-20T22:07:53.666109+0000","last_became_peered":"2024-05-20T22:07:53.666109+0000","last_unstale":"2024-05-20T22:07:58.705411+0000","last_undegraded":"2024-05-20T22:07:58.705411+0000","last_fullsized":"2024-05-20T22:07:58.705411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:03.694258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:08.748697+0000","last_change":"2024-05-20T22:07:52.661686+0000","last_active":"2024-05-20T22:08:08.748697+0000","last_peered":"2024-05-20T22:08:08.748697+0000","last_clean":"2024-05-20T22:08:08.748697+0000","last_became_active":"2024-05-20T22:07:52.661192+0000","last_became_peered":"2024-05-20T22:07:52.661192+0000","last_unstale":"2024-05-20T22:08:08.748697+0000","last_undegraded":"2024-05-20T22:08:08.748697+0000","last_fullsized":"2024-05-20T22:08:08.748697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093188+0000","last_change":"2024-05-20T22:07:57.733250+0000","last_active":"2024-05-20T22:08:00.093188+0000","last_peered":"2024-05-20T22:08:00.093188+0000","last_clean":"2024-05-20T22:08:00.093188+0000","last_became_active":"2024-05-20T22:07:57.732096+0000","last_became_peered":"2024-05-20T22:07:57.732096+0000","last_unstale":"2024-05-20T22:08:00.093188+0000","last_undegraded":"2024-05-20T22:08:00.093188+0000","last_fullsized":"2024-05-20T22:08:00.093188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:30.960467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093127+0000","last_change":"2024-05-20T22:07:53.664378+0000","last_active":"2024-05-20T22:08:00.093127+0000","last_peered":"2024-05-20T22:08:00.093127+0000","last_clean":"2024-05-20T22:08:00.093127+0000","last_became_active":"2024-05-20T22:07:53.664061+0000","last_became_peered":"2024-05-20T22:07:53.664061+0000","last_unstale":"2024-05-20T22:08:00.093127+0000","last_undegraded":"2024-05-20T22:08:00.093127+0000","last_fullsized":"2024-05-20T22:08:00.093127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:42.167135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092791+0000","last_change":"2024-05-20T22:07:52.665754+0000","last_active":"2024-05-20T22:08:00.092791+0000","last_peered":"2024-05-20T22:08:00.092791+0000","last_clean":"2024-05-20T22:08:00.092791+0000","last_became_active":"2024-05-20T22:07:52.663248+0000","last_became_peered":"2024-05-20T22:07:52.663248+0000","last_unstale":"2024-05-20T22:08:00.092791+0000","last_undegraded":"2024-05-20T22:08:00.092791+0000","last_fullsized":"2024-05-20T22:08:00.092791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092574+0000","last_change":"2024-05-20T22:07:57.732444+0000","last_active":"2024-05-20T22:08:00.092574+0000","last_peered":"2024-05-20T22:08:00.092574+0000","last_clean":"2024-05-20T22:08:00.092574+0000","last_became_active":"2024-05-20T22:07:57.731063+0000","last_became_peered":"2024-05-20T22:07:57.731063+0000","last_unstale":"2024-05-20T22:08:00.092574+0000","last_undegraded":"2024-05-20T22:08:00.092574+0000","last_fullsized":"2024-05-20T22:08:00.092574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:17.240623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581288+0000","last_change":"2024-05-20T22:07:53.669268+0000","last_active":"2024-05-20T22:07:59.581288+0000","last_peered":"2024-05-20T22:07:59.581288+0000","last_clean":"2024-05-20T22:07:59.581288+0000","last_became_active":"2024-05-20T22:07:53.666127+0000","last_became_peered":"2024-05-20T22:07:53.666127+0000","last_unstale":"2024-05-20T22:07:59.581288+0000","last_undegraded":"2024-05-20T22:07:59.581288+0000","last_fullsized":"2024-05-20T22:07:59.581288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093187+0000","last_change":"2024-05-20T22:07:52.660573+0000","last_active":"2024-05-20T22:08:00.093187+0000","last_peered":"2024-05-20T22:08:00.093187+0000","last_clean":"2024-05-20T22:08:00.093187+0000","last_became_active":"2024-05-20T22:07:52.660227+0000","last_became_peered":"2024-05-20T22:07:52.660227+0000","last_unstale":"2024-05-20T22:08:00.093187+0000","last_undegraded":"2024-05-20T22:08:00.093187+0000","last_fullsized":"2024-05-20T22:08:00.093187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092346+0000","last_change":"2024-05-20T22:07:57.726865+0000","last_active":"2024-05-20T22:08:00.092346+0000","last_peered":"2024-05-20T22:08:00.092346+0000","last_clean":"2024-05-20T22:08:00.092346+0000","last_became_active":"2024-05-20T22:07:57.726411+0000","last_became_peered":"2024-05-20T22:07:57.726411+0000","last_unstale":"2024-05-20T22:08:00.092346+0000","last_undegraded":"2024-05-20T22:08:00.092346+0000","last_fullsized":"2024-05-20T22:08:00.092346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:14.532804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093027+0000","last_change":"2024-05-20T22:07:53.674069+0000","last_active":"2024-05-20T22:08:00.093027+0000","last_peered":"2024-05-20T22:08:00.093027+0000","last_clean":"2024-05-20T22:08:00.093027+0000","last_became_active":"2024-05-20T22:07:53.673668+0000","last_became_peered":"2024-05-20T22:07:53.673668+0000","last_unstale":"2024-05-20T22:08:00.093027+0000","last_undegraded":"2024-05-20T22:08:00.093027+0000","last_fullsized":"2024-05-20T22:08:00.093027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:03.694258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706745+0000","last_change":"2024-05-20T22:07:52.659232+0000","last_active":"2024-05-20T22:07:58.706745+0000","last_peered":"2024-05-20T22:07:58.706745+0000","last_clean":"2024-05-20T22:07:58.706745+0000","last_became_active":"2024-05-20T22:07:52.657718+0000","last_became_peered":"2024-05-20T22:07:52.657718+0000","last_unstale":"2024-05-20T22:07:58.706745+0000","last_undegraded":"2024-05-20T22:07:58.706745+0000","last_fullsized":"2024-05-20T22:07:58.706745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092928+0000","last_change":"2024-05-20T22:07:57.730328+0000","last_active":"2024-05-20T22:08:00.092928+0000","last_peered":"2024-05-20T22:08:00.092928+0000","last_clean":"2024-05-20T22:08:00.092928+0000","last_became_active":"2024-05-20T22:07:57.729093+0000","last_became_peered":"2024-05-20T22:07:57.729093+0000","last_unstale":"2024-05-20T22:08:00.092928+0000","last_undegraded":"2024-05-20T22:08:00.092928+0000","last_fullsized":"2024-05-20T22:08:00.092928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:42:24.385164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581132+0000","last_change":"2024-05-20T22:07:57.738586+0000","last_active":"2024-05-20T22:07:59.581132+0000","last_peered":"2024-05-20T22:07:59.581132+0000","last_clean":"2024-05-20T22:07:59.581132+0000","last_became_active":"2024-05-20T22:07:57.736969+0000","last_became_peered":"2024-05-20T22:07:57.736969+0000","last_unstale":"2024-05-20T22:07:59.581132+0000","last_undegraded":"2024-05-20T22:07:59.581132+0000","last_fullsized":"2024-05-20T22:07:59.581132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:03:25.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092006+0000","last_change":"2024-05-20T22:07:53.677156+0000","last_active":"2024-05-20T22:08:00.092006+0000","last_peered":"2024-05-20T22:08:00.092006+0000","last_clean":"2024-05-20T22:08:00.092006+0000","last_became_active":"2024-05-20T22:07:53.676805+0000","last_became_peered":"2024-05-20T22:07:53.676805+0000","last_unstale":"2024-05-20T22:08:00.092006+0000","last_undegraded":"2024-05-20T22:08:00.092006+0000","last_fullsized":"2024-05-20T22:08:00.092006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092929+0000","last_change":"2024-05-20T22:07:52.659254+0000","last_active":"2024-05-20T22:08:00.092929+0000","last_peered":"2024-05-20T22:08:00.092929+0000","last_clean":"2024-05-20T22:08:00.092929+0000","last_became_active":"2024-05-20T22:07:52.658938+0000","last_became_peered":"2024-05-20T22:07:52.658938+0000","last_unstale":"2024-05-20T22:08:00.092929+0000","last_undegraded":"2024-05-20T22:08:00.092929+0000","last_fullsized":"2024-05-20T22:08:00.092929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706615+0000","last_change":"2024-05-20T22:07:57.731289+0000","last_active":"2024-05-20T22:07:58.706615+0000","last_peered":"2024-05-20T22:07:58.706615+0000","last_clean":"2024-05-20T22:07:58.706615+0000","last_became_active":"2024-05-20T22:07:57.730844+0000","last_became_peered":"2024-05-20T22:07:57.730844+0000","last_unstale":"2024-05-20T22:07:58.706615+0000","last_undegraded":"2024-05-20T22:07:58.706615+0000","last_fullsized":"2024-05-20T22:07:58.706615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092411+0000","last_change":"2024-05-20T22:07:53.676088+0000","last_active":"2024-05-20T22:08:00.092411+0000","last_peered":"2024-05-20T22:08:00.092411+0000","last_clean":"2024-05-20T22:08:00.092411+0000","last_became_active":"2024-05-20T22:07:53.675756+0000","last_became_peered":"2024-05-20T22:07:53.675756+0000","last_unstale":"2024-05-20T22:08:00.092411+0000","last_undegraded":"2024-05-20T22:08:00.092411+0000","last_fullsized":"2024-05-20T22:08:00.092411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093073+0000","last_change":"2024-05-20T22:07:52.660691+0000","last_active":"2024-05-20T22:08:00.093073+0000","last_peered":"2024-05-20T22:08:00.093073+0000","last_clean":"2024-05-20T22:08:00.093073+0000","last_became_active":"2024-05-20T22:07:52.660324+0000","last_became_peered":"2024-05-20T22:07:52.660324+0000","last_unstale":"2024-05-20T22:08:00.093073+0000","last_undegraded":"2024-05-20T22:08:00.093073+0000","last_fullsized":"2024-05-20T22:08:00.093073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:37.756571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706140+0000","last_change":"2024-05-20T22:07:52.660131+0000","last_active":"2024-05-20T22:07:58.706140+0000","last_peered":"2024-05-20T22:07:58.706140+0000","last_clean":"2024-05-20T22:07:58.706140+0000","last_became_active":"2024-05-20T22:07:52.659784+0000","last_became_peered":"2024-05-20T22:07:52.659784+0000","last_unstale":"2024-05-20T22:07:58.706140+0000","last_undegraded":"2024-05-20T22:07:58.706140+0000","last_fullsized":"2024-05-20T22:07:58.706140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581103+0000","last_change":"2024-05-20T22:07:53.659806+0000","last_active":"2024-05-20T22:07:59.581103+0000","last_peered":"2024-05-20T22:07:59.581103+0000","last_clean":"2024-05-20T22:07:59.581103+0000","last_became_active":"2024-05-20T22:07:53.659275+0000","last_became_peered":"2024-05-20T22:07:53.659275+0000","last_unstale":"2024-05-20T22:07:59.581103+0000","last_undegraded":"2024-05-20T22:07:59.581103+0000","last_fullsized":"2024-05-20T22:07:59.581103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706349+0000","last_change":"2024-05-20T22:07:57.732871+0000","last_active":"2024-05-20T22:07:58.706349+0000","last_peered":"2024-05-20T22:07:58.706349+0000","last_clean":"2024-05-20T22:07:58.706349+0000","last_became_active":"2024-05-20T22:07:57.732391+0000","last_became_peered":"2024-05-20T22:07:57.732391+0000","last_unstale":"2024-05-20T22:07:58.706349+0000","last_undegraded":"2024-05-20T22:07:58.706349+0000","last_fullsized":"2024-05-20T22:07:58.706349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:30.967143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092608+0000","last_change":"2024-05-20T22:07:52.665108+0000","last_active":"2024-05-20T22:08:00.092608+0000","last_peered":"2024-05-20T22:08:00.092608+0000","last_clean":"2024-05-20T22:08:00.092608+0000","last_became_active":"2024-05-20T22:07:52.664750+0000","last_became_peered":"2024-05-20T22:07:52.664750+0000","last_unstale":"2024-05-20T22:08:00.092608+0000","last_undegraded":"2024-05-20T22:08:00.092608+0000","last_fullsized":"2024-05-20T22:08:00.092608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:30.832431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092570+0000","last_change":"2024-05-20T22:07:53.671188+0000","last_active":"2024-05-20T22:08:00.092570+0000","last_peered":"2024-05-20T22:08:00.092570+0000","last_clean":"2024-05-20T22:08:00.092570+0000","last_became_active":"2024-05-20T22:07:53.670842+0000","last_became_peered":"2024-05-20T22:07:53.670842+0000","last_unstale":"2024-05-20T22:08:00.092570+0000","last_undegraded":"2024-05-20T22:08:00.092570+0000","last_fullsized":"2024-05-20T22:08:00.092570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:34.677214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093617+0000","last_change":"2024-05-20T22:07:57.725024+0000","last_active":"2024-05-20T22:08:00.093617+0000","last_peered":"2024-05-20T22:08:00.093617+0000","last_clean":"2024-05-20T22:08:00.093617+0000","last_became_active":"2024-05-20T22:07:57.724576+0000","last_became_peered":"2024-05-20T22:07:57.724576+0000","last_unstale":"2024-05-20T22:08:00.093617+0000","last_undegraded":"2024-05-20T22:08:00.093617+0000","last_fullsized":"2024-05-20T22:08:00.093617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:46:27.003055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705082+0000","last_change":"2024-05-20T22:07:53.680796+0000","last_active":"2024-05-20T22:07:58.705082+0000","last_peered":"2024-05-20T22:07:58.705082+0000","last_clean":"2024-05-20T22:07:58.705082+0000","last_became_active":"2024-05-20T22:07:53.680460+0000","last_became_peered":"2024-05-20T22:07:53.680460+0000","last_unstale":"2024-05-20T22:07:58.705082+0000","last_undegraded":"2024-05-20T22:07:58.705082+0000","last_fullsized":"2024-05-20T22:07:58.705082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705955+0000","last_change":"2024-05-20T22:07:53.664643+0000","last_active":"2024-05-20T22:07:58.705955+0000","last_peered":"2024-05-20T22:07:58.705955+0000","last_clean":"2024-05-20T22:07:58.705955+0000","last_became_active":"2024-05-20T22:07:53.664307+0000","last_became_peered":"2024-05-20T22:07:53.664307+0000","last_unstale":"2024-05-20T22:07:58.705955+0000","last_undegraded":"2024-05-20T22:07:58.705955+0000","last_fullsized":"2024-05-20T22:07:58.705955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:08.402858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093600+0000","last_change":"2024-05-20T22:07:53.669161+0000","last_active":"2024-05-20T22:08:00.093600+0000","last_peered":"2024-05-20T22:08:00.093600+0000","last_clean":"2024-05-20T22:08:00.093600+0000","last_became_active":"2024-05-20T22:07:53.668720+0000","last_became_peered":"2024-05-20T22:07:53.668720+0000","last_unstale":"2024-05-20T22:08:00.093600+0000","last_undegraded":"2024-05-20T22:08:00.093600+0000","last_fullsized":"2024-05-20T22:08:00.093600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:42.757306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580972+0000","last_change":"2024-05-20T22:07:53.667728+0000","last_active":"2024-05-20T22:07:59.580972+0000","last_peered":"2024-05-20T22:07:59.580972+0000","last_clean":"2024-05-20T22:07:59.580972+0000","last_became_active":"2024-05-20T22:07:53.665722+0000","last_became_peered":"2024-05-20T22:07:53.665722+0000","last_unstale":"2024-05-20T22:07:59.580972+0000","last_undegraded":"2024-05-20T22:07:59.580972+0000","last_fullsized":"2024-05-20T22:07:59.580972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:48.605408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093771+0000","last_change":"2024-05-20T22:07:53.665673+0000","last_active":"2024-05-20T22:08:00.093771+0000","last_peered":"2024-05-20T22:08:00.093771+0000","last_clean":"2024-05-20T22:08:00.093771+0000","last_became_active":"2024-05-20T22:07:53.665322+0000","last_became_peered":"2024-05-20T22:07:53.665322+0000","last_unstale":"2024-05-20T22:08:00.093771+0000","last_undegraded":"2024-05-20T22:08:00.093771+0000","last_fullsized":"2024-05-20T22:08:00.093771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092400+0000","last_change":"2024-05-20T22:07:53.673539+0000","last_active":"2024-05-20T22:08:00.092400+0000","last_peered":"2024-05-20T22:08:00.092400+0000","last_clean":"2024-05-20T22:08:00.092400+0000","last_became_active":"2024-05-20T22:07:53.673213+0000","last_became_peered":"2024-05-20T22:07:53.673213+0000","last_unstale":"2024-05-20T22:08:00.092400+0000","last_undegraded":"2024-05-20T22:08:00.092400+0000","last_fullsized":"2024-05-20T22:08:00.092400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706633+0000","last_change":"2024-05-20T22:07:53.676094+0000","last_active":"2024-05-20T22:07:58.706633+0000","last_peered":"2024-05-20T22:07:58.706633+0000","last_clean":"2024-05-20T22:07:58.706633+0000","last_became_active":"2024-05-20T22:07:53.675752+0000","last_became_peered":"2024-05-20T22:07:53.675752+0000","last_unstale":"2024-05-20T22:07:58.706633+0000","last_undegraded":"2024-05-20T22:07:58.706633+0000","last_fullsized":"2024-05-20T22:07:58.706633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093799+0000","last_change":"2024-05-20T22:07:53.665953+0000","last_active":"2024-05-20T22:08:00.093799+0000","last_peered":"2024-05-20T22:08:00.093799+0000","last_clean":"2024-05-20T22:08:00.093799+0000","last_became_active":"2024-05-20T22:07:53.665631+0000","last_became_peered":"2024-05-20T22:07:53.665631+0000","last_unstale":"2024-05-20T22:08:00.093799+0000","last_undegraded":"2024-05-20T22:08:00.093799+0000","last_fullsized":"2024-05-20T22:08:00.093799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092802+0000","last_change":"2024-05-20T22:07:53.669986+0000","last_active":"2024-05-20T22:08:00.092802+0000","last_peered":"2024-05-20T22:08:00.092802+0000","last_clean":"2024-05-20T22:08:00.092802+0000","last_became_active":"2024-05-20T22:07:53.669565+0000","last_became_peered":"2024-05-20T22:07:53.669565+0000","last_unstale":"2024-05-20T22:08:00.092802+0000","last_undegraded":"2024-05-20T22:08:00.092802+0000","last_fullsized":"2024-05-20T22:08:00.092802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092993+0000","last_change":"2024-05-20T22:07:53.667290+0000","last_active":"2024-05-20T22:08:00.092993+0000","last_peered":"2024-05-20T22:08:00.092993+0000","last_clean":"2024-05-20T22:08:00.092993+0000","last_became_active":"2024-05-20T22:07:53.666953+0000","last_became_peered":"2024-05-20T22:07:53.666953+0000","last_unstale":"2024-05-20T22:08:00.092993+0000","last_undegraded":"2024-05-20T22:08:00.092993+0000","last_fullsized":"2024-05-20T22:08:00.092993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:48.605408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093536+0000","last_change":"2024-05-20T22:07:53.673713+0000","last_active":"2024-05-20T22:08:00.093536+0000","last_peered":"2024-05-20T22:08:00.093536+0000","last_clean":"2024-05-20T22:08:00.093536+0000","last_became_active":"2024-05-20T22:07:53.673419+0000","last_became_peered":"2024-05-20T22:07:53.673419+0000","last_unstale":"2024-05-20T22:08:00.093536+0000","last_undegraded":"2024-05-20T22:08:00.093536+0000","last_fullsized":"2024-05-20T22:08:00.093536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580805+0000","last_change":"2024-05-20T22:07:53.657177+0000","last_active":"2024-05-20T22:07:59.580805+0000","last_peered":"2024-05-20T22:07:59.580805+0000","last_clean":"2024-05-20T22:07:59.580805+0000","last_became_active":"2024-05-20T22:07:53.656847+0000","last_became_peered":"2024-05-20T22:07:53.656847+0000","last_unstale":"2024-05-20T22:07:59.580805+0000","last_undegraded":"2024-05-20T22:07:59.580805+0000","last_fullsized":"2024-05-20T22:07:59.580805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706361+0000","last_change":"2024-05-20T22:07:53.675562+0000","last_active":"2024-05-20T22:07:58.706361+0000","last_peered":"2024-05-20T22:07:58.706361+0000","last_clean":"2024-05-20T22:07:58.706361+0000","last_became_active":"2024-05-20T22:07:53.673333+0000","last_became_peered":"2024-05-20T22:07:53.673333+0000","last_unstale":"2024-05-20T22:07:58.706361+0000","last_undegraded":"2024-05-20T22:07:58.706361+0000","last_fullsized":"2024-05-20T22:07:58.706361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706689+0000","last_change":"2024-05-20T22:07:53.664701+0000","last_active":"2024-05-20T22:07:58.706689+0000","last_peered":"2024-05-20T22:07:58.706689+0000","last_clean":"2024-05-20T22:07:58.706689+0000","last_became_active":"2024-05-20T22:07:53.663423+0000","last_became_peered":"2024-05-20T22:07:53.663423+0000","last_unstale":"2024-05-20T22:07:58.706689+0000","last_undegraded":"2024-05-20T22:07:58.706689+0000","last_fullsized":"2024-05-20T22:07:58.706689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706441+0000","last_change":"2024-05-20T22:07:53.677915+0000","last_active":"2024-05-20T22:07:58.706441+0000","last_peered":"2024-05-20T22:07:58.706441+0000","last_clean":"2024-05-20T22:07:58.706441+0000","last_became_active":"2024-05-20T22:07:53.677479+0000","last_became_peered":"2024-05-20T22:07:53.677479+0000","last_unstale":"2024-05-20T22:07:58.706441+0000","last_undegraded":"2024-05-20T22:07:58.706441+0000","last_fullsized":"2024-05-20T22:07:58.706441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:42.167135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092947+0000","last_change":"2024-05-20T22:07:53.674336+0000","last_active":"2024-05-20T22:08:00.092947+0000","last_peered":"2024-05-20T22:08:00.092947+0000","last_clean":"2024-05-20T22:08:00.092947+0000","last_became_active":"2024-05-20T22:07:53.673643+0000","last_became_peered":"2024-05-20T22:07:53.673643+0000","last_unstale":"2024-05-20T22:08:00.092947+0000","last_undegraded":"2024-05-20T22:08:00.092947+0000","last_fullsized":"2024-05-20T22:08:00.092947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:26.916927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092025+0000","last_change":"2024-05-20T22:07:53.680694+0000","last_active":"2024-05-20T22:08:00.092025+0000","last_peered":"2024-05-20T22:08:00.092025+0000","last_clean":"2024-05-20T22:08:00.092025+0000","last_became_active":"2024-05-20T22:07:53.680338+0000","last_became_peered":"2024-05-20T22:07:53.680338+0000","last_unstale":"2024-05-20T22:08:00.092025+0000","last_undegraded":"2024-05-20T22:08:00.092025+0000","last_fullsized":"2024-05-20T22:08:00.092025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092124+0000","last_change":"2024-05-20T22:07:53.668909+0000","last_active":"2024-05-20T22:08:00.092124+0000","last_peered":"2024-05-20T22:08:00.092124+0000","last_clean":"2024-05-20T22:08:00.092124+0000","last_became_active":"2024-05-20T22:07:53.668540+0000","last_became_peered":"2024-05-20T22:07:53.668540+0000","last_unstale":"2024-05-20T22:08:00.092124+0000","last_undegraded":"2024-05-20T22:08:00.092124+0000","last_fullsized":"2024-05-20T22:08:00.092124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580610+0000","last_change":"2024-05-20T22:07:53.671760+0000","last_active":"2024-05-20T22:07:59.580610+0000","last_peered":"2024-05-20T22:07:59.580610+0000","last_clean":"2024-05-20T22:07:59.580610+0000","last_became_active":"2024-05-20T22:07:53.671454+0000","last_became_peered":"2024-05-20T22:07:53.671454+0000","last_unstale":"2024-05-20T22:07:59.580610+0000","last_undegraded":"2024-05-20T22:07:59.580610+0000","last_fullsized":"2024-05-20T22:07:59.580610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:56.576260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092384+0000","last_change":"2024-05-20T22:07:53.678862+0000","last_active":"2024-05-20T22:08:00.092384+0000","last_peered":"2024-05-20T22:08:00.092384+0000","last_clean":"2024-05-20T22:08:00.092384+0000","last_became_active":"2024-05-20T22:07:53.678323+0000","last_became_peered":"2024-05-20T22:07:53.678323+0000","last_unstale":"2024-05-20T22:08:00.092384+0000","last_undegraded":"2024-05-20T22:08:00.092384+0000","last_fullsized":"2024-05-20T22:08:00.092384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093008+0000","last_change":"2024-05-20T22:07:53.677127+0000","last_active":"2024-05-20T22:08:00.093008+0000","last_peered":"2024-05-20T22:08:00.093008+0000","last_clean":"2024-05-20T22:08:00.093008+0000","last_became_active":"2024-05-20T22:07:53.676804+0000","last_became_peered":"2024-05-20T22:07:53.676804+0000","last_unstale":"2024-05-20T22:08:00.093008+0000","last_undegraded":"2024-05-20T22:08:00.093008+0000","last_fullsized":"2024-05-20T22:08:00.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:42.524695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706507+0000","last_change":"2024-05-20T22:07:53.677487+0000","last_active":"2024-05-20T22:07:58.706507+0000","last_peered":"2024-05-20T22:07:58.706507+0000","last_clean":"2024-05-20T22:07:58.706507+0000","last_became_active":"2024-05-20T22:07:53.677160+0000","last_became_peered":"2024-05-20T22:07:53.677160+0000","last_unstale":"2024-05-20T22:07:58.706507+0000","last_undegraded":"2024-05-20T22:07:58.706507+0000","last_fullsized":"2024-05-20T22:07:58.706507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706545+0000","last_change":"2024-05-20T22:07:53.675130+0000","last_active":"2024-05-20T22:07:58.706545+0000","last_peered":"2024-05-20T22:07:58.706545+0000","last_clean":"2024-05-20T22:07:58.706545+0000","last_became_active":"2024-05-20T22:07:53.674566+0000","last_became_peered":"2024-05-20T22:07:53.674566+0000","last_unstale":"2024-05-20T22:07:58.706545+0000","last_undegraded":"2024-05-20T22:07:58.706545+0000","last_fullsized":"2024-05-20T22:07:58.706545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707188+0000","last_change":"2024-05-20T22:07:53.668737+0000","last_active":"2024-05-20T22:07:58.707188+0000","last_peered":"2024-05-20T22:07:58.707188+0000","last_clean":"2024-05-20T22:07:58.707188+0000","last_became_active":"2024-05-20T22:07:53.668436+0000","last_became_peered":"2024-05-20T22:07:53.668436+0000","last_unstale":"2024-05-20T22:07:58.707188+0000","last_undegraded":"2024-05-20T22:07:58.707188+0000","last_fullsized":"2024-05-20T22:07:58.707188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:26.916927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092105+0000","last_change":"2024-05-20T22:07:53.667651+0000","last_active":"2024-05-20T22:08:00.092105+0000","last_peered":"2024-05-20T22:08:00.092105+0000","last_clean":"2024-05-20T22:08:00.092105+0000","last_became_active":"2024-05-20T22:07:53.667290+0000","last_became_peered":"2024-05-20T22:07:53.667290+0000","last_unstale":"2024-05-20T22:08:00.092105+0000","last_undegraded":"2024-05-20T22:08:00.092105+0000","last_fullsized":"2024-05-20T22:08:00.092105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705225+0000","last_change":"2024-05-20T22:07:53.655295+0000","last_active":"2024-05-20T22:07:58.705225+0000","last_peered":"2024-05-20T22:07:58.705225+0000","last_clean":"2024-05-20T22:07:58.705225+0000","last_became_active":"2024-05-20T22:07:53.654945+0000","last_became_peered":"2024-05-20T22:07:53.654945+0000","last_unstale":"2024-05-20T22:07:58.705225+0000","last_undegraded":"2024-05-20T22:07:58.705225+0000","last_fullsized":"2024-05-20T22:07:58.705225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:42.757306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093460+0000","last_change":"2024-05-20T22:07:53.681737+0000","last_active":"2024-05-20T22:08:00.093460+0000","last_peered":"2024-05-20T22:08:00.093460+0000","last_clean":"2024-05-20T22:08:00.093460+0000","last_became_active":"2024-05-20T22:07:53.681436+0000","last_became_peered":"2024-05-20T22:07:53.681436+0000","last_unstale":"2024-05-20T22:08:00.093460+0000","last_undegraded":"2024-05-20T22:08:00.093460+0000","last_fullsized":"2024-05-20T22:08:00.093460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705391+0000","last_change":"2024-05-20T22:07:53.662939+0000","last_active":"2024-05-20T22:07:58.705391+0000","last_peered":"2024-05-20T22:07:58.705391+0000","last_clean":"2024-05-20T22:07:58.705391+0000","last_became_active":"2024-05-20T22:07:53.662545+0000","last_became_peered":"2024-05-20T22:07:53.662545+0000","last_unstale":"2024-05-20T22:07:58.705391+0000","last_undegraded":"2024-05-20T22:07:58.705391+0000","last_fullsized":"2024-05-20T22:07:58.705391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092568+0000","last_change":"2024-05-20T22:07:53.677132+0000","last_active":"2024-05-20T22:08:00.092568+0000","last_peered":"2024-05-20T22:08:00.092568+0000","last_clean":"2024-05-20T22:08:00.092568+0000","last_became_active":"2024-05-20T22:07:53.676804+0000","last_became_peered":"2024-05-20T22:07:53.676804+0000","last_unstale":"2024-05-20T22:08:00.092568+0000","last_undegraded":"2024-05-20T22:08:00.092568+0000","last_fullsized":"2024-05-20T22:08:00.092568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580901+0000","last_change":"2024-05-20T22:07:53.669468+0000","last_active":"2024-05-20T22:07:59.580901+0000","last_peered":"2024-05-20T22:07:59.580901+0000","last_clean":"2024-05-20T22:07:59.580901+0000","last_became_active":"2024-05-20T22:07:53.669074+0000","last_became_peered":"2024-05-20T22:07:53.669074+0000","last_unstale":"2024-05-20T22:07:59.580901+0000","last_undegraded":"2024-05-20T22:07:59.580901+0000","last_fullsized":"2024-05-20T22:07:59.580901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:26.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092451+0000","last_change":"2024-05-20T22:07:53.680851+0000","last_active":"2024-05-20T22:08:00.092451+0000","last_peered":"2024-05-20T22:08:00.092451+0000","last_clean":"2024-05-20T22:08:00.092451+0000","last_became_active":"2024-05-20T22:07:53.680468+0000","last_became_peered":"2024-05-20T22:07:53.680468+0000","last_unstale":"2024-05-20T22:08:00.092451+0000","last_undegraded":"2024-05-20T22:08:00.092451+0000","last_fullsized":"2024-05-20T22:08:00.092451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707096+0000","last_change":"2024-05-20T22:07:53.675836+0000","last_active":"2024-05-20T22:07:58.707096+0000","last_peered":"2024-05-20T22:07:58.707096+0000","last_clean":"2024-05-20T22:07:58.707096+0000","last_became_active":"2024-05-20T22:07:53.675544+0000","last_became_peered":"2024-05-20T22:07:53.675544+0000","last_unstale":"2024-05-20T22:07:58.707096+0000","last_undegraded":"2024-05-20T22:07:58.707096+0000","last_fullsized":"2024-05-20T22:07:58.707096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:42.044571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:08:13.194 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T22:08:13.195 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-20T22:08:13.195 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-20T22:08:13.195 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-20T22:08:13.195 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-20T22:08:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.372+0000 7f892ae51640 1 Processor -- start 2024-05-20T22:08:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.372+0000 7f892ae51640 1 -- start start 2024-05-20T22:08:13.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.373+0000 7f892ae51640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f89240b8720 0x7f892415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.373+0000 7f892ae51640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892406b870 con 0x7f89240b3c90 2024-05-20T22:08:13.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.373+0000 7f892ae51640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892406bb50 con 0x7f89240d3390 2024-05-20T22:08:13.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f892ae51640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f892406be30 con 0x7f89240b8720 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f8923fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f89240b3c90 0x7f89240b8010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46304/0 (socket says 172.21.15.55:46304) 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f8923fff640 1 -- 172.21.15.55:0/687680085 learned_addr learned my addr 172.21.15.55:0/687680085 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89293ee640 1 --2- 172.21.15.55:0/687680085 >> v2:172.21.15.138:3300/0 conn(0x7f89240b8720 0x7f892415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2862510054 0 0) 0x7f892406b870 con 0x7f89240b3c90 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8914003520 con 0x7f89240b3c90 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3473725184 0 0) 0x7f8914003520 con 0x7f89240b3c90 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 >> v2:172.21.15.138:3300/0 conn(0x7f89240b8720 msgr2=0x7f892415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 --2- 172.21.15.55:0/687680085 >> v2:172.21.15.138:3300/0 conn(0x7f89240b8720 0x7f892415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 >> v1:172.21.15.138:6789/0 conn(0x7f89240d3390 legacy=0x7f892406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892406c110 con 0x7f89240b3c90 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f8910001670 con 0x7f89240b3c90 2024-05-20T22:08:13.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 >> v1:172.21.15.55:6789/0 conn(0x7f89240b3c90 legacy=0x7f89240b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89140039b0 0x7f8914003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.374+0000 7f89237fe640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8914004800 0x7f8914004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89237fe640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8914005250 0x7f891400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f892406be30 con 0x7f8914005250 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f891400c1e0 con 0x7f8914004800 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f891400c3e0 con 0x7f89140039b0 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89293ee640 1 --2- 172.21.15.55:0/687680085 >> v2:172.21.15.138:3300/0 conn(0x7f89240b8720 0x7f892415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f8923fff640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8914004800 0x7f8914004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f8928bed640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89140039b0 0x7f8914003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89293ee640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8914005250 0x7f891400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89293ee640 1 -- 172.21.15.55:0/687680085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89140039b0 msgr2=0x7f8914003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89293ee640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89140039b0 0x7f8914003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89293ee640 1 -- 172.21.15.55:0/687680085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8914004800 msgr2=0x7f8914004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:13.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89293ee640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8914004800 0x7f8914004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.375+0000 7f89293ee640 1 -- 172.21.15.55:0/687680085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8914003520 con 0x7f8914005250 2024-05-20T22:08:13.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.376+0000 7f89293ee640 1 -- 172.21.15.55:0/687680085 reap_dead start 2024-05-20T22:08:13.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.376+0000 7f89293ee640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8914005250 0x7f891400b9c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f891c00e350 tx=0x7f8914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d109b21063072a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:13.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.376+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c009dc0 con 0x7f8914005250 2024-05-20T22:08:13.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.376+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891c003bb0 con 0x7f8914005250 2024-05-20T22:08:13.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.376+0000 7f89237fe640 1 -- 172.21.15.55:0/687680085 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c004020 con 0x7f8914005250 2024-05-20T22:08:13.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.377+0000 7f892ae51640 1 -- 172.21.15.55:0/687680085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8914005250 msgr2=0x7f891400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.377+0000 7f892ae51640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8914005250 0x7f891400b9c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f891c00e350 tx=0x7f8914005830 comp rx=0 tx=0).stop 2024-05-20T22:08:13.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.377+0000 7f892ae51640 1 -- 172.21.15.55:0/687680085 shutdown_connections 2024-05-20T22:08:13.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.377+0000 7f892ae51640 1 --2- 172.21.15.55:0/687680085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8914005250 0x7f891400b9c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.377+0000 7f892ae51640 1 -- 172.21.15.55:0/687680085 >> 172.21.15.55:0/687680085 conn(0x7f89240b65f0 msgr2=0x7f89240b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:13.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.377+0000 7f892ae51640 1 -- 172.21.15.55:0/687680085 shutdown_connections 2024-05-20T22:08:13.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.377+0000 7f892ae51640 1 -- 172.21.15.55:0/687680085 wait complete. 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.379+0000 7f892ae51640 1 Processor -- start 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.379+0000 7f892ae51640 1 -- start start 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f892ae51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8914005250 0x7f89240cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f892ae51640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89240ce7c0 0x7f89240cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f892ae51640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89240cebd0 0x7f89240ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f892ae51640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f89240b8b30 con 0x7f89240cebd0 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f892ae51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f892406b500 con 0x7f8914005250 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f892ae51640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f89240b8cb0 con 0x7f89240ce7c0 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f89293ee640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89240cebd0 0x7f89240ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f89293ee640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89240cebd0 0x7f89240ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57910/0 (socket says 172.21.15.55:57910) 2024-05-20T22:08:13.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f89293ee640 1 -- 172.21.15.55:0/2943477772 learned_addr learned my addr 172.21.15.55:0/2943477772 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f8928bed640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8914005250 0x7f89240cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.380+0000 7f8923fff640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89240ce7c0 0x7f89240cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f89293ee640 1 -- 172.21.15.55:0/2943477772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89240ce7c0 msgr2=0x7f89240cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f89293ee640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89240ce7c0 0x7f89240cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f89293ee640 1 -- 172.21.15.55:0/2943477772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8914005250 msgr2=0x7f89240cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f89293ee640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8914005250 0x7f89240cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f89293ee640 1 -- 172.21.15.55:0/2943477772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89241f8f20 con 0x7f89240cebd0 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f89293ee640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89240cebd0 0x7f89240ccdb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f891c00e470 tx=0x7f891c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f89217fa640 1 -- 172.21.15.55:0/2943477772 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c02e430 con 0x7f89240cebd0 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f89217fa640 1 -- 172.21.15.55:0/2943477772 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891c02e5f0 con 0x7f89240cebd0 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f892ae51640 1 -- 172.21.15.55:0/2943477772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89241f9200 con 0x7f89240cebd0 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f89217fa640 1 -- 172.21.15.55:0/2943477772 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c02e9e0 con 0x7f89240cebd0 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.381+0000 7f892ae51640 1 -- 172.21.15.55:0/2943477772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89241f97c0 con 0x7f89240cebd0 2024-05-20T22:08:13.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.382+0000 7f892ae51640 1 -- 172.21.15.55:0/2943477772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89241f8c90 con 0x7f89240cebd0 2024-05-20T22:08:13.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.383+0000 7f89217fa640 1 -- 172.21.15.55:0/2943477772 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f891c040070 con 0x7f89240cebd0 2024-05-20T22:08:13.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.383+0000 7f89217fa640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f890806be30 0x7f890806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.383+0000 7f89217fa640 1 -- 172.21.15.55:0/2943477772 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f891c0abf50 con 0x7f89240cebd0 2024-05-20T22:08:13.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.383+0000 7f8928bed640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f890806be30 0x7f890806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.384+0000 7f8928bed640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f890806be30 0x7f890806e340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8918000c00 tx=0x7f8918009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:13.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.392+0000 7f89217fa640 1 -- 172.21.15.55:0/2943477772 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f891c048060 con 0x7f89240cebd0 2024-05-20T22:08:13.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.625+0000 7f892ae51640 1 -- 172.21.15.55:0/2943477772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8924069090 con 0x7f89240cebd0 2024-05-20T22:08:13.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.628+0000 7f89217fa640 1 -- 172.21.15.55:0/2943477772 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172935 (secure 0 0 0) 0x7f891c032030 con 0x7f89240cebd0 2024-05-20T22:08:13.631 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:13.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 -- 172.21.15.55:0/2943477772 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f890806be30 msgr2=0x7f890806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f890806be30 0x7f890806e340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8918000c00 tx=0x7f8918009040 comp rx=0 tx=0).stop 2024-05-20T22:08:13.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 -- 172.21.15.55:0/2943477772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89240cebd0 msgr2=0x7f89240ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89240cebd0 0x7f89240ccdb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f891c00e470 tx=0x7f891c009750 comp rx=0 tx=0).stop 2024-05-20T22:08:13.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 -- 172.21.15.55:0/2943477772 shutdown_connections 2024-05-20T22:08:13.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f890806be30 0x7f890806e340 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89240cebd0 0x7f89240ccdb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89240ce7c0 0x7f89240cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 --2- 172.21.15.55:0/2943477772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8914005250 0x7f89240cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 -- 172.21.15.55:0/2943477772 >> 172.21.15.55:0/2943477772 conn(0x7f89240b65f0 msgr2=0x7f89240cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:13.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 -- 172.21.15.55:0/2943477772 shutdown_connections 2024-05-20T22:08:13.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.631+0000 7f892ae51640 1 -- 172.21.15.55:0/2943477772 wait complete. 2024-05-20T22:08:13.643 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":5,"flags":0,"active_gid":4098,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":115760224},{"type":"v1","addr":"172.21.15.55:6833","nonce":115760224}]},"active_addr":"172.21.15.55:6833/115760224","active_change":"2024-05-20T22:07:32.318113+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":3836561843}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":2453376285}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":110335641}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":4266526929}]}]} 2024-05-20T22:08:13.647 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-20T22:08:13.647 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-20T22:08:13.647 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.764+0000 7f5613a4b640 1 Processor -- start 2024-05-20T22:08:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.764+0000 7f5613a4b640 1 -- start start 2024-05-20T22:08:13.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f5613a4b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f560c0b0e00 0x7f560c0d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f5613a4b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560c0b2d50 0x7f560c0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f5613a4b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f560c09b220 con 0x7f560c0cc6e0 2024-05-20T22:08:13.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f5613a4b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f560c0d1340 con 0x7f560c0b0e00 2024-05-20T22:08:13.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f5613a4b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f560c15f610 con 0x7f560c0b2d50 2024-05-20T22:08:13.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f56117e7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f560c0b0e00 0x7f560c0d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f5611fe8640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f560c0cc6e0 0x7f560c0ccaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46308/0 (socket says 172.21.15.55:46308) 2024-05-20T22:08:13.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f5611fe8640 1 -- 172.21.15.55:0/1154279985 learned_addr learned my addr 172.21.15.55:0/1154279985 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:13.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f5610fe6640 1 --2- 172.21.15.55:0/1154279985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560c0b2d50 0x7f560c0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f5603fff640 1 -- 172.21.15.55:0/1154279985 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2067386747 0 0) 0x7f560c09b220 con 0x7f560c0cc6e0 2024-05-20T22:08:13.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f5603fff640 1 -- 172.21.15.55:0/1154279985 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f8003520 con 0x7f560c0cc6e0 2024-05-20T22:08:13.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f56117e7640 1 -- 172.21.15.55:0/1154279985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560c0b2d50 msgr2=0x7f560c0d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:13.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f56117e7640 1 --2- 172.21.15.55:0/1154279985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560c0b2d50 0x7f560c0d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f56117e7640 1 -- 172.21.15.55:0/1154279985 >> v1:172.21.15.55:6789/0 conn(0x7f560c0cc6e0 legacy=0x7f560c0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.767+0000 7f56117e7640 1 -- 172.21.15.55:0/1154279985 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c09ac60 con 0x7f560c0b0e00 2024-05-20T22:08:13.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.768+0000 7f56117e7640 1 --2- 172.21.15.55:0/1154279985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0b0e00 0x7f560c0d08c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f560c0d14c0 tx=0x7f560802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4849a080931997 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:13.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.768+0000 7f5603fff640 1 -- 172.21.15.55:0/1154279985 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608037070 con 0x7f560c0b0e00 2024-05-20T22:08:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.768+0000 7f5603fff640 1 -- 172.21.15.55:0/1154279985 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f560802ae50 con 0x7f560c0b0e00 2024-05-20T22:08:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.768+0000 7f5603fff640 1 -- 172.21.15.55:0/1154279985 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608032660 con 0x7f560c0b0e00 2024-05-20T22:08:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.769+0000 7f5613a4b640 1 -- 172.21.15.55:0/1154279985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0b0e00 msgr2=0x7f560c0d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.769+0000 7f5613a4b640 1 --2- 172.21.15.55:0/1154279985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0b0e00 0x7f560c0d08c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f560c0d14c0 tx=0x7f560802a100 comp rx=0 tx=0).stop 2024-05-20T22:08:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.770+0000 7f5613a4b640 1 -- 172.21.15.55:0/1154279985 shutdown_connections 2024-05-20T22:08:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.770+0000 7f5613a4b640 1 --2- 172.21.15.55:0/1154279985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560c0b2d50 0x7f560c0d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.770+0000 7f5613a4b640 1 --2- 172.21.15.55:0/1154279985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0b0e00 0x7f560c0d08c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.770+0000 7f5613a4b640 1 -- 172.21.15.55:0/1154279985 >> 172.21.15.55:0/1154279985 conn(0x7f560c0b6420 msgr2=0x7f560c0b6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.770+0000 7f5613a4b640 1 -- 172.21.15.55:0/1154279985 shutdown_connections 2024-05-20T22:08:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.770+0000 7f5613a4b640 1 -- 172.21.15.55:0/1154279985 wait complete. 2024-05-20T22:08:13.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.772+0000 7f5613a4b640 1 Processor -- start 2024-05-20T22:08:13.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.772+0000 7f5613a4b640 1 -- start start 2024-05-20T22:08:13.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f5613a4b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560c0b0e00 0x7f560c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f5613a4b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f560c0b2d50 0x7f560c06bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f5613a4b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0cc6e0 0x7f560c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f5613a4b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f560c0cc1c0 con 0x7f560c0cc6e0 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f5613a4b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f560c0cbee0 con 0x7f560c0b2d50 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f5613a4b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f560c06e800 con 0x7f560c0b0e00 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f5611fe8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0cc6e0 0x7f560c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f5611fe8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0cc6e0 0x7f560c06c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57930/0 (socket says 172.21.15.55:57930) 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f5611fe8640 1 -- 172.21.15.55:0/839083301 learned_addr learned my addr 172.21.15.55:0/839083301 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f56117e7640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560c0b0e00 0x7f560c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.773+0000 7f5610fe6640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f560c0b2d50 0x7f560c06bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.774+0000 7f5611fe8640 1 -- 172.21.15.55:0/839083301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560c0b0e00 msgr2=0x7f560c06d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.774+0000 7f5611fe8640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560c0b0e00 0x7f560c06d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.774+0000 7f5611fe8640 1 -- 172.21.15.55:0/839083301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f560c0b2d50 msgr2=0x7f560c06bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.774+0000 7f5611fe8640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f560c0b2d50 0x7f560c06bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.774+0000 7f5611fe8640 1 -- 172.21.15.55:0/839083301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c0c7dc0 con 0x7f560c0cc6e0 2024-05-20T22:08:13.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.774+0000 7f5611fe8640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0cc6e0 0x7f560c06c290 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f560400caa0 tx=0x7f560400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:13.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.775+0000 7f5601ffb640 1 -- 172.21.15.55:0/839083301 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604017070 con 0x7f560c0cc6e0 2024-05-20T22:08:13.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.775+0000 7f5601ffb640 1 -- 172.21.15.55:0/839083301 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f560400ac00 con 0x7f560c0cc6e0 2024-05-20T22:08:13.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.775+0000 7f5613a4b640 1 -- 172.21.15.55:0/839083301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c0c7ae0 con 0x7f560c0cc6e0 2024-05-20T22:08:13.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.775+0000 7f5601ffb640 1 -- 172.21.15.55:0/839083301 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604012400 con 0x7f560c0cc6e0 2024-05-20T22:08:13.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.775+0000 7f5613a4b640 1 -- 172.21.15.55:0/839083301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c149820 con 0x7f560c0cc6e0 2024-05-20T22:08:13.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.776+0000 7f5613a4b640 1 -- 172.21.15.55:0/839083301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560c069090 con 0x7f560c0cc6e0 2024-05-20T22:08:13.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.777+0000 7f5601ffb640 1 -- 172.21.15.55:0/839083301 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f560401a070 con 0x7f560c0cc6e0 2024-05-20T22:08:13.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.777+0000 7f5601ffb640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55f006be30 0x7f55f006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:13.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.777+0000 7f5601ffb640 1 -- 172.21.15.55:0/839083301 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f560408add0 con 0x7f560c0cc6e0 2024-05-20T22:08:13.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.777+0000 7f56117e7640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55f006be30 0x7f55f006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:13.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.778+0000 7f56117e7640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55f006be30 0x7f55f006e340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5608034000 tx=0x7f560802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:13.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.782+0000 7f5601ffb640 1 -- 172.21.15.55:0/839083301 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5604058df0 con 0x7f560c0cc6e0 2024-05-20T22:08:13.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.968+0000 7f5613a4b640 1 -- 172.21.15.55:0/839083301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f560c0cdb00 con 0x7f560c0cc6e0 2024-05-20T22:08:13.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.969+0000 7f5601ffb640 1 -- 172.21.15.55:0/839083301 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f560408c090 con 0x7f560c0cc6e0 2024-05-20T22:08:13.970 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:13.971 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:13.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 -- 172.21.15.55:0/839083301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55f006be30 msgr2=0x7f55f006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55f006be30 0x7f55f006e340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5608034000 tx=0x7f560802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 -- 172.21.15.55:0/839083301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0cc6e0 msgr2=0x7f560c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0cc6e0 0x7f560c06c290 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f560400caa0 tx=0x7f560400cf60 comp rx=0 tx=0).stop 2024-05-20T22:08:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 -- 172.21.15.55:0/839083301 shutdown_connections 2024-05-20T22:08:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55f006be30 0x7f55f006e340 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f560c0cc6e0 0x7f560c06c290 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f560c0b2d50 0x7f560c06bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 --2- 172.21.15.55:0/839083301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560c0b0e00 0x7f560c06d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:13.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 -- 172.21.15.55:0/839083301 >> 172.21.15.55:0/839083301 conn(0x7f560c0b6420 msgr2=0x7f560c0ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:13.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 -- 172.21.15.55:0/839083301 shutdown_connections 2024-05-20T22:08:13.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:13.972+0000 7f5613a4b640 1 -- 172.21.15.55:0/839083301 wait complete. 2024-05-20T22:08:13.985 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-20T22:08:13.986 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.100+0000 7fda943aa640 1 Processor -- start 2024-05-20T22:08:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.100+0000 7fda943aa640 1 -- start start 2024-05-20T22:08:14.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda943aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c0d1f40 0x7fda8c0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda943aa640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda8c0b2720 con 0x7fda8c06e520 2024-05-20T22:08:14.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda943aa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda8c0cd560 con 0x7fda8c06dfd0 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda943aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fda8c06bd90 con 0x7fda8c0d1f40 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda92146640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fda8c06e520 0x7fda8c0d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46322/0 (socket says 172.21.15.55:46322) 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda92146640 1 -- 172.21.15.55:0/4007686589 learned_addr learned my addr 172.21.15.55:0/4007686589 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda91945640 1 --2- 172.21.15.55:0/4007686589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c0d1f40 0x7fda8c0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda91144640 1 -- 172.21.15.55:0/4007686589 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1402983536 0 0) 0x7fda8c0b2720 con 0x7fda8c06e520 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda91144640 1 -- 172.21.15.55:0/4007686589 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda78003520 con 0x7fda8c06e520 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda91144640 1 -- 172.21.15.55:0/4007686589 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1735762648 0 0) 0x7fda8c0cd560 con 0x7fda8c06dfd0 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda91144640 1 -- 172.21.15.55:0/4007686589 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda8c0b2720 con 0x7fda8c06dfd0 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.102+0000 7fda91945640 1 -- 172.21.15.55:0/4007686589 >> v1:172.21.15.138:6789/0 conn(0x7fda8c06dfd0 legacy=0x7fda8c0ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.103+0000 7fda91945640 1 -- 172.21.15.55:0/4007686589 >> v1:172.21.15.55:6789/0 conn(0x7fda8c06e520 legacy=0x7fda8c0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.103+0000 7fda91945640 1 -- 172.21.15.55:0/4007686589 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c0b2440 con 0x7fda8c0d1f40 2024-05-20T22:08:14.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.103+0000 7fda91945640 1 --2- 172.21.15.55:0/4007686589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c0d1f40 0x7fda8c0ce000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fda800097d0 tx=0x7fda8002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483ab2ee6a891521 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.104+0000 7fda91144640 1 -- 172.21.15.55:0/4007686589 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80037070 con 0x7fda8c0d1f40 2024-05-20T22:08:14.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.104+0000 7fda91144640 1 -- 172.21.15.55:0/4007686589 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda80032440 con 0x7fda8c0d1f40 2024-05-20T22:08:14.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.104+0000 7fda91144640 1 -- 172.21.15.55:0/4007686589 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80032810 con 0x7fda8c0d1f40 2024-05-20T22:08:14.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.104+0000 7fda943aa640 1 -- 172.21.15.55:0/4007686589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c0d1f40 msgr2=0x7fda8c0ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.104+0000 7fda943aa640 1 --2- 172.21.15.55:0/4007686589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c0d1f40 0x7fda8c0ce000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fda800097d0 tx=0x7fda8002a210 comp rx=0 tx=0).stop 2024-05-20T22:08:14.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.104+0000 7fda943aa640 1 -- 172.21.15.55:0/4007686589 shutdown_connections 2024-05-20T22:08:14.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.104+0000 7fda943aa640 1 --2- 172.21.15.55:0/4007686589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c0d1f40 0x7fda8c0ce000 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.104+0000 7fda943aa640 1 -- 172.21.15.55:0/4007686589 >> 172.21.15.55:0/4007686589 conn(0x7fda8c0b6580 msgr2=0x7fda8c0b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.104+0000 7fda943aa640 1 -- 172.21.15.55:0/4007686589 shutdown_connections 2024-05-20T22:08:14.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.104+0000 7fda943aa640 1 -- 172.21.15.55:0/4007686589 wait complete. 2024-05-20T22:08:14.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.106+0000 7fda943aa640 1 Processor -- start 2024-05-20T22:08:14.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.106+0000 7fda943aa640 1 -- start start 2024-05-20T22:08:14.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.107+0000 7fda943aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c06dfd0 0x7fda8c09d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.107+0000 7fda943aa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda8c06e520 0x7fda8c09b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda943aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda8c1491d0 0x7fda8c1495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda943aa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c0b2160 con 0x7fda8c06e520 2024-05-20T22:08:14.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda943aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c15d4f0 con 0x7fda8c1491d0 2024-05-20T22:08:14.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda943aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fda8c15d670 con 0x7fda8c06dfd0 2024-05-20T22:08:14.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda91945640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda8c06e520 0x7fda8c09b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda91945640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda8c06e520 0x7fda8c09b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57934/0 (socket says 172.21.15.55:57934) 2024-05-20T22:08:14.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda91945640 1 -- 172.21.15.55:0/2825309843 learned_addr learned my addr 172.21.15.55:0/2825309843 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda92947640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda8c1491d0 0x7fda8c1495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda92146640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c06dfd0 0x7fda8c09d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda91945640 1 -- 172.21.15.55:0/2825309843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c06dfd0 msgr2=0x7fda8c09d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda91945640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c06dfd0 0x7fda8c09d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda91945640 1 -- 172.21.15.55:0/2825309843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda8c1491d0 msgr2=0x7fda8c1495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda91945640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda8c1491d0 0x7fda8c1495e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda91945640 1 -- 172.21.15.55:0/2825309843 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c148130 con 0x7fda8c06e520 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda91945640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda8c06e520 0x7fda8c09b810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda800098f0 tx=0x7fda800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda7effd640 1 -- 172.21.15.55:0/2825309843 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80041020 con 0x7fda8c06e520 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda7effd640 1 -- 172.21.15.55:0/2825309843 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda8003d6e0 con 0x7fda8c06e520 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda943aa640 1 -- 172.21.15.55:0/2825309843 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c148410 con 0x7fda8c06e520 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.108+0000 7fda7effd640 1 -- 172.21.15.55:0/2825309843 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80037040 con 0x7fda8c06e520 2024-05-20T22:08:14.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.109+0000 7fda943aa640 1 -- 172.21.15.55:0/2825309843 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c1489d0 con 0x7fda8c06e520 2024-05-20T22:08:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.109+0000 7fda943aa640 1 -- 172.21.15.55:0/2825309843 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8c069090 con 0x7fda8c06e520 2024-05-20T22:08:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.110+0000 7fda7effd640 1 -- 172.21.15.55:0/2825309843 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda80044020 con 0x7fda8c06e520 2024-05-20T22:08:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.111+0000 7fda7effd640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fda7006be30 0x7fda7006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.111+0000 7fda7effd640 1 -- 172.21.15.55:0/2825309843 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fda800ab560 con 0x7fda8c06e520 2024-05-20T22:08:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.111+0000 7fda92146640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fda7006be30 0x7fda7006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.111+0000 7fda92146640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fda7006be30 0x7fda7006e340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda88000f60 tx=0x7fda88007d50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.115+0000 7fda7effd640 1 -- 172.21.15.55:0/2825309843 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fda8004b030 con 0x7fda8c06e520 2024-05-20T22:08:14.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.302+0000 7fda943aa640 1 -- 172.21.15.55:0/2825309843 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda8c15c7e0 con 0x7fda8c06e520 2024-05-20T22:08:14.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.303+0000 7fda7effd640 1 -- 172.21.15.55:0/2825309843 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fda80079590 con 0x7fda8c06e520 2024-05-20T22:08:14.304 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:14.304 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.305+0000 7fda943aa640 1 -- 172.21.15.55:0/2825309843 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fda7006be30 msgr2=0x7fda7006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.305+0000 7fda943aa640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fda7006be30 0x7fda7006e340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda88000f60 tx=0x7fda88007d50 comp rx=0 tx=0).stop 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.305+0000 7fda943aa640 1 -- 172.21.15.55:0/2825309843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda8c06e520 msgr2=0x7fda8c09b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.305+0000 7fda943aa640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda8c06e520 0x7fda8c09b810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda800098f0 tx=0x7fda800046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.306+0000 7fda943aa640 1 -- 172.21.15.55:0/2825309843 shutdown_connections 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.306+0000 7fda943aa640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fda7006be30 0x7fda7006e340 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.306+0000 7fda943aa640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda8c1491d0 0x7fda8c1495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.306+0000 7fda943aa640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda8c06e520 0x7fda8c09b810 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.306+0000 7fda943aa640 1 --2- 172.21.15.55:0/2825309843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fda8c06dfd0 0x7fda8c09d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.306+0000 7fda943aa640 1 -- 172.21.15.55:0/2825309843 >> 172.21.15.55:0/2825309843 conn(0x7fda8c0b6580 msgr2=0x7fda8c070120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.306+0000 7fda943aa640 1 -- 172.21.15.55:0/2825309843 shutdown_connections 2024-05-20T22:08:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.306+0000 7fda943aa640 1 -- 172.21.15.55:0/2825309843 wait complete. 2024-05-20T22:08:14.318 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-20T22:08:14.319 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-20T22:08:14.319 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-20T22:08:14.319 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-20T22:08:14.320 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-20T22:08:14.320 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-20T22:08:14.321 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-20T22:08:14.321 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-20T22:08:14.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.532+0000 7f1671479640 1 Processor -- start 2024-05-20T22:08:14.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.532+0000 7f1671479640 1 -- start start 2024-05-20T22:08:14.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.534+0000 7f1671479640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f166c0ce230 0x7f166c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.535+0000 7f166b7fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f166c09add0 0x7f166c15fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46336/0 (socket says 172.21.15.55:46336) 2024-05-20T22:08:14.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.535+0000 7f166b7fe640 1 -- 172.21.15.55:0/3063637049 learned_addr learned my addr 172.21.15.55:0/3063637049 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:14.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.535+0000 7f1671479640 1 -- 172.21.15.55:0/3063637049 --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166c0b3180 con 0x7f166c09add0 2024-05-20T22:08:14.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.535+0000 7f1671479640 1 -- 172.21.15.55:0/3063637049 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166c0ce730 con 0x7f166c0cf3f0 2024-05-20T22:08:14.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.536+0000 7f1671479640 1 -- 172.21.15.55:0/3063637049 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f166c0cea10 con 0x7f166c0ce230 2024-05-20T22:08:14.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.536+0000 7f80e0b55640 1 Processor -- start 2024-05-20T22:08:14.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.537+0000 7f80e0b55640 1 -- start start 2024-05-20T22:08:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.539+0000 7f80e0b55640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d4053dd0 0x7f80d40e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.537+0000 7f166affd640 1 --2- 172.21.15.55:0/3063637049 >> v2:172.21.15.138:3300/0 conn(0x7f166c0ce230 0x7f166c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294624513 0 0) 0x7f166c0b3180 con 0x7f166c09add0 2024-05-20T22:08:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1654003520 con 0x7f166c09add0 2024-05-20T22:08:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1648488333 0 0) 0x7f166c0ce730 con 0x7f166c0cf3f0 2024-05-20T22:08:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f166c0b3180 con 0x7f166c0cf3f0 2024-05-20T22:08:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 943498269 0 0) 0x7f166c0b3180 con 0x7f166c0cf3f0 2024-05-20T22:08:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 >> v2:172.21.15.138:3300/0 conn(0x7f166c0ce230 msgr2=0x7f166c0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 --2- 172.21.15.55:0/3063637049 >> v2:172.21.15.138:3300/0 conn(0x7f166c0ce230 0x7f166c0b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 >> v1:172.21.15.55:6789/0 conn(0x7f166c09add0 legacy=0x7f166c15fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c0cecf0 con 0x7f166c0cf3f0 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f1660001540 con 0x7f166c0cf3f0 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 >> v1:172.21.15.138:6789/0 conn(0x7f166c0cf3f0 legacy=0x7f166c09a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 --2- 172.21.15.55:0/3063637049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1654004ba0 0x7f1654005030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 --2- 172.21.15.55:0/3063637049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16540037a0 0x7f1654003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 --2- 172.21.15.55:0/3063637049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1654004260 0x7f165400ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f166c0cea10 con 0x7f16540037a0 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f165400c2b0 con 0x7f1654004ba0 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.538+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f165400c4b0 con 0x7f1654004260 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.539+0000 7f166b7fe640 1 --2- 172.21.15.55:0/3063637049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1654004260 0x7f165400ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.539+0000 7f166a7fc640 1 --2- 172.21.15.55:0/3063637049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16540037a0 0x7f1654003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.539+0000 7f80e0b55640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d40e3510 con 0x7f80d4055800 2024-05-20T22:08:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.539+0000 7f80e0b55640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d40506f0 con 0x7f80d4034050 2024-05-20T22:08:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.539+0000 7f80e0b55640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80d40e3840 con 0x7f80d4053dd0 2024-05-20T22:08:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.540+0000 7f166a7fc640 1 -- 172.21.15.55:0/3063637049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1654004260 msgr2=0x7f165400ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.540+0000 7f80daffd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f80d4055800 0x7f80d4033940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46338/0 (socket says 172.21.15.55:46338) 2024-05-20T22:08:14.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.540+0000 7f166a7fc640 1 --2- 172.21.15.55:0/3063637049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1654004260 0x7f165400ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.540+0000 7f80daffd640 1 -- 172.21.15.55:0/2181470451 learned_addr learned my addr 172.21.15.55:0/2181470451 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.540+0000 7f166a7fc640 1 -- 172.21.15.55:0/3063637049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1654004ba0 msgr2=0x7f1654005030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:08:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.540+0000 7f166a7fc640 1 --2- 172.21.15.55:0/3063637049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1654004ba0 0x7f1654005030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.540+0000 7f166b7fe640 1 -- 172.21.15.55:0/3063637049 reap_dead start 2024-05-20T22:08:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.540+0000 7f166a7fc640 1 -- 172.21.15.55:0/3063637049 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c0cecf0 con 0x7f16540037a0 2024-05-20T22:08:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.540+0000 7f80db7fe640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d4053dd0 0x7f80d40e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f2683c5f640 1 Processor -- start 2024-05-20T22:08:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1758171231 0 0) 0x7f80d40e3510 con 0x7f80d4055800 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80cc003520 con 0x7f80d4055800 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 995086329 0 0) 0x7f80cc003520 con 0x7f80d4055800 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d4053dd0 msgr2=0x7f80d40e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d4053dd0 0x7f80d40e3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 >> v1:172.21.15.138:6789/0 conn(0x7f80d4034050 legacy=0x7f80d4006080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d40509d0 con 0x7f80d4055800 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f80c8001690 con 0x7f80d4055800 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 >> v1:172.21.15.55:6789/0 conn(0x7f80d4055800 legacy=0x7f80d4033940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80cc0039e0 0x7f80cc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 0x7f80cc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc0051a0 0x7f80cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f80d40e3840 con 0x7f80cc004800 2024-05-20T22:08:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc00c110 con 0x7f80cc0051a0 2024-05-20T22:08:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc00c310 con 0x7f80cc0039e0 2024-05-20T22:08:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.542+0000 7f80daffd640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 0x7f80cc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.542+0000 7f166a7fc640 1 --2- 172.21.15.55:0/3063637049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16540037a0 0x7f1654003bb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f166c0b3350 tx=0x7f1654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07736f68399cf8c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.541+0000 7f2683c5f640 1 -- start start 2024-05-20T22:08:14.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.542+0000 7f80daffd640 1 -- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80cc0039e0 msgr2=0x7f80cc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.542+0000 7f80db7fe640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80cc0039e0 0x7f80cc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.542+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166003a070 con 0x7f16540037a0 2024-05-20T22:08:14.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.542+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166002d7f0 con 0x7f16540037a0 2024-05-20T22:08:14.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.542+0000 7f1669ffb640 1 -- 172.21.15.55:0/3063637049 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166003a070 con 0x7f16540037a0 2024-05-20T22:08:14.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.543+0000 7f80daffd640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80cc0039e0 0x7f80cc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.543+0000 7f80daffd640 1 -- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc0051a0 msgr2=0x7f80cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.543+0000 7f80daffd640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc0051a0 0x7f80cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.543+0000 7f80daffd640 1 -- 172.21.15.55:0/2181470451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d40509d0 con 0x7f80cc004800 2024-05-20T22:08:14.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f1671479640 1 -- 172.21.15.55:0/3063637049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16540037a0 msgr2=0x7f1654003bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f80db7fe640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80cc0039e0 0x7f80cc003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:14.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f1671479640 1 --2- 172.21.15.55:0/3063637049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16540037a0 0x7f1654003bb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f166c0b3350 tx=0x7f1654005830 comp rx=0 tx=0).stop 2024-05-20T22:08:14.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f2683c5f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f267c073400 0x7f267c06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f2683c5f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f267c15b160 0x7f267c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f2683c5f640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267c06faa0 con 0x7f267c15ad50 2024-05-20T22:08:14.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f2683c5f640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f267c0b4450 con 0x7f267c15b160 2024-05-20T22:08:14.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f2683c5f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f267c0b4730 con 0x7f267c073400 2024-05-20T22:08:14.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f26811fa640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f267c15ad50 0x7f267c06f390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46344/0 (socket says 172.21.15.55:46344) 2024-05-20T22:08:14.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f1671479640 1 -- 172.21.15.55:0/3063637049 shutdown_connections 2024-05-20T22:08:14.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f1671479640 1 --2- 172.21.15.55:0/3063637049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16540037a0 0x7f1654003bb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f1671479640 1 -- 172.21.15.55:0/3063637049 >> 172.21.15.55:0/3063637049 conn(0x7f166c0737c0 msgr2=0x7f166c0b6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f80daffd640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 0x7f80cc004c10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f80d4052cf0 tx=0x7f80cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae28008acfda0903 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8031070 con 0x7f80cc004800 2024-05-20T22:08:14.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c8005ad0 con 0x7f80cc004800 2024-05-20T22:08:14.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f80da7fc640 1 -- 172.21.15.55:0/2181470451 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8007e20 con 0x7f80cc004800 2024-05-20T22:08:14.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.545+0000 7fdd1360d640 1 Processor -- start 2024-05-20T22:08:14.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.544+0000 7f26811fa640 1 -- 172.21.15.55:0/3959024016 learned_addr learned my addr 172.21.15.55:0/3959024016 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:14.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.545+0000 7f26819fb640 1 --2- 172.21.15.55:0/3959024016 >> v2:172.21.15.138:3300/0 conn(0x7f267c073400 0x7f267c06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.545+0000 7f26821fc640 1 --2- 172.21.15.55:0/3959024016 >> v2:172.21.15.55:3300/0 conn(0x7f267c15b160 0x7f267c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.545+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3348090531 0 0) 0x7f267c06faa0 con 0x7f267c15ad50 2024-05-20T22:08:14.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.545+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2668003520 con 0x7f267c15ad50 2024-05-20T22:08:14.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f80e0b55640 1 -- 172.21.15.55:0/2181470451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 msgr2=0x7f80cc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f80e0b55640 1 --2- 172.21.15.55:0/2181470451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 0x7f80cc004c10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f80d4052cf0 tx=0x7f80cc005830 comp rx=0 tx=0).stop 2024-05-20T22:08:14.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f1671479640 1 -- 172.21.15.55:0/3063637049 shutdown_connections 2024-05-20T22:08:14.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.545+0000 7fdd1360d640 1 -- start start 2024-05-20T22:08:14.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f80dbfff640 1 -- 172.21.15.55:0/2181470451 reap_dead start 2024-05-20T22:08:14.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f72e09d5640 1 Processor -- start 2024-05-20T22:08:14.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f72e09d5640 1 -- start start 2024-05-20T22:08:14.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.545+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 675091740 0 0) 0x7f2668003520 con 0x7f267c15ad50 2024-05-20T22:08:14.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.545+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 >> v2:172.21.15.138:3300/0 conn(0x7f267c073400 msgr2=0x7f267c06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.545+0000 7f26809f9640 1 --2- 172.21.15.55:0/3959024016 >> v2:172.21.15.138:3300/0 conn(0x7f267c073400 0x7f267c06b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 >> v2:172.21.15.55:3300/0 conn(0x7f267c15b160 msgr2=0x7f267c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 --2- 172.21.15.55:0/3959024016 >> v2:172.21.15.55:3300/0 conn(0x7f267c15b160 0x7f267c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c0aec00 con 0x7f267c15ad50 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f2678001670 con 0x7f267c15ad50 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 >> v1:172.21.15.55:6789/0 conn(0x7f267c15ad50 legacy=0x7f267c06f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2668003a00 0x7f2668003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2668004800 0x7f2668004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2668005150 0x7f266800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0b4450 con 0x7f2668003a00 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0b4730 con 0x7f2668005150 2024-05-20T22:08:14.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f266800c090 con 0x7f2668004800 2024-05-20T22:08:14.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f80e0b55640 1 -- 172.21.15.55:0/2181470451 shutdown_connections 2024-05-20T22:08:14.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f80e0b55640 1 -- 172.21.15.55:0/2181470451 >> 172.21.15.55:0/2181470451 conn(0x7f80d4054ce0 msgr2=0x7f80d40550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26811fa640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2668004800 0x7f2668004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26821fc640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2668005150 0x7f266800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.546+0000 7f26819fb640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2668003a00 0x7f2668003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.547+0000 7f80e0b55640 1 -- 172.21.15.55:0/2181470451 shutdown_connections 2024-05-20T22:08:14.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.547+0000 7f215e3b5640 1 Processor -- start 2024-05-20T22:08:14.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.547+0000 7f215e3b5640 1 -- start start 2024-05-20T22:08:14.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7fdd1360d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c0ae6c0 0x7fdd0c15b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7fdd1360d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd0c0aead0 con 0x7fdd0c09a260 2024-05-20T22:08:14.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7fdd1360d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd0c0afd10 con 0x7fdd0c15f660 2024-05-20T22:08:14.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7fdd1360d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0c0b03f0 con 0x7fdd0c0ae6c0 2024-05-20T22:08:14.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.547+0000 7f26819fb640 1 -- 172.21.15.55:0/3959024016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2668004800 msgr2=0x7f2668004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.547+0000 7f26819fb640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2668004800 0x7f2668004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.547+0000 7f26819fb640 1 -- 172.21.15.55:0/3959024016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2668005150 msgr2=0x7f266800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.547+0000 7f26819fb640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2668005150 0x7f266800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.547+0000 7f26819fb640 1 -- 172.21.15.55:0/3959024016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2668003520 con 0x7f2668003a00 2024-05-20T22:08:14.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7f26821fc640 1 -- 172.21.15.55:0/3959024016 reap_dead start 2024-05-20T22:08:14.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7f26819fb640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2668003a00 0x7f2668003e70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2674009fa0 tx=0x7f2668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3165f0d371f2de3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7fdd11baa640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fdd0c15f660 0x7fdd0c0b0b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:37754/0 (socket says 172.21.15.55:37754) 2024-05-20T22:08:14.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7fdd11baa640 1 -- 172.21.15.55:0/1951247468 learned_addr learned my addr 172.21.15.55:0/1951247468 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:14.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.547+0000 7f80e0b55640 1 -- 172.21.15.55:0/2181470451 wait complete. 2024-05-20T22:08:14.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.549+0000 7fdd10ba8640 1 --2- 172.21.15.55:0/1951247468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c0ae6c0 0x7fdd0c15b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7f80e0b55640 1 Processor -- start 2024-05-20T22:08:14.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7f215e3b5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f215806c2b0 0x7f2158163e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7f215e3b5640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2158072690 con 0x7f2158163a50 2024-05-20T22:08:14.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7f215e3b5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2158164560 con 0x7f215815a6f0 2024-05-20T22:08:14.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7f215e3b5640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f2158164840 con 0x7f215806c2b0 2024-05-20T22:08:14.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7fdd10ba8640 1 -- 172.21.15.55:0/1951247468 >> v1:172.21.15.138:6789/0 conn(0x7fdd0c15f660 legacy=0x7fdd0c0b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.549+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674034020 con 0x7f2668003a00 2024-05-20T22:08:14.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.549+0000 7f1671479640 1 -- 172.21.15.55:0/3063637049 wait complete. 2024-05-20T22:08:14.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7fdd10ba8640 1 -- 172.21.15.55:0/1951247468 >> v1:172.21.15.55:6789/0 conn(0x7fdd0c09a260 legacy=0x7fdd0c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7fdd10ba8640 1 -- 172.21.15.55:0/1951247468 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c162080 con 0x7fdd0c0ae6c0 2024-05-20T22:08:14.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.549+0000 7f26809f9640 1 -- 172.21.15.55:0/3959024016 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26740306b0 con 0x7f2668003a00 2024-05-20T22:08:14.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7f80e0b55640 1 -- start start 2024-05-20T22:08:14.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7f2683c5f640 1 -- 172.21.15.55:0/3959024016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2668003a00 msgr2=0x7f2668003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.550+0000 7f2683c5f640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2668003a00 0x7f2668003e70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2674009fa0 tx=0x7f2668005830 comp rx=0 tx=0).stop 2024-05-20T22:08:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.551+0000 7f2157fff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f215815a6f0 0x7f2158073130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:37760/0 (socket says 172.21.15.55:37760) 2024-05-20T22:08:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.551+0000 7f2157fff640 1 -- 172.21.15.55:0/3536354578 learned_addr learned my addr 172.21.15.55:0/3536354578 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.551+0000 7fdd10ba8640 1 --2- 172.21.15.55:0/1951247468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c0ae6c0 0x7fdd0c15b350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009750 tx=0x7fdd0802a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb54aca05af5a51 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7f72e09d5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc15b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7f72e09d5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f72dc15f670 0x7f72dc06bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7f72e09d5640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72dc06cc80 con 0x7f72dc06c490 2024-05-20T22:08:14.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7f72e09d5640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f72dc15ee70 con 0x7f72dc15f670 2024-05-20T22:08:14.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.548+0000 7f72e09d5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f72dc162fe0 con 0x7f72dc070780 2024-05-20T22:08:14.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.551+0000 7f72da59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.551+0000 7f72da59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc15b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47164/0 (socket says 172.21.15.55:47164) 2024-05-20T22:08:14.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.551+0000 7f72da59c640 1 -- 172.21.15.55:0/1200225792 learned_addr learned my addr 172.21.15.55:0/1200225792 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.551+0000 7fdd03fff640 1 -- 172.21.15.55:0/1951247468 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08037070 con 0x7fdd0c0ae6c0 2024-05-20T22:08:14.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.551+0000 7f72d9d9b640 1 --2- 172.21.15.55:0/1200225792 >> v2:172.21.15.55:3300/0 conn(0x7f72dc15f670 0x7f72dc06bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7fdd03fff640 1 -- 172.21.15.55:0/1951247468 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd0802ad50 con 0x7fdd0c0ae6c0 2024-05-20T22:08:14.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.551+0000 7f2683c5f640 1 -- 172.21.15.55:0/3959024016 shutdown_connections 2024-05-20T22:08:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7f2683c5f640 1 --2- 172.21.15.55:0/3959024016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2668003a00 0x7f2668003e70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7f2683c5f640 1 -- 172.21.15.55:0/3959024016 >> 172.21.15.55:0/3959024016 conn(0x7f267c0d0730 msgr2=0x7f267c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7f215c952640 1 --2- 172.21.15.55:0/3536354578 >> v2:172.21.15.55:3300/0 conn(0x7f215806c2b0 0x7f2158163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7f80e0b55640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 0x7f80d404b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7f80db7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 0x7f80d404b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80db7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 0x7f80d404b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57988/0 (socket says 172.21.15.55:57988) 2024-05-20T22:08:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80db7fe640 1 -- 172.21.15.55:0/3168802828 learned_addr learned my addr 172.21.15.55:0/3168802828 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80e0b55640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d4046600 0x7f80d4046a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80e0b55640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80d4049610 0x7f80d4049a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4006910 con 0x7f80cc004800 2024-05-20T22:08:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4006790 con 0x7f80d4049610 2024-05-20T22:08:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80d4006a90 con 0x7f80d4046600 2024-05-20T22:08:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7fdd1360d640 1 -- 172.21.15.55:0/1951247468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c0ae6c0 msgr2=0x7fdd0c15b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7fdd1360d640 1 --2- 172.21.15.55:0/1951247468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c0ae6c0 0x7fdd0c15b350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009750 tx=0x7fdd0802a040 comp rx=0 tx=0).stop 2024-05-20T22:08:14.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7fdd1360d640 1 -- 172.21.15.55:0/1951247468 shutdown_connections 2024-05-20T22:08:14.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7fdd1360d640 1 --2- 172.21.15.55:0/1951247468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c0ae6c0 0x7fdd0c15b350 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009750 tx=0x7fdd0802a040 comp rx=0 tx=0).stop 2024-05-20T22:08:14.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7fdd1360d640 1 -- 172.21.15.55:0/1951247468 >> 172.21.15.55:0/1951247468 conn(0x7fdd0c0b62b0 msgr2=0x7fdd0c0b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7f72da59c640 1 -- 172.21.15.55:0/1200225792 >> v2:172.21.15.55:3300/0 conn(0x7f72dc15f670 msgr2=0x7f72dc06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7f72da59c640 1 --2- 172.21.15.55:0/1200225792 >> v2:172.21.15.55:3300/0 conn(0x7f72dc15f670 0x7f72dc06bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7f72da59c640 1 -- 172.21.15.55:0/1200225792 >> v1:172.21.15.55:6789/0 conn(0x7f72dc06c490 legacy=0x7f72dc0d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7f72da59c640 1 -- 172.21.15.55:0/1200225792 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc15fa80 con 0x7f72dc070780 2024-05-20T22:08:14.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80dbfff640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80d4049610 0x7f80d4049a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.552+0000 7f1671479640 1 Processor -- start 2024-05-20T22:08:14.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f215c952640 1 -- 172.21.15.55:0/3536354578 >> v1:172.21.15.138:6789/0 conn(0x7f215815a6f0 legacy=0x7f2158073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f215c952640 1 -- 172.21.15.55:0/3536354578 >> v1:172.21.15.55:6789/0 conn(0x7f2158163a50 legacy=0x7f215806bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f2683c5f640 1 -- 172.21.15.55:0/3959024016 shutdown_connections 2024-05-20T22:08:14.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f2683c5f640 1 -- 172.21.15.55:0/3959024016 wait complete. 2024-05-20T22:08:14.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80db7fe640 1 -- 172.21.15.55:0/3168802828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d4046600 msgr2=0x7f80d4046a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T22:08:14.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80db7fe640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80d4046600 0x7f80d4046a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80db7fe640 1 -- 172.21.15.55:0/3168802828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80d4049610 msgr2=0x7f80d4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80db7fe640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80d4049610 0x7f80d4049a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f80db7fe640 1 -- 172.21.15.55:0/3168802828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d404b760 con 0x7f80cc004800 2024-05-20T22:08:14.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.553+0000 7f215c952640 1 -- 172.21.15.55:0/3536354578 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2158072930 con 0x7f215806c2b0 2024-05-20T22:08:14.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.554+0000 7f80db7fe640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 0x7f80d404b220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f80d00107c0 tx=0x7f80d0010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.554+0000 7f1671479640 1 -- start start 2024-05-20T22:08:14.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.554+0000 7f80c7fff640 1 -- 172.21.15.55:0/3168802828 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d000f040 con 0x7f80cc004800 2024-05-20T22:08:14.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.554+0000 7f80c7fff640 1 -- 172.21.15.55:0/3168802828 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d000e8f0 con 0x7f80cc004800 2024-05-20T22:08:14.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.554+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d4046f50 con 0x7f80cc004800 2024-05-20T22:08:14.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.555+0000 7f215c952640 1 --2- 172.21.15.55:0/3536354578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f215806c2b0 0x7f2158163e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f214c0097a0 tx=0x7f214c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862e61c705736db4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.555+0000 7fdd1360d640 1 -- 172.21.15.55:0/1951247468 shutdown_connections 2024-05-20T22:08:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.555+0000 7fdd1360d640 1 -- 172.21.15.55:0/1951247468 wait complete. 2024-05-20T22:08:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.554+0000 7f72da59c640 1 --2- 172.21.15.55:0/1200225792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc15b090 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009870 tx=0x7f72d002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8938d6ddc063ccd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.555+0000 7f72d959a640 1 -- 172.21.15.55:0/1200225792 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0037070 con 0x7f72dc070780 2024-05-20T22:08:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.555+0000 7f72d959a640 1 -- 172.21.15.55:0/1200225792 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d0032440 con 0x7f72dc070780 2024-05-20T22:08:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.555+0000 7f72d959a640 1 -- 172.21.15.55:0/1200225792 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0032810 con 0x7f72dc070780 2024-05-20T22:08:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f2683c5f640 1 Processor -- start 2024-05-20T22:08:14.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.555+0000 7f80c7fff640 1 -- 172.21.15.55:0/3168802828 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d000eca0 con 0x7f80cc004800 2024-05-20T22:08:14.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d40471d0 con 0x7f80cc004800 2024-05-20T22:08:14.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f1671479640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16540037a0 0x7f166c0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f2156ffd640 1 -- 172.21.15.55:0/3536354578 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c037070 con 0x7f215806c2b0 2024-05-20T22:08:14.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f1671479640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c2012e0 0x7f166c0b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f2156ffd640 1 -- 172.21.15.55:0/3536354578 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c02ad40 con 0x7f215806c2b0 2024-05-20T22:08:14.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f2156ffd640 1 -- 172.21.15.55:0/3536354578 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c032660 con 0x7f215806c2b0 2024-05-20T22:08:14.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.557+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80d4005560 con 0x7f80cc004800 2024-05-20T22:08:14.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f1671479640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f166c2016f0 0x7f166c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f1671479640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f166c15c200 con 0x7f166c2016f0 2024-05-20T22:08:14.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f1671479640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f166c15f950 con 0x7f16540037a0 2024-05-20T22:08:14.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f1671479640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f166c06e7f0 con 0x7f166c2012e0 2024-05-20T22:08:14.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.557+0000 7f80c7fff640 1 -- 172.21.15.55:0/3168802828 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80d001c400 con 0x7f80cc004800 2024-05-20T22:08:14.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.557+0000 7f215e3b5640 1 -- 172.21.15.55:0/3536354578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f215806c2b0 msgr2=0x7f2158163e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.557+0000 7f215e3b5640 1 --2- 172.21.15.55:0/3536354578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f215806c2b0 0x7f2158163e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f214c0097a0 tx=0x7f214c02a0b0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f166affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16540037a0 0x7f166c0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f166a7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c2012e0 0x7f166c0b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f166affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16540037a0 0x7f166c0ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50836/0 (socket says 172.21.15.55:50836) 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f166affd640 1 -- 172.21.15.55:0/4122763354 learned_addr learned my addr 172.21.15.55:0/4122763354 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f166b7fe640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f166c2016f0 0x7f166c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.557+0000 7f166a7fc640 1 -- 172.21.15.55:0/4122763354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16540037a0 msgr2=0x7f166c0ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.557+0000 7f166a7fc640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16540037a0 0x7f166c0ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.557+0000 7f166a7fc640 1 -- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f166c2016f0 msgr2=0x7f166c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.557+0000 7f166a7fc640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f166c2016f0 0x7f166c0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.557+0000 7f166a7fc640 1 -- 172.21.15.55:0/4122763354 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c15d300 con 0x7f166c2012e0 2024-05-20T22:08:14.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.557+0000 7f166a7fc640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c2012e0 0x7f166c0b00b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1660034000 tx=0x7f1660001890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7fdd1360d640 1 Processor -- start 2024-05-20T22:08:14.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f16537fe640 1 -- 172.21.15.55:0/4122763354 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166003a070 con 0x7f166c2012e0 2024-05-20T22:08:14.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f215e3b5640 1 -- 172.21.15.55:0/3536354578 shutdown_connections 2024-05-20T22:08:14.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f215e3b5640 1 --2- 172.21.15.55:0/3536354578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f215806c2b0 0x7f2158163e60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f215e3b5640 1 -- 172.21.15.55:0/3536354578 >> 172.21.15.55:0/3536354578 conn(0x7f21580b3980 msgr2=0x7f21580b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f16537fe640 1 -- 172.21.15.55:0/4122763354 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1660005070 con 0x7f166c2012e0 2024-05-20T22:08:14.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c15d5e0 con 0x7f166c2012e0 2024-05-20T22:08:14.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7fdd1360d640 1 -- start start 2024-05-20T22:08:14.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f16537fe640 1 -- 172.21.15.55:0/4122763354 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660007bb0 con 0x7f166c2012e0 2024-05-20T22:08:14.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c15dba0 con 0x7f166c2012e0 2024-05-20T22:08:14.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f80c7fff640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f80c006be30 0x7f80c006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.559+0000 7f80daffd640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f80c006be30 0x7f80c006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.559+0000 7f80c7fff640 1 -- 172.21.15.55:0/3168802828 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f80d008b530 con 0x7f80cc004800 2024-05-20T22:08:14.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.559+0000 7f215e3b5640 1 -- 172.21.15.55:0/3536354578 shutdown_connections 2024-05-20T22:08:14.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.556+0000 7f2683c5f640 1 -- start start 2024-05-20T22:08:14.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f2683c5f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2668003a00 0x7f267c144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f2683c5f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c15f690 0x7f267c1446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f2683c5f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f267c0cee00 0x7f267c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f26819fb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2668003a00 0x7f267c144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f26819fb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2668003a00 0x7f267c144180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47176/0 (socket says 172.21.15.55:47176) 2024-05-20T22:08:14.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f26811fa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c15f690 0x7f267c1446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f26811fa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c15f690 0x7f267c1446c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50840/0 (socket says 172.21.15.55:50840) 2024-05-20T22:08:14.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f26819fb640 1 -- 172.21.15.55:0/2453954917 learned_addr learned my addr 172.21.15.55:0/2453954917 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f2683c5f640 1 -- 172.21.15.55:0/2453954917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f267c070030 con 0x7f267c0cee00 2024-05-20T22:08:14.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f2683c5f640 1 -- 172.21.15.55:0/2453954917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f267c06feb0 con 0x7f267c15f690 2024-05-20T22:08:14.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f2683c5f640 1 -- 172.21.15.55:0/2453954917 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f267c06f390 con 0x7f2668003a00 2024-05-20T22:08:14.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.558+0000 7f26821fc640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f267c0cee00 0x7f267c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.560+0000 7f80daffd640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f80c006be30 0x7f80c006e340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80c8033000 tx=0x7f80c8009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.559+0000 7f26811fa640 1 -- 172.21.15.55:0/2453954917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2668003a00 msgr2=0x7f267c144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.559+0000 7f26811fa640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2668003a00 0x7f267c144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.559+0000 7f26811fa640 1 -- 172.21.15.55:0/2453954917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f267c0cee00 msgr2=0x7f267c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.559+0000 7f26811fa640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f267c0cee00 0x7f267c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.559+0000 7f26811fa640 1 -- 172.21.15.55:0/2453954917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c0ccf80 con 0x7f267c15f690 2024-05-20T22:08:14.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.560+0000 7f26811fa640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c15f690 0x7f267c1446c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2678004e00 tx=0x7f2678007430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.560+0000 7f215e3b5640 1 -- 172.21.15.55:0/3536354578 wait complete. 2024-05-20T22:08:14.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.559+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f166c069090 con 0x7f166c2012e0 2024-05-20T22:08:14.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.560+0000 7fdd1360d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c09a260 0x7fdd0c152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.560+0000 7f80c7fff640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] conn(0x7f80c0075570 0x7f80c0077a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.560+0000 7f80c7fff640 1 -- 172.21.15.55:0/3168802828 --> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80d0024ec0 con 0x7f80c0075570 2024-05-20T22:08:14.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.560+0000 7fdd1360d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd0c15f660 0x7fdd0c151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.560+0000 7f80c7fff640 1 -- 172.21.15.55:0/3168802828 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f80d0092050 con 0x7f80cc004800 2024-05-20T22:08:14.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7f80dbfff640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] conn(0x7f80c0075570 0x7f80c0077a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd1360d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd0c152390 0x7fdd0c15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7f16537fe640 1 -- 172.21.15.55:0/4122763354 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1660044020 con 0x7f166c2012e0 2024-05-20T22:08:14.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd113a9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c09a260 0x7fdd0c152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd10ba8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd0c15f660 0x7fdd0c151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd10ba8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd0c15f660 0x7fdd0c151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50854/0 (socket says 172.21.15.55:50854) 2024-05-20T22:08:14.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd113a9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c09a260 0x7fdd0c152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47190/0 (socket says 172.21.15.55:47190) 2024-05-20T22:08:14.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7f80dbfff640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] conn(0x7f80c0075570 0x7f80c0077a00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7f16537fe640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f165806bf00 0x7f165806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7f166affd640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f165806bf00 0x7f165806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd10ba8640 1 -- 172.21.15.55:0/872528963 learned_addr learned my addr 172.21.15.55:0/872528963 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c15b890 con 0x7fdd0c152390 2024-05-20T22:08:14.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c0b06d0 con 0x7fdd0c15f660 2024-05-20T22:08:14.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0c0af6b0 con 0x7fdd0c09a260 2024-05-20T22:08:14.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd11baa640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd0c152390 0x7fdd0c15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7fdd10ba8640 1 -- 172.21.15.55:0/872528963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c09a260 msgr2=0x7fdd0c152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7fdd10ba8640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd0c09a260 0x7fdd0c152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7fdd10ba8640 1 -- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd0c152390 msgr2=0x7fdd0c15ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7fdd10ba8640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd0c152390 0x7fdd0c15ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7fdd10ba8640 1 -- 172.21.15.55:0/872528963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c1539f0 con 0x7fdd0c15f660 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7fdd10ba8640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd0c15f660 0x7fdd0c151e50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd0802a630 tx=0x7fdd080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7f72e09d5640 1 -- 172.21.15.55:0/1200225792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 msgr2=0x7f72dc15b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7f72e09d5640 1 --2- 172.21.15.55:0/1200225792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc15b090 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009870 tx=0x7f72d002a210 comp rx=0 tx=0).stop 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7f72e09d5640 1 -- 172.21.15.55:0/1200225792 shutdown_connections 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7f72e09d5640 1 --2- 172.21.15.55:0/1200225792 >> v2:172.21.15.55:3300/0 conn(0x7f72dc15f670 0x7f72dc06bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7f72e09d5640 1 --2- 172.21.15.55:0/1200225792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc15b090 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7f72e09d5640 1 -- 172.21.15.55:0/1200225792 >> 172.21.15.55:0/1200225792 conn(0x7f72dc0b3980 msgr2=0x7f72dc0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.560+0000 7f26727fc640 1 -- 172.21.15.55:0/2453954917 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780209c0 con 0x7f267c15f690 2024-05-20T22:08:14.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.560+0000 7f26727fc640 1 -- 172.21.15.55:0/2453954917 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2678029070 con 0x7f267c15f690 2024-05-20T22:08:14.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7f26727fc640 1 -- 172.21.15.55:0/2453954917 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678020d60 con 0x7f267c15f690 2024-05-20T22:08:14.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7f2683c5f640 1 -- 172.21.15.55:0/2453954917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c0cd220 con 0x7f267c15f690 2024-05-20T22:08:14.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.561+0000 7f2683c5f640 1 -- 172.21.15.55:0/2453954917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c0cfdd0 con 0x7f267c15f690 2024-05-20T22:08:14.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7f72e09d5640 1 -- 172.21.15.55:0/1200225792 shutdown_connections 2024-05-20T22:08:14.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.563+0000 7f80c7fff640 1 -- 172.21.15.55:0/3168802828 <== osd.1 v2:172.21.15.55:6816/518932600 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80d0024ec0 con 0x7f80c0075570 2024-05-20T22:08:14.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.563+0000 7f265ffff640 1 -- 172.21.15.55:0/2453954917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2658001030 con 0x7f267c15f690 2024-05-20T22:08:14.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.563+0000 7f166affd640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f165806bf00 0x7f165806e410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1664009f80 tx=0x7f166400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.563+0000 7f16537fe640 1 -- 172.21.15.55:0/4122763354 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f16600ab1f0 con 0x7f166c2012e0 2024-05-20T22:08:14.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.563+0000 7f215e3b5640 1 Processor -- start 2024-05-20T22:08:14.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.563+0000 7f215e3b5640 1 -- start start 2024-05-20T22:08:14.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.564+0000 7f16537fe640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] conn(0x7f1658075650 0x7f1658077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.564+0000 7f16537fe640 1 -- 172.21.15.55:0/4122763354 --> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1660042ec0 con 0x7f1658075650 2024-05-20T22:08:14.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.564+0000 7f16537fe640 1 -- 172.21.15.55:0/4122763354 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f16600ab5e0 con 0x7f166c2012e0 2024-05-20T22:08:14.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.562+0000 7f72e09d5640 1 -- 172.21.15.55:0/1200225792 wait complete. 2024-05-20T22:08:14.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.563+0000 7fdd01ffb640 1 -- 172.21.15.55:0/872528963 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08043070 con 0x7fdd0c15f660 2024-05-20T22:08:14.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.563+0000 7fdd01ffb640 1 -- 172.21.15.55:0/872528963 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08038040 con 0x7fdd0c15f660 2024-05-20T22:08:14.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.563+0000 7fdd01ffb640 1 -- 172.21.15.55:0/872528963 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08037050 con 0x7fdd0c15f660 2024-05-20T22:08:14.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.564+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c153cd0 con 0x7fdd0c15f660 2024-05-20T22:08:14.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.564+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c154290 con 0x7fdd0c15f660 2024-05-20T22:08:14.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.564+0000 7f26727fc640 1 -- 172.21.15.55:0/2453954917 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2678023070 con 0x7f267c15f690 2024-05-20T22:08:14.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.565+0000 7f26727fc640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266006bf00 0x7f266006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.565+0000 7f166b7fe640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] conn(0x7f1658075650 0x7f1658077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f215e3b5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f215815a6f0 0x7f21581f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f215e3b5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2158163a50 0x7f2158153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f72e09d5640 1 Processor -- start 2024-05-20T22:08:14.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd0c069090 con 0x7fdd0c15f660 2024-05-20T22:08:14.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f215e3b5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2158145ec0 0x7f21581455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f21577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2158163a50 0x7f2158153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f21577fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2158163a50 0x7f2158153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50864/0 (socket says 172.21.15.55:50864) 2024-05-20T22:08:14.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f215c952640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2158145ec0 0x7f21581455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f215c952640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2158145ec0 0x7f21581455d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58030/0 (socket says 172.21.15.55:58030) 2024-05-20T22:08:14.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f21577fe640 1 -- 172.21.15.55:0/559909518 learned_addr learned my addr 172.21.15.55:0/559909518 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f21580ae090 con 0x7f2158145ec0 2024-05-20T22:08:14.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f215806c900 con 0x7f2158163a50 2024-05-20T22:08:14.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.567+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f21580ae210 con 0x7f215815a6f0 2024-05-20T22:08:14.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.565+0000 7f26819fb640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266006bf00 0x7f266006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.565+0000 7f26727fc640 1 -- 172.21.15.55:0/2453954917 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f267809bdf0 con 0x7f267c15f690 2024-05-20T22:08:14.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f26727fc640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] conn(0x7f26600756c0 0x7f2660077b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f26727fc640 1 -- 172.21.15.55:0/2453954917 --> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2678026e70 con 0x7f26600756c0 2024-05-20T22:08:14.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f26727fc640 1 -- 172.21.15.55:0/2453954917 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f267809e450 con 0x7f267c15f690 2024-05-20T22:08:14.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.567+0000 7f26819fb640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266006bf00 0x7f266006e410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2674033040 tx=0x7f267402f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.567+0000 7f166b7fe640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] conn(0x7f1658075650 0x7f1658077ae0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.567+0000 7f21577fe640 1 -- 172.21.15.55:0/559909518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f215815a6f0 msgr2=0x7f21581f9790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T22:08:14.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.567+0000 7f21577fe640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f215815a6f0 0x7f21581f9790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.567+0000 7fdd01ffb640 1 -- 172.21.15.55:0/872528963 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd0803cab0 con 0x7fdd0c15f660 2024-05-20T22:08:14.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.567+0000 7f21577fe640 1 -- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2158145ec0 msgr2=0x7f21581455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.567+0000 7f21577fe640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2158145ec0 0x7f21581455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7fdd01ffb640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcf006bf30 0x7fdcf006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.567+0000 7f21577fe640 1 -- 172.21.15.55:0/559909518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2158153e90 con 0x7f2158163a50 2024-05-20T22:08:14.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7f21577fe640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2158163a50 0x7f2158153950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f215000c8c0 tx=0x7f215000cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7f2154ff9640 1 -- 172.21.15.55:0/559909518 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150003b90 con 0x7f2158163a50 2024-05-20T22:08:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7f2154ff9640 1 -- 172.21.15.55:0/559909518 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2150003d30 con 0x7f2158163a50 2024-05-20T22:08:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.566+0000 7f72e09d5640 1 -- start start 2024-05-20T22:08:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7f72e09d5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc06c490 0x7f72dc13d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7fdd01ffb640 1 -- 172.21.15.55:0/872528963 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdd080ab430 con 0x7fdd0c15f660 2024-05-20T22:08:14.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7fdd01ffb640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x7fdcf0075700 0x7fdcf0077b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7fdd01ffb640 1 -- 172.21.15.55:0/872528963 --> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd0802de50 con 0x7fdcf0075700 2024-05-20T22:08:14.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7fdd01ffb640 1 -- 172.21.15.55:0/872528963 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdd080ab820 con 0x7fdd0c15f660 2024-05-20T22:08:14.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7f2154ff9640 1 -- 172.21.15.55:0/559909518 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215000aa90 con 0x7f2158163a50 2024-05-20T22:08:14.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7f72e09d5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7f72da59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc06c490 0x7f72dc13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21581fa030 con 0x7f2158163a50 2024-05-20T22:08:14.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21581fa5f0 con 0x7f2158163a50 2024-05-20T22:08:14.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.568+0000 7f72da59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc06c490 0x7f72dc13d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50878/0 (socket says 172.21.15.55:50878) 2024-05-20T22:08:14.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7f16537fe640 1 -- 172.21.15.55:0/4122763354 <== osd.2 v2:172.21.15.55:6800/3066480496 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1660042ec0 con 0x7f1658075650 2024-05-20T22:08:14.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7f26821fc640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] conn(0x7f26600756c0 0x7f2660077b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7fdd11baa640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x7fdcf0075700 0x7fdcf0077b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7fdd113a9640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcf006bf30 0x7fdcf006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7fdd113a9640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcf006bf30 0x7fdcf006e440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c15eeb0 tx=0x7fdcfc009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7f72d9d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7f72d9d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47206/0 (socket says 172.21.15.55:47206) 2024-05-20T22:08:14.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7f72e09d5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72dc15f670 0x7f72dc144c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7f72e09d5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc0d0ea0 con 0x7f72dc15f670 2024-05-20T22:08:14.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7f72e09d5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc163160 con 0x7f72dc06c490 2024-05-20T22:08:14.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7f72e09d5640 1 -- 172.21.15.55:0/3748007883 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f72dc0d1020 con 0x7f72dc070780 2024-05-20T22:08:14.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.569+0000 7f72da59c640 1 -- 172.21.15.55:0/3748007883 learned_addr learned my addr 172.21.15.55:0/3748007883 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7f26821fc640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] conn(0x7f26600756c0 0x7f2660077b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2158069090 con 0x7f2158163a50 2024-05-20T22:08:14.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7fdd11baa640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x7fdcf0075700 0x7fdcf0077b90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7f72dad9d640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72dc15f670 0x7f72dc144c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7f72d9d9b640 1 -- 172.21.15.55:0/3748007883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc06c490 msgr2=0x7f72dc13d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7f72d9d9b640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc06c490 0x7f72dc13d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7f72d9d9b640 1 -- 172.21.15.55:0/3748007883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72dc15f670 msgr2=0x7f72dc144c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7f72d9d9b640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72dc15f670 0x7f72dc144c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7f72d9d9b640 1 -- 172.21.15.55:0/3748007883 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc0af410 con 0x7f72dc070780 2024-05-20T22:08:14.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.570+0000 7f72dad9d640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72dc15f670 0x7f72dc144c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:14.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.571+0000 7f72d9d9b640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc144720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f72cc007db0 tx=0x7f72cc00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.571+0000 7f26727fc640 1 -- 172.21.15.55:0/2453954917 <== osd.6 v2:172.21.15.138:6824/1454288856 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2678026e70 con 0x7f26600756c0 2024-05-20T22:08:14.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.571+0000 7f2154ff9640 1 -- 172.21.15.55:0/559909518 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f215000ac30 con 0x7f2158163a50 2024-05-20T22:08:14.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.571+0000 7f72caffd640 1 -- 172.21.15.55:0/3748007883 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc014420 con 0x7f72dc070780 2024-05-20T22:08:14.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.571+0000 7f72caffd640 1 -- 172.21.15.55:0/3748007883 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72cc019070 con 0x7f72dc070780 2024-05-20T22:08:14.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.571+0000 7f72e09d5640 1 -- 172.21.15.55:0/3748007883 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72dc0af6f0 con 0x7f72dc070780 2024-05-20T22:08:14.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.572+0000 7f2154ff9640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f214406be80 0x7f214406e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.572+0000 7f2157fff640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f214406be80 0x7f214406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.572+0000 7fdd01ffb640 1 -- 172.21.15.55:0/872528963 <== osd.3 v2:172.21.15.55:6808/3969115886 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd0802de50 con 0x7fdcf0075700 2024-05-20T22:08:14.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.572+0000 7f72e09d5640 1 -- 172.21.15.55:0/3748007883 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f72dc09d1e0 con 0x7f72dc070780 2024-05-20T22:08:14.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.573+0000 7f2154ff9640 1 -- 172.21.15.55:0/559909518 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f215008b1d0 con 0x7f2158163a50 2024-05-20T22:08:14.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.573+0000 7f2154ff9640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] conn(0x7f21440755d0 0x7f2144077a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.573+0000 7f2154ff9640 1 -- 172.21.15.55:0/559909518 --> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2150010e50 con 0x7f21440755d0 2024-05-20T22:08:14.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.573+0000 7f2157fff640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f214406be80 0x7f214406e390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2148001530 tx=0x7f2148007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.573+0000 7f2154ff9640 1 -- 172.21.15.55:0/559909518 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2150092050 con 0x7f2158163a50 2024-05-20T22:08:14.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.574+0000 7f215c952640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] conn(0x7f21440755d0 0x7f2144077a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.574+0000 7f72c8ff9640 1 -- 172.21.15.55:0/3748007883 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f72c0001030 con 0x7f72dc070780 2024-05-20T22:08:14.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.575+0000 7f215c952640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] conn(0x7f21440755d0 0x7f2144077a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.576+0000 7f72caffd640 1 -- 172.21.15.55:0/3748007883 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0147e0 con 0x7f72dc070780 2024-05-20T22:08:14.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.576+0000 7f72caffd640 1 -- 172.21.15.55:0/3748007883 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72cc014a00 con 0x7f72dc070780 2024-05-20T22:08:14.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.576+0000 7f2154ff9640 1 -- 172.21.15.55:0/559909518 <== osd.0 v2:172.21.15.55:6824/2576057485 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2150010e50 con 0x7f21440755d0 2024-05-20T22:08:14.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.577+0000 7f72caffd640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f72bc06bf60 0x7f72bc06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.577+0000 7f72da59c640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f72bc06bf60 0x7f72bc06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.578+0000 7f72da59c640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f72bc06bf60 0x7f72bc06e470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f72d002a250 tx=0x7f72d002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.579+0000 7f72caffd640 1 -- 172.21.15.55:0/3748007883 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f72cc08b960 con 0x7f72dc070780 2024-05-20T22:08:14.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.579+0000 7f72caffd640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] conn(0x7f72bc0756a0 0x7f72bc077b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.579+0000 7f72caffd640 1 -- 172.21.15.55:0/3748007883 --> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72cc00ce50 con 0x7f72bc0756a0 2024-05-20T22:08:14.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.579+0000 7f72caffd640 1 -- 172.21.15.55:0/3748007883 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72cc090220 con 0x7f72dc070780 2024-05-20T22:08:14.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.579+0000 7f72dad9d640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] conn(0x7f72bc0756a0 0x7f72bc077b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.580+0000 7f72dad9d640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] conn(0x7f72bc0756a0 0x7f72bc077b30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.581+0000 7f72caffd640 1 -- 172.21.15.55:0/3748007883 <== osd.4 v2:172.21.15.138:6808/3059706956 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72cc00ce50 con 0x7f72bc0756a0 2024-05-20T22:08:14.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.594+0000 7f1d96338640 1 Processor -- start 2024-05-20T22:08:14.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.594+0000 7f1d96338640 1 -- start start 2024-05-20T22:08:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.595+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 --> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f80d404efb0 con 0x7f80c0075570 2024-05-20T22:08:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.596+0000 7f80c7fff640 1 -- 172.21.15.55:0/3168802828 <== osd.1 v2:172.21.15.55:6816/518932600 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f80d404efb0 con 0x7f80c0075570 2024-05-20T22:08:14.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.596+0000 7f1d96338640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1d900c45c0 0x7f1d900c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.596+0000 7f1d96338640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d900c4f10 con 0x7f1d90163760 2024-05-20T22:08:14.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.596+0000 7f1d96338640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d900b2f60 con 0x7f1d900cfc90 2024-05-20T22:08:14.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.596+0000 7f1d96338640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1d900b4000 con 0x7f1d900c45c0 2024-05-20T22:08:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.596+0000 7f1d8ffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1d90163760 0x7f1d9006e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46388/0 (socket says 172.21.15.55:46388) 2024-05-20T22:08:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.596+0000 7f1d8ffff640 1 -- 172.21.15.55:0/3491312928 learned_addr learned my addr 172.21.15.55:0/3491312928 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.597+0000 7f1d8f7fe640 1 --2- 172.21.15.55:0/3491312928 >> v2:172.21.15.138:3300/0 conn(0x7f1d900c45c0 0x7f1d900c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.597+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] conn(0x7f80c0075570 msgr2=0x7f80c0077a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.597+0000 7f80e0b55640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] conn(0x7f80c0075570 0x7f80c0077a00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.597+0000 7f1d8f7fe640 1 -- 172.21.15.55:0/3491312928 >> v1:172.21.15.138:6789/0 conn(0x7f1d900cfc90 legacy=0x7f1d900d00a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T22:08:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.597+0000 7f1d8f7fe640 1 -- 172.21.15.55:0/3491312928 >> v1:172.21.15.55:6789/0 conn(0x7f1d90163760 legacy=0x7f1d9006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.597+0000 7f1d8f7fe640 1 -- 172.21.15.55:0/3491312928 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d900b3240 con 0x7f1d900c45c0 2024-05-20T22:08:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.598+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f80c006be30 msgr2=0x7f80c006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.598+0000 7f80e0b55640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f80c006be30 0x7f80c006e340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80c8033000 tx=0x7f80c8009030 comp rx=0 tx=0).stop 2024-05-20T22:08:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.598+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 msgr2=0x7f80d404b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.598+0000 7f80e0b55640 1 --2- 172.21.15.55:0/3168802828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80cc004800 0x7f80d404b220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f80d00107c0 tx=0x7f80d0010c80 comp rx=0 tx=0).stop 2024-05-20T22:08:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.598+0000 7f80dbfff640 1 -- 172.21.15.55:0/3168802828 reap_dead start 2024-05-20T22:08:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.598+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 shutdown_connections 2024-05-20T22:08:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.598+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 >> 172.21.15.55:0/3168802828 conn(0x7f80d4054ce0 msgr2=0x7f80d404f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.598+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 shutdown_connections 2024-05-20T22:08:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.599+0000 7f80e0b55640 1 -- 172.21.15.55:0/3168802828 wait complete. 2024-05-20T22:08:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.599+0000 7f1d8f7fe640 1 --2- 172.21.15.55:0/3491312928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d900c45c0 0x7f1d900c49d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0097a0 tx=0x7f1d7c02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97bc46d434ad9e1d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.599+0000 7f1d8effd640 1 -- 172.21.15.55:0/3491312928 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c037070 con 0x7f1d900c45c0 2024-05-20T22:08:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.599+0000 7f1d8effd640 1 -- 172.21.15.55:0/3491312928 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d7c032440 con 0x7f1d900c45c0 2024-05-20T22:08:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.600+0000 7fb09daa8640 1 Processor -- start 2024-05-20T22:08:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.601+0000 7f1d96338640 1 -- 172.21.15.55:0/3491312928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d900c45c0 msgr2=0x7f1d900c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.601+0000 7f1d96338640 1 --2- 172.21.15.55:0/3491312928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d900c45c0 0x7f1d900c49d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0097a0 tx=0x7f1d7c02a1e0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.601+0000 7f1d96338640 1 -- 172.21.15.55:0/3491312928 shutdown_connections 2024-05-20T22:08:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.601+0000 7f1d96338640 1 --2- 172.21.15.55:0/3491312928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d900c45c0 0x7f1d900c49d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.601+0000 7f1d96338640 1 -- 172.21.15.55:0/3491312928 >> 172.21.15.55:0/3491312928 conn(0x7f1d9009a260 msgr2=0x7f1d9009a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.601+0000 7f1d96338640 1 -- 172.21.15.55:0/3491312928 shutdown_connections 2024-05-20T22:08:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.602+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 --> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f166c15d0b0 con 0x7f1658075650 2024-05-20T22:08:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.603+0000 7f16537fe640 1 -- 172.21.15.55:0/4122763354 <== osd.2 v2:172.21.15.55:6800/3066480496 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f166c15d0b0 con 0x7f1658075650 2024-05-20T22:08:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.600+0000 7fb09daa8640 1 -- start start 2024-05-20T22:08:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.603+0000 7fb09daa8640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb09806b500 0x7fb098163360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.603+0000 7fb09daa8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980b8260 0x7fb0980b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.603+0000 7fb09daa8640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09806bc70 con 0x7fb09815f280 2024-05-20T22:08:14.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.603+0000 7fb09daa8640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fb09806c880 con 0x7fb09806b500 2024-05-20T22:08:14.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.603+0000 7fb09daa8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb09806bf50 con 0x7fb0980b8260 2024-05-20T22:08:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.603+0000 7fb0977fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb09815f280 0x7fb098162c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46400/0 (socket says 172.21.15.55:46400) 2024-05-20T22:08:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.603+0000 7fb0977fe640 1 -- 172.21.15.55:0/1460448136 learned_addr learned my addr 172.21.15.55:0/1460448136 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7fb096ffd640 1 --2- 172.21.15.55:0/1460448136 >> v2:172.21.15.55:3300/0 conn(0x7fb09806b500 0x7fb098163360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1345436040 0 0) 0x7fb09806bc70 con 0x7fb09815f280 2024-05-20T22:08:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb080003520 con 0x7fb09815f280 2024-05-20T22:08:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3654328234 0 0) 0x7fb080003520 con 0x7fb09815f280 2024-05-20T22:08:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980b8260 msgr2=0x7fb0980b8670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T22:08:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7fb0967fc640 1 --2- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980b8260 0x7fb0980b8670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.601+0000 7f1d96338640 1 -- 172.21.15.55:0/3491312928 wait complete. 2024-05-20T22:08:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7f1d96338640 1 Processor -- start 2024-05-20T22:08:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 >> v2:172.21.15.55:3300/0 conn(0x7fb09806b500 msgr2=0x7fb098163360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7fb0967fc640 1 --2- 172.21.15.55:0/1460448136 >> v2:172.21.15.55:3300/0 conn(0x7fb09806b500 0x7fb098163360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] conn(0x7f1658075650 msgr2=0x7f1658077ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7f72c8ff9640 1 -- 172.21.15.55:0/3748007883 --> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f72c0002dd0 con 0x7f72bc0756a0 2024-05-20T22:08:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09806c230 con 0x7fb09815f280 2024-05-20T22:08:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.604+0000 7f1671479640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] conn(0x7f1658075650 0x7f1658077ae0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f165806bf00 msgr2=0x7f165806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7f1671479640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f165806bf00 0x7f165806e410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1664009f80 tx=0x7f166400b040 comp rx=0 tx=0).stop 2024-05-20T22:08:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c2012e0 msgr2=0x7f166c0b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fb08c001670 con 0x7fb09815f280 2024-05-20T22:08:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7f1671479640 1 --2- 172.21.15.55:0/4122763354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f166c2012e0 0x7f166c0b00b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1660034000 tx=0x7f1660001890 comp rx=0 tx=0).stop 2024-05-20T22:08:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 >> v1:172.21.15.55:6789/0 conn(0x7fb09815f280 legacy=0x7fb098162c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7f72caffd640 1 -- 172.21.15.55:0/3748007883 <== osd.4 v2:172.21.15.138:6808/3059706956 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f72c0002dd0 con 0x7f72bc0756a0 2024-05-20T22:08:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7f166b7fe640 1 -- 172.21.15.55:0/4122763354 reap_dead start 2024-05-20T22:08:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 shutdown_connections 2024-05-20T22:08:14.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 >> 172.21.15.55:0/4122763354 conn(0x7f166c0737c0 msgr2=0x7f166c0cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7fb0967fc640 1 --2- 172.21.15.55:0/1460448136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb080003a30 0x7fb080003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7fb0967fc640 1 --2- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980b8260 0x7fb080004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb0967fc640 1 --2- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb080004d40 0x7fb0800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb09806c880 con 0x7fb080003a30 2024-05-20T22:08:14.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb09806bf50 con 0x7fb080004d40 2024-05-20T22:08:14.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7f72c8ff9640 1 -- 172.21.15.55:0/3748007883 >> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] conn(0x7f72bc0756a0 msgr2=0x7f72bc077b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7f72c8ff9640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] conn(0x7f72bc0756a0 0x7f72bc077b30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb08000bc00 con 0x7fb0980b8260 2024-05-20T22:08:14.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb096ffd640 1 --2- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980b8260 0x7fb080004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb097fff640 1 --2- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb080004d40 0x7fb0800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 --> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdd0c1537a0 con 0x7fdcf0075700 2024-05-20T22:08:14.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb097fff640 1 -- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980b8260 msgr2=0x7fb080004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7f72c8ff9640 1 -- 172.21.15.55:0/3748007883 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f72bc06bf60 msgr2=0x7f72bc06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb097fff640 1 --2- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980b8260 0x7fb080004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb097fff640 1 -- 172.21.15.55:0/1460448136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb080003a30 msgr2=0x7fb080003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb097fff640 1 --2- 172.21.15.55:0/1460448136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb080003a30 0x7fb080003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7fb097fff640 1 -- 172.21.15.55:0/1460448136 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb080003520 con 0x7fb080004d40 2024-05-20T22:08:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7f72c8ff9640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f72bc06bf60 0x7f72bc06e470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f72d002a250 tx=0x7f72d002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7f72c8ff9640 1 -- 172.21.15.55:0/3748007883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 msgr2=0x7f72dc144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7f72c8ff9640 1 --2- 172.21.15.55:0/3748007883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72dc070780 0x7f72dc144720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f72cc007db0 tx=0x7f72cc00e4b0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.606+0000 7f72dad9d640 1 -- 172.21.15.55:0/3748007883 reap_dead start 2024-05-20T22:08:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.605+0000 7f1d96338640 1 -- start start 2024-05-20T22:08:14.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.607+0000 7f72c8ff9640 1 -- 172.21.15.55:0/3748007883 shutdown_connections 2024-05-20T22:08:14.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.607+0000 7f72c8ff9640 1 -- 172.21.15.55:0/3748007883 >> 172.21.15.55:0/3748007883 conn(0x7f72dc0b3980 msgr2=0x7f72dc06f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.607+0000 7f72c8ff9640 1 -- 172.21.15.55:0/3748007883 shutdown_connections 2024-05-20T22:08:14.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.607+0000 7f72c8ff9640 1 -- 172.21.15.55:0/3748007883 wait complete. 2024-05-20T22:08:14.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.607+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 shutdown_connections 2024-05-20T22:08:14.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.607+0000 7f1671479640 1 -- 172.21.15.55:0/4122763354 wait complete. 2024-05-20T22:08:14.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.607+0000 7fdd01ffb640 1 -- 172.21.15.55:0/872528963 <== osd.3 v2:172.21.15.55:6808/3969115886 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdd0c1537a0 con 0x7fdcf0075700 2024-05-20T22:08:14.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.607+0000 7fb097fff640 1 --2- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb080004d40 0x7fb0800051d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb0980d48b0 tx=0x7fb080005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beb7f34842ad3d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7f1d96338640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d900c45c0 0x7f1d90201590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x7fdcf0075700 msgr2=0x7fdcf0077b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7fdd1360d640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x7fdcf0075700 0x7fdcf0077b90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7f1d96338640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d900cfc90 0x7f1d900c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088037070 con 0x7fb080004d40 2024-05-20T22:08:14.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb08802a9b0 con 0x7fb080004d40 2024-05-20T22:08:14.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7fb0967fc640 1 -- 172.21.15.55:0/1460448136 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880093f0 con 0x7fb080004d40 2024-05-20T22:08:14.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7f1d8ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d900c45c0 0x7f1d90201590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7f1d8ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d900c45c0 0x7f1d90201590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50904/0 (socket says 172.21.15.55:50904) 2024-05-20T22:08:14.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7f1d8f7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d900cfc90 0x7f1d900c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7f1d8f7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d900cfc90 0x7f1d900c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47238/0 (socket says 172.21.15.55:47238) 2024-05-20T22:08:14.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcf006bf30 msgr2=0x7fdcf006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.608+0000 7fdd1360d640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcf006bf30 0x7fdcf006e440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c15eeb0 tx=0x7fdcfc009060 comp rx=0 tx=0).stop 2024-05-20T22:08:14.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd0c15f660 msgr2=0x7fdd0c151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7fdd1360d640 1 --2- 172.21.15.55:0/872528963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd0c15f660 0x7fdd0c151e50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd0802a630 tx=0x7fdd080046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7fdd11baa640 1 -- 172.21.15.55:0/872528963 reap_dead start 2024-05-20T22:08:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7f1d96338640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d90163760 0x7f1d900af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7f1d8ffff640 1 -- 172.21.15.55:0/3389392513 learned_addr learned my addr 172.21.15.55:0/3389392513 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1d900b3ae0 con 0x7f1d90163760 2024-05-20T22:08:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d900b3800 con 0x7f1d900c45c0 2024-05-20T22:08:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1d900d1b60 con 0x7f1d900cfc90 2024-05-20T22:08:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 shutdown_connections 2024-05-20T22:08:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 >> 172.21.15.55:0/872528963 conn(0x7fdd0c0b62b0 msgr2=0x7fdd0c0b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 shutdown_connections 2024-05-20T22:08:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7f1d8f7fe640 1 -- 172.21.15.55:0/3389392513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d900c45c0 msgr2=0x7f1d90201590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7f1d8f7fe640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d900c45c0 0x7f1d90201590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7f1d8f7fe640 1 -- 172.21.15.55:0/3389392513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d90163760 msgr2=0x7f1d900af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7f265ffff640 1 -- 172.21.15.55:0/2453954917 --> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2658002dd0 con 0x7f26600756c0 2024-05-20T22:08:14.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7f1d948d5640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d90163760 0x7f1d900af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7fdd1360d640 1 -- 172.21.15.55:0/872528963 wait complete. 2024-05-20T22:08:14.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7fb09daa8640 1 -- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb080004d40 msgr2=0x7fb0800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7fb09daa8640 1 --2- 172.21.15.55:0/1460448136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb080004d40 0x7fb0800051d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb0980d48b0 tx=0x7fb080005830 comp rx=0 tx=0).stop 2024-05-20T22:08:14.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 --> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21581f9de0 con 0x7f21440755d0 2024-05-20T22:08:14.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7f1d8f7fe640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d90163760 0x7f1d900af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.609+0000 7fb097fff640 1 -- 172.21.15.55:0/1460448136 reap_dead start 2024-05-20T22:08:14.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7f1d8f7fe640 1 -- 172.21.15.55:0/3389392513 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d90201ad0 con 0x7f1d900cfc90 2024-05-20T22:08:14.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7f1d948d5640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d90163760 0x7f1d900af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:14.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7f26727fc640 1 -- 172.21.15.55:0/2453954917 <== osd.6 v2:172.21.15.138:6824/1454288856 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2658002dd0 con 0x7f26600756c0 2024-05-20T22:08:14.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7fb09daa8640 1 -- 172.21.15.55:0/1460448136 shutdown_connections 2024-05-20T22:08:14.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7fb09daa8640 1 -- 172.21.15.55:0/1460448136 >> 172.21.15.55:0/1460448136 conn(0x7fb0980d1e30 msgr2=0x7fb0980d4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7fb09daa8640 1 -- 172.21.15.55:0/1460448136 shutdown_connections 2024-05-20T22:08:14.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7f1d8f7fe640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d900cfc90 0x7f1d900c5b10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c031000 tx=0x7f1d7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7fb09daa8640 1 -- 172.21.15.55:0/1460448136 wait complete. 2024-05-20T22:08:14.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.610+0000 7f1d8cff9640 1 -- 172.21.15.55:0/3389392513 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c037050 con 0x7f1d900cfc90 2024-05-20T22:08:14.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.611+0000 7f1d8cff9640 1 -- 172.21.15.55:0/3389392513 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d7c038040 con 0x7f1d900cfc90 2024-05-20T22:08:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.611+0000 7f2154ff9640 1 -- 172.21.15.55:0/559909518 <== osd.0 v2:172.21.15.55:6824/2576057485 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21581f9de0 con 0x7f21440755d0 2024-05-20T22:08:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.611+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d900c67d0 con 0x7f1d900cfc90 2024-05-20T22:08:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.611+0000 7f1d8cff9640 1 -- 172.21.15.55:0/3389392513 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c03e450 con 0x7f1d900cfc90 2024-05-20T22:08:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.611+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d90160550 con 0x7f1d900cfc90 2024-05-20T22:08:14.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.613+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d900758b0 con 0x7f1d900cfc90 2024-05-20T22:08:14.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.613+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] conn(0x7f21440755d0 msgr2=0x7f2144077a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.613+0000 7f215e3b5640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] conn(0x7f21440755d0 0x7f2144077a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.613+0000 7f265ffff640 1 -- 172.21.15.55:0/2453954917 >> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] conn(0x7f26600756c0 msgr2=0x7f2660077b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.613+0000 7f265ffff640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] conn(0x7f26600756c0 0x7f2660077b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.613+0000 7fb09daa8640 1 Processor -- start 2024-05-20T22:08:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.613+0000 7f265ffff640 1 -- 172.21.15.55:0/2453954917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266006bf00 msgr2=0x7f266006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.613+0000 7f265ffff640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266006bf00 0x7f266006e410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2674033040 tx=0x7f267402f000 comp rx=0 tx=0).stop 2024-05-20T22:08:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.613+0000 7f265ffff640 1 -- 172.21.15.55:0/2453954917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c15f690 msgr2=0x7f267c1446c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.613+0000 7f265ffff640 1 --2- 172.21.15.55:0/2453954917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c15f690 0x7f267c1446c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2678004e00 tx=0x7f2678007430 comp rx=0 tx=0).stop 2024-05-20T22:08:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f26821fc640 1 -- 172.21.15.55:0/2453954917 reap_dead start 2024-05-20T22:08:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7fb09daa8640 1 -- start start 2024-05-20T22:08:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f214406be80 msgr2=0x7f214406e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f215e3b5640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f214406be80 0x7f214406e390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2148001530 tx=0x7f2148007a40 comp rx=0 tx=0).stop 2024-05-20T22:08:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2158163a50 msgr2=0x7f2158153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f215e3b5640 1 --2- 172.21.15.55:0/559909518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2158163a50 0x7f2158153950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f215000c8c0 tx=0x7f215000cd80 comp rx=0 tx=0).stop 2024-05-20T22:08:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f215c952640 1 -- 172.21.15.55:0/559909518 reap_dead start 2024-05-20T22:08:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 shutdown_connections 2024-05-20T22:08:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 >> 172.21.15.55:0/559909518 conn(0x7f21580b3980 msgr2=0x7f2158159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 shutdown_connections 2024-05-20T22:08:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f265ffff640 1 -- 172.21.15.55:0/2453954917 shutdown_connections 2024-05-20T22:08:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f265ffff640 1 -- 172.21.15.55:0/2453954917 >> 172.21.15.55:0/2453954917 conn(0x7f267c0d0730 msgr2=0x7f267c0af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.614+0000 7f215e3b5640 1 -- 172.21.15.55:0/559909518 wait complete. 2024-05-20T22:08:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.615+0000 7f1d8cff9640 1 -- 172.21.15.55:0/3389392513 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d7c045020 con 0x7f1d900cfc90 2024-05-20T22:08:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.615+0000 7f265ffff640 1 -- 172.21.15.55:0/2453954917 shutdown_connections 2024-05-20T22:08:14.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.615+0000 7f265ffff640 1 -- 172.21.15.55:0/2453954917 wait complete. 2024-05-20T22:08:14.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.616+0000 7f1d8cff9640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1d7406bf30 0x7f1d7406e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.616+0000 7f1d8ffff640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1d7406bf30 0x7f1d7406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.617+0000 7f1d8ffff640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1d7406bf30 0x7f1d7406e440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1d84005880 tx=0x7f1d84009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.617+0000 7f1d8cff9640 1 -- 172.21.15.55:0/3389392513 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1d7c0ab5e0 con 0x7f1d900cfc90 2024-05-20T22:08:14.656 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-20T22:08:14.656 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-20T22:08:14.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7f1d8cff9640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] conn(0x7f1d74075680 0x7f1d74077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7f1d8cff9640 1 -- 172.21.15.55:0/3389392513 --> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d7c00af30 con 0x7f1d74075680 2024-05-20T22:08:14.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7f1d8cff9640 1 -- 172.21.15.55:0/3389392513 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1d7c0b2050 con 0x7f1d900cfc90 2024-05-20T22:08:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.617+0000 7fb09daa8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb080004d40 0x7fb0980cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.617+0000 7fb09daa8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0981593b0 0x7fb09814bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.617+0000 7fb09daa8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb09815ad90 0x7fb09815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.617+0000 7fb09daa8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb09806cd10 con 0x7fb09815ad90 2024-05-20T22:08:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.617+0000 7fb09daa8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb09806b910 con 0x7fb080004d40 2024-05-20T22:08:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.617+0000 7fb09daa8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0980ce030 con 0x7fb0981593b0 2024-05-20T22:08:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.617+0000 7fb096ffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0981593b0 0x7fb09814bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7fb096ffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0981593b0 0x7fb09814bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47240/0 (socket says 172.21.15.55:47240) 2024-05-20T22:08:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7fb096ffd640 1 -- 172.21.15.55:0/1877969988 learned_addr learned my addr 172.21.15.55:0/1877969988 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7fb0977fe640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb080004d40 0x7fb0980cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7fb097fff640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb09815ad90 0x7fb09815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7fb096ffd640 1 -- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb080004d40 msgr2=0x7fb0980cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7fb096ffd640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb080004d40 0x7fb0980cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7fb096ffd640 1 -- 172.21.15.55:0/1877969988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb09815ad90 msgr2=0x7fb09815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7fb096ffd640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb09815ad90 0x7fb09815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7fb096ffd640 1 -- 172.21.15.55:0/1877969988 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09815b6e0 con 0x7fb0981593b0 2024-05-20T22:08:14.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.618+0000 7f1d948d5640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] conn(0x7f1d74075680 0x7f1d74077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.619+0000 7fb096ffd640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0981593b0 0x7fb09814bc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb090011e30 tx=0x7fb09000f580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.619+0000 7fb07ffff640 1 -- 172.21.15.55:0/1877969988 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090015630 con 0x7fb0981593b0 2024-05-20T22:08:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.619+0000 7fb07ffff640 1 -- 172.21.15.55:0/1877969988 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0900157d0 con 0x7fb0981593b0 2024-05-20T22:08:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.619+0000 7fb07ffff640 1 -- 172.21.15.55:0/1877969988 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090015ba0 con 0x7fb0981593b0 2024-05-20T22:08:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.619+0000 7f1d948d5640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] conn(0x7f1d74075680 0x7f1d74077b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.619+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb098159b90 con 0x7fb0981593b0 2024-05-20T22:08:14.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.620+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09815a150 con 0x7fb0981593b0 2024-05-20T22:08:14.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.620+0000 7f1d8cff9640 1 -- 172.21.15.55:0/3389392513 <== osd.5 v2:172.21.15.138:6800/2873568230 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d7c00af30 con 0x7f1d74075680 2024-05-20T22:08:14.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.622+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb098069090 con 0x7fb0981593b0 2024-05-20T22:08:14.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.622+0000 7fb07ffff640 1 -- 172.21.15.55:0/1877969988 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb09001d400 con 0x7fb0981593b0 2024-05-20T22:08:14.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.623+0000 7fb07ffff640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb08406be60 0x7fb08406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.623+0000 7fb0977fe640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb08406be60 0x7fb08406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.624+0000 7fb07ffff640 1 -- 172.21.15.55:0/1877969988 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb09008c290 con 0x7fb0981593b0 2024-05-20T22:08:14.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.624+0000 7fb07ffff640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] conn(0x7fb084075530 0x7fb0840779c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.624+0000 7fb07ffff640 1 -- 172.21.15.55:0/1877969988 --> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb090006b90 con 0x7fb084075530 2024-05-20T22:08:14.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.624+0000 7fb0977fe640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb08406be60 0x7fb08406e370 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb08c004ad0 tx=0x7fb08c007df0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.625+0000 7fb07ffff640 1 -- 172.21.15.55:0/1877969988 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb090093050 con 0x7fb0981593b0 2024-05-20T22:08:14.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.625+0000 7fb097fff640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] conn(0x7fb084075530 0x7fb0840779c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.625+0000 7fb097fff640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] conn(0x7fb084075530 0x7fb0840779c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.626+0000 7fb07ffff640 1 -- 172.21.15.55:0/1877969988 <== osd.7 v2:172.21.15.138:6816/2068592032 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb090006b90 con 0x7fb084075530 2024-05-20T22:08:14.663 INFO:teuthology.orchestra.run.smithi055.stdout:55834574859 2024-05-20T22:08:14.664 INFO:teuthology.orchestra.run.smithi055.stdout:55834574857 2024-05-20T22:08:14.664 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-20T22:08:14.665 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-20T22:08:14.665 INFO:teuthology.orchestra.run.smithi055.stdout:55834574859 2024-05-20T22:08:14.666 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-20T22:08:14.666 INFO:teuthology.orchestra.run.smithi055.stdout:55834574859 2024-05-20T22:08:14.667 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-20T22:08:14.667 INFO:teuthology.orchestra.run.smithi055.stdout:55834574859 2024-05-20T22:08:14.668 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-20T22:08:14.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.639+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 --> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d9009abe0 con 0x7f1d74075680 2024-05-20T22:08:14.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d8cff9640 1 -- 172.21.15.55:0/3389392513 <== osd.5 v2:172.21.15.138:6800/2873568230 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1d9009abe0 con 0x7f1d74075680 2024-05-20T22:08:14.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 >> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] conn(0x7f1d74075680 msgr2=0x7f1d74077b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d96338640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] conn(0x7f1d74075680 0x7f1d74077b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1d7406bf30 msgr2=0x7f1d7406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d96338640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1d7406bf30 0x7f1d7406e440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1d84005880 tx=0x7f1d84009040 comp rx=0 tx=0).stop 2024-05-20T22:08:14.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d900cfc90 msgr2=0x7f1d900c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d96338640 1 --2- 172.21.15.55:0/3389392513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1d900cfc90 0x7f1d900c5b10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c031000 tx=0x7f1d7c0046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d948d5640 1 -- 172.21.15.55:0/3389392513 reap_dead start 2024-05-20T22:08:14.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 shutdown_connections 2024-05-20T22:08:14.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 >> 172.21.15.55:0/3389392513 conn(0x7f1d9009a260 msgr2=0x7f1d900cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.640+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 shutdown_connections 2024-05-20T22:08:14.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.641+0000 7f1d96338640 1 -- 172.21.15.55:0/3389392513 wait complete. 2024-05-20T22:08:14.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.643+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 --> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb098159940 con 0x7fb084075530 2024-05-20T22:08:14.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb07ffff640 1 -- 172.21.15.55:0/1877969988 <== osd.7 v2:172.21.15.138:6816/2068592032 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb098159940 con 0x7fb084075530 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] conn(0x7fb084075530 msgr2=0x7fb0840779c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb09daa8640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] conn(0x7fb084075530 0x7fb0840779c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb08406be60 msgr2=0x7fb08406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb09daa8640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb08406be60 0x7fb08406e370 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb08c004ad0 tx=0x7fb08c007df0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0981593b0 msgr2=0x7fb09814bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb09daa8640 1 --2- 172.21.15.55:0/1877969988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0981593b0 0x7fb09814bc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb090011e30 tx=0x7fb09000f580 comp rx=0 tx=0).stop 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb097fff640 1 -- 172.21.15.55:0/1877969988 reap_dead start 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 shutdown_connections 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 >> 172.21.15.55:0/1877969988 conn(0x7fb0980d1e30 msgr2=0x7fb0980b9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 shutdown_connections 2024-05-20T22:08:14.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.644+0000 7fb09daa8640 1 -- 172.21.15.55:0/1877969988 wait complete. 2024-05-20T22:08:14.673 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-20T22:08:14.673 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-20T22:08:14.674 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-20T22:08:14.674 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-20T22:08:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.924+0000 7f094974b640 1 Processor -- start 2024-05-20T22:08:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.924+0000 7f094974b640 1 -- start start 2024-05-20T22:08:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f094974b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f09440b5420 0x7f094406fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f094974b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440af880 0x7f09440b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f094974b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09440afc90 0x7f09440b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f094974b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f094415be30 con 0x7f09440afc90 2024-05-20T22:08:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f094974b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0944161df0 con 0x7f09440b5420 2024-05-20T22:08:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f094974b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f094415b7e0 con 0x7f09440af880 2024-05-20T22:08:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f09437fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09440afc90 0x7f09440b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f09427fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440af880 0x7f09440b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f09437fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09440afc90 0x7f09440b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58092/0 (socket says 172.21.15.55:58092) 2024-05-20T22:08:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.928+0000 7f09427fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440af880 0x7f09440b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47246/0 (socket says 172.21.15.55:47246) 2024-05-20T22:08:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.928+0000 7f09427fc640 1 -- 172.21.15.55:0/1634646269 learned_addr learned my addr 172.21.15.55:0/1634646269 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.928+0000 7f0942ffd640 1 --2- 172.21.15.55:0/1634646269 >> v2:172.21.15.138:3300/0 conn(0x7f09440b5420 0x7f094406fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f83c6303640 1 Processor -- start 2024-05-20T22:08:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.927+0000 7f83c6303640 1 -- start start 2024-05-20T22:08:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.928+0000 7f09427fc640 1 -- 172.21.15.55:0/1634646269 >> v2:172.21.15.138:3300/0 conn(0x7f09440b5420 msgr2=0x7f094406fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.928+0000 7f09427fc640 1 --2- 172.21.15.55:0/1634646269 >> v2:172.21.15.138:3300/0 conn(0x7f09440b5420 0x7f094406fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.928+0000 7f09427fc640 1 -- 172.21.15.55:0/1634646269 >> v2:172.21.15.55:3300/0 conn(0x7f09440afc90 msgr2=0x7f09440b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.928+0000 7f09427fc640 1 --2- 172.21.15.55:0/1634646269 >> v2:172.21.15.55:3300/0 conn(0x7f09440afc90 0x7f09440b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.928+0000 7f09427fc640 1 -- 172.21.15.55:0/1634646269 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094415bac0 con 0x7f09440af880 2024-05-20T22:08:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.928+0000 7fbbfa63d640 1 Processor -- start 2024-05-20T22:08:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.928+0000 7fbbfa63d640 1 -- start start 2024-05-20T22:08:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.929+0000 7fd0c8808640 1 Processor -- start 2024-05-20T22:08:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.929+0000 7f010a43f640 1 Processor -- start 2024-05-20T22:08:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.929+0000 7f09427fc640 1 --2- 172.21.15.55:0/1634646269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440af880 0x7f09440b41c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0944071fe0 tx=0x7f093402a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8f015003027cc7a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.929+0000 7f010a43f640 1 -- start start 2024-05-20T22:08:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.930+0000 7f0941ffb640 1 -- 172.21.15.55:0/1634646269 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934037070 con 0x7f09440af880 2024-05-20T22:08:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.930+0000 7f0941ffb640 1 -- 172.21.15.55:0/1634646269 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0934032440 con 0x7f09440af880 2024-05-20T22:08:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.930+0000 7fbbfa63d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 0x7fbbf406f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.930+0000 7f83c6303640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f83b80e7de0 0x7f83b804fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.930+0000 7f83c6303640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83b8050390 0x7f83b8038340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.930+0000 7f83c6303640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b80507a0 con 0x7f83b804f440 2024-05-20T22:08:14.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.930+0000 7f83c6303640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f83b80e81f0 con 0x7f83b80e7de0 2024-05-20T22:08:14.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.930+0000 7f83c6303640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f83b80067a0 con 0x7f83b8050390 2024-05-20T22:08:14.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.929+0000 7fd0c8808640 1 -- start start 2024-05-20T22:08:14.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f83c4b00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f83b80e7de0 0x7f83b804fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f83c4b00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f83b80e7de0 0x7f83b804fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50932/0 (socket says 172.21.15.55:50932) 2024-05-20T22:08:14.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f83c4b00640 1 -- 172.21.15.55:0/2648439246 learned_addr learned my addr 172.21.15.55:0/2648439246 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f094974b640 1 -- 172.21.15.55:0/1634646269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440af880 msgr2=0x7f09440b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f094974b640 1 --2- 172.21.15.55:0/1634646269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440af880 0x7f09440b41c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0944071fe0 tx=0x7f093402a1d0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7fbbf3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 0x7fbbf406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7fbbf3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 0x7fbbf406f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47252/0 (socket says 172.21.15.55:47252) 2024-05-20T22:08:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7fbbf37fe640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fbbf406fcb0 0x7fbbf406b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:37798/0 (socket says 172.21.15.55:37798) 2024-05-20T22:08:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7fbbf3fff640 1 -- 172.21.15.55:0/2246624675 learned_addr learned my addr 172.21.15.55:0/2246624675 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7fbbfa63d640 1 -- 172.21.15.55:0/2246624675 --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf40700c0 con 0x7fbbf406bc10 2024-05-20T22:08:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7fbbfa63d640 1 -- 172.21.15.55:0/2246624675 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf406c0e0 con 0x7fbbf406fcb0 2024-05-20T22:08:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f094974b640 1 -- 172.21.15.55:0/1634646269 shutdown_connections 2024-05-20T22:08:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f094974b640 1 --2- 172.21.15.55:0/1634646269 >> v2:172.21.15.55:3300/0 conn(0x7f09440afc90 0x7f09440b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f094974b640 1 --2- 172.21.15.55:0/1634646269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440af880 0x7f09440b41c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f094974b640 1 --2- 172.21.15.55:0/1634646269 >> v2:172.21.15.138:3300/0 conn(0x7f09440b5420 0x7f094406fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f094974b640 1 -- 172.21.15.55:0/1634646269 >> 172.21.15.55:0/1634646269 conn(0x7f09440732e0 msgr2=0x7f094406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7fbbfa63d640 1 -- 172.21.15.55:0/2246624675 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf406ccf0 con 0x7fbbf40b4b70 2024-05-20T22:08:14.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f094974b640 1 -- 172.21.15.55:0/1634646269 shutdown_connections 2024-05-20T22:08:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f83b7fff640 1 -- 172.21.15.55:0/2648439246 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 476793911 0 0) 0x7f83c004fc90 con 0x7f83b804f440 2024-05-20T22:08:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f83b7fff640 1 -- 172.21.15.55:0/2648439246 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83ac003520 con 0x7f83b804f440 2024-05-20T22:08:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7f010a43f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f010415a6f0 0x7f0104073130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.931+0000 7f094974b640 1 -- 172.21.15.55:0/1634646269 wait complete. 2024-05-20T22:08:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fbbf3fff640 1 -- 172.21.15.55:0/2246624675 >> v1:172.21.15.138:6789/0 conn(0x7fbbf406fcb0 legacy=0x7fbbf406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fbbf3fff640 1 -- 172.21.15.55:0/2246624675 >> v1:172.21.15.55:6789/0 conn(0x7fbbf406bc10 legacy=0x7fbbf415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c8808640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd0c406f1a0 0x7fd0c40b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c8808640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c40af880 0x7fd0c40b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c8808640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0c40afc90 0x7fd0c40b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c8808640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fd0c415be30 con 0x7fd0c40afc90 2024-05-20T22:08:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c8808640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd0c40b5a00 con 0x7fd0c406f1a0 2024-05-20T22:08:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c8808640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c415b7e0 con 0x7fd0c40af880 2024-05-20T22:08:14.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c2d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0c40afc90 0x7fd0c40b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c2d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0c40afc90 0x7fd0c40b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58098/0 (socket says 172.21.15.55:58098) 2024-05-20T22:08:14.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c2d9d640 1 -- 172.21.15.55:0/3340572568 learned_addr learned my addr 172.21.15.55:0/3340572568 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c259c640 1 --2- 172.21.15.55:0/3340572568 >> v2:172.21.15.138:3300/0 conn(0x7fd0c406f1a0 0x7fd0c40b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fd0c1d9b640 1 --2- 172.21.15.55:0/3340572568 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c40af880 0x7fd0c40b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7f83c4b00640 1 -- 172.21.15.55:0/2648439246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83b8050390 msgr2=0x7f83b8038340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T22:08:14.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7f83c4b00640 1 --2- 172.21.15.55:0/2648439246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83b8050390 0x7f83b8038340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7f83c4b00640 1 -- 172.21.15.55:0/2648439246 >> v1:172.21.15.138:6789/0 conn(0x7f83b804f440 legacy=0x7f83b8006090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7fd0c259c640 1 -- 172.21.15.55:0/3340572568 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c40af880 msgr2=0x7fd0c40b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7f83c4b00640 1 -- 172.21.15.55:0/2648439246 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b8006a00 con 0x7f83b80e7de0 2024-05-20T22:08:14.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7f010a43f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f010406c2b0 0x7f0104163e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7f010a43f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0104072690 con 0x7f0104163a50 2024-05-20T22:08:14.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7f010a43f640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f0104164560 con 0x7f010415a6f0 2024-05-20T22:08:14.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7f010a43f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0104164840 con 0x7f010406c2b0 2024-05-20T22:08:14.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7f0103fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f010415a6f0 0x7f0104073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7fd0c259c640 1 --2- 172.21.15.55:0/3340572568 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c40af880 0x7fd0c40b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7fd0c259c640 1 -- 172.21.15.55:0/3340572568 >> v2:172.21.15.55:3300/0 conn(0x7fd0c40afc90 msgr2=0x7fd0c40b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7fd0c259c640 1 --2- 172.21.15.55:0/3340572568 >> v2:172.21.15.55:3300/0 conn(0x7fd0c40afc90 0x7fd0c40b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7f0103fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f010415a6f0 0x7f0104073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58096/0 (socket says 172.21.15.55:58096) 2024-05-20T22:08:14.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7f0103fff640 1 -- 172.21.15.55:0/2582725075 learned_addr learned my addr 172.21.15.55:0/2582725075 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7fd0c259c640 1 -- 172.21.15.55:0/3340572568 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c415bac0 con 0x7fd0c406f1a0 2024-05-20T22:08:14.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7f83c4b00640 1 --2- 172.21.15.55:0/2648439246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b80e7de0 0x7f83b804fe50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009870 tx=0x7f83bc02a310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=474aa98d5059797b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.932+0000 7fbbf3fff640 1 -- 172.21.15.55:0/2246624675 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf406c3c0 con 0x7fbbf40b4b70 2024-05-20T22:08:14.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.933+0000 7f01089dc640 1 --2- 172.21.15.55:0/2582725075 >> v2:172.21.15.138:3300/0 conn(0x7f010406c2b0 0x7f0104163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7f0103fff640 1 -- 172.21.15.55:0/2582725075 >> v2:172.21.15.138:3300/0 conn(0x7f010406c2b0 msgr2=0x7f0104163e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7f0103fff640 1 --2- 172.21.15.55:0/2582725075 >> v2:172.21.15.138:3300/0 conn(0x7f010406c2b0 0x7f0104163e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7f0103fff640 1 -- 172.21.15.55:0/2582725075 >> v1:172.21.15.138:6789/0 conn(0x7f0104163a50 legacy=0x7f010406bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7f0103fff640 1 -- 172.21.15.55:0/2582725075 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0104072930 con 0x7f010415a6f0 2024-05-20T22:08:14.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7fbbf3fff640 1 --2- 172.21.15.55:0/2246624675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 0x7fbbf406f770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbbf406ce70 tx=0x7fbbec02a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c6554f22fc41b62 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7f83b7fff640 1 -- 172.21.15.55:0/2648439246 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc037070 con 0x7f83b80e7de0 2024-05-20T22:08:14.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7f83b7fff640 1 -- 172.21.15.55:0/2648439246 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83bc032640 con 0x7f83b80e7de0 2024-05-20T22:08:14.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7f83b7fff640 1 -- 172.21.15.55:0/2648439246 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc02ae60 con 0x7f83b80e7de0 2024-05-20T22:08:14.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7fb3ed40a640 1 Processor -- start 2024-05-20T22:08:14.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7fbbf2ffd640 1 -- 172.21.15.55:0/2246624675 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec036070 con 0x7fbbf40b4b70 2024-05-20T22:08:14.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7fbbf2ffd640 1 -- 172.21.15.55:0/2246624675 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbec031600 con 0x7fbbf40b4b70 2024-05-20T22:08:14.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7fbbf2ffd640 1 -- 172.21.15.55:0/2246624675 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec0319d0 con 0x7fbbf40b4b70 2024-05-20T22:08:14.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.934+0000 7fb3ed40a640 1 -- start start 2024-05-20T22:08:14.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7fd0c259c640 1 --2- 172.21.15.55:0/3340572568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c406f1a0 0x7fd0c40b5420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4071fe0 tx=0x7fd0b802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd1702e8ef26e180 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7f094974b640 1 Processor -- start 2024-05-20T22:08:14.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7f094974b640 1 -- start start 2024-05-20T22:08:14.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7fd0c159a640 1 -- 172.21.15.55:0/3340572568 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8037070 con 0x7fd0c406f1a0 2024-05-20T22:08:14.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7fd0c159a640 1 -- 172.21.15.55:0/3340572568 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b8032440 con 0x7fd0c406f1a0 2024-05-20T22:08:14.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7fd0c159a640 1 -- 172.21.15.55:0/3340572568 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b802ae60 con 0x7fd0c406f1a0 2024-05-20T22:08:14.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7f83c6303640 1 -- 172.21.15.55:0/2648439246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b80e7de0 msgr2=0x7f83b804fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7f83c6303640 1 --2- 172.21.15.55:0/2648439246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b80e7de0 0x7f83b804fe50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009870 tx=0x7f83bc02a310 comp rx=0 tx=0).stop 2024-05-20T22:08:14.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7f0103fff640 1 --2- 172.21.15.55:0/2582725075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010415a6f0 0x7f0104073130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f01040d1990 tx=0x7f00fc02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504cc3fc7e7c3574 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7fbbfa63d640 1 -- 172.21.15.55:0/2246624675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 msgr2=0x7fbbf406f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7fbbfa63d640 1 --2- 172.21.15.55:0/2246624675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 0x7fbbf406f770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbbf406ce70 tx=0x7fbbec02a610 comp rx=0 tx=0).stop 2024-05-20T22:08:14.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7f0102ffd640 1 -- 172.21.15.55:0/2582725075 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc036070 con 0x7f010415a6f0 2024-05-20T22:08:14.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7fbbfa63d640 1 -- 172.21.15.55:0/2246624675 shutdown_connections 2024-05-20T22:08:14.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7fbbfa63d640 1 --2- 172.21.15.55:0/2246624675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 0x7fbbf406f770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7fbbfa63d640 1 -- 172.21.15.55:0/2246624675 >> 172.21.15.55:0/2246624675 conn(0x7fbbf40d06f0 msgr2=0x7fbbf40d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7f0102ffd640 1 -- 172.21.15.55:0/2582725075 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00fc031600 con 0x7f010415a6f0 2024-05-20T22:08:14.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.935+0000 7f0102ffd640 1 -- 172.21.15.55:0/2582725075 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc0319d0 con 0x7f010415a6f0 2024-05-20T22:08:14.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.936+0000 7f122e5ef640 1 Processor -- start 2024-05-20T22:08:14.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.936+0000 7f122e5ef640 1 -- start start 2024-05-20T22:08:14.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.936+0000 7fb3ed40a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 0x7fb3e004ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.936+0000 7fbbfa63d640 1 -- 172.21.15.55:0/2246624675 shutdown_connections 2024-05-20T22:08:14.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.936+0000 7fbbfa63d640 1 -- 172.21.15.55:0/2246624675 wait complete. 2024-05-20T22:08:14.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.936+0000 7fd0c8808640 1 -- 172.21.15.55:0/3340572568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c406f1a0 msgr2=0x7fd0c40b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.936+0000 7fd0c8808640 1 --2- 172.21.15.55:0/3340572568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c406f1a0 0x7fd0c40b5420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4071fe0 tx=0x7fd0b802a110 comp rx=0 tx=0).stop 2024-05-20T22:08:14.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.936+0000 7fb3ed40a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3e0053680 0x7fb3e0053a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fb3ed40a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e0050490 con 0x7fb3e00e7ed0 2024-05-20T22:08:14.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fb3ed40a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e003b510 con 0x7fb3e0053680 2024-05-20T22:08:14.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fb3e7fff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb3e00e7ed0 0x7fb3e00341d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:37810/0 (socket says 172.21.15.55:37810) 2024-05-20T22:08:14.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fb3e7fff640 1 -- 172.21.15.55:0/905535134 learned_addr learned my addr 172.21.15.55:0/905535134 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:14.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fb3ecc09640 1 --2- 172.21.15.55:0/905535134 >> v2:172.21.15.55:3300/0 conn(0x7fb3e0053680 0x7fb3e0053a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f83c6303640 1 -- 172.21.15.55:0/2648439246 shutdown_connections 2024-05-20T22:08:14.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f83c6303640 1 --2- 172.21.15.55:0/2648439246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83b8050390 0x7f83b8038340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f83c6303640 1 --2- 172.21.15.55:0/2648439246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b80e7de0 0x7f83b804fe50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f83c6303640 1 -- 172.21.15.55:0/2648439246 >> 172.21.15.55:0/2648439246 conn(0x7f83b8054cf0 msgr2=0x7f83b8055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f094974b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09440af880 0x7f0944152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f094974b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440b5420 0x7f0944160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f010a43f640 1 -- 172.21.15.55:0/2582725075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010415a6f0 msgr2=0x7f0104073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f010a43f640 1 --2- 172.21.15.55:0/2582725075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010415a6f0 0x7f0104073130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f01040d1990 tx=0x7f00fc02a640 comp rx=0 tx=0).stop 2024-05-20T22:08:14.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f094974b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09440ce660 0x7f0944160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f675b4ca640 1 Processor -- start 2024-05-20T22:08:14.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f675b4ca640 1 -- start start 2024-05-20T22:08:14.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f094974b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f094415c440 con 0x7f09440af880 2024-05-20T22:08:14.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f094974b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f094415c2c0 con 0x7f09440ce660 2024-05-20T22:08:14.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f094974b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f094415ea80 con 0x7f09440b5420 2024-05-20T22:08:14.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f0942ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09440af880 0x7f0944152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7f0942ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09440af880 0x7f0944152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58124/0 (socket says 172.21.15.55:58124) 2024-05-20T22:08:14.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f0942ffd640 1 -- 172.21.15.55:0/2687454048 learned_addr learned my addr 172.21.15.55:0/2687454048 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fb3ed40a640 1 -- 172.21.15.55:0/905535134 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e003b230 con 0x7fb3e004fb40 2024-05-20T22:08:14.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fb3e77fe640 1 --2- 172.21.15.55:0/905535134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 0x7fb3e004ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7fb3e6ffd640 1 -- 172.21.15.55:0/905535134 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2391962641 0 0) 0x7fb3e804fc90 con 0x7fb3e00e7ed0 2024-05-20T22:08:14.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7fb3e6ffd640 1 -- 172.21.15.55:0/905535134 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3d8003520 con 0x7fb3e00e7ed0 2024-05-20T22:08:14.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7fb3e77fe640 1 -- 172.21.15.55:0/905535134 >> v2:172.21.15.55:3300/0 conn(0x7fb3e0053680 msgr2=0x7fb3e0053a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fd0c8808640 1 -- 172.21.15.55:0/3340572568 shutdown_connections 2024-05-20T22:08:14.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fd0c8808640 1 --2- 172.21.15.55:0/3340572568 >> v2:172.21.15.55:3300/0 conn(0x7fd0c40afc90 0x7fd0c40b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fd0c8808640 1 --2- 172.21.15.55:0/3340572568 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c40af880 0x7fd0c40b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fd0c8808640 1 --2- 172.21.15.55:0/3340572568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c406f1a0 0x7fd0c40b5420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fd0c8808640 1 -- 172.21.15.55:0/3340572568 >> 172.21.15.55:0/3340572568 conn(0x7fd0c40732e0 msgr2=0x7fd0c406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fd0c8808640 1 -- 172.21.15.55:0/3340572568 shutdown_connections 2024-05-20T22:08:14.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.937+0000 7fd0c8808640 1 -- 172.21.15.55:0/3340572568 wait complete. 2024-05-20T22:08:14.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f09427fc640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440b5420 0x7f0944160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f0942ffd640 1 -- 172.21.15.55:0/2687454048 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440b5420 msgr2=0x7f0944160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f0942ffd640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440b5420 0x7f0944160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f0942ffd640 1 -- 172.21.15.55:0/2687454048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09440ce660 msgr2=0x7f0944160c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T22:08:14.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f0942ffd640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09440ce660 0x7f0944160c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f0942ffd640 1 -- 172.21.15.55:0/2687454048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09440cedd0 con 0x7f09440af880 2024-05-20T22:08:14.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f122e5ef640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280cb530 0x7f12281634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f122e5ef640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f12280d4100 0x7f122806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f122e5ef640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12280c8880 con 0x7f12280c8470 2024-05-20T22:08:14.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f122e5ef640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f12280cfdb0 con 0x7f12280d4100 2024-05-20T22:08:14.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f122e5ef640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f12280cf7f0 con 0x7f12280cb530 2024-05-20T22:08:14.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f0942ffd640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09440af880 0x7f0944152530 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f093c00ef00 tx=0x7f093c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7fb3e77fe640 1 --2- 172.21.15.55:0/905535134 >> v2:172.21.15.55:3300/0 conn(0x7fb3e0053680 0x7fb3e0053a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7fb3e77fe640 1 -- 172.21.15.55:0/905535134 >> v1:172.21.15.138:6789/0 conn(0x7fb3e00e7ed0 legacy=0x7fb3e00341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7fb3e77fe640 1 -- 172.21.15.55:0/905535134 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e0054fe0 con 0x7fb3e004fb40 2024-05-20T22:08:14.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f122cdec640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f12280c8470 0x7f12280d39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46432/0 (socket says 172.21.15.55:46432) 2024-05-20T22:08:14.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f122d5ed640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280cb530 0x7f12281634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.938+0000 7f122cdec640 1 -- 172.21.15.55:0/808568266 learned_addr learned my addr 172.21.15.55:0/808568266 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:14.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f122ddee640 1 --2- 172.21.15.55:0/808568266 >> v2:172.21.15.138:3300/0 conn(0x7f12280d4100 0x7f122806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f092b7fe640 1 -- 172.21.15.55:0/2687454048 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c018020 con 0x7f09440af880 2024-05-20T22:08:14.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f010a43f640 1 -- 172.21.15.55:0/2582725075 shutdown_connections 2024-05-20T22:08:14.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f010a43f640 1 --2- 172.21.15.55:0/2582725075 >> v2:172.21.15.138:3300/0 conn(0x7f010406c2b0 0x7f0104163e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f010a43f640 1 --2- 172.21.15.55:0/2582725075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010415a6f0 0x7f0104073130 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f094974b640 1 -- 172.21.15.55:0/2687454048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09440cf0b0 con 0x7f09440af880 2024-05-20T22:08:14.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f010a43f640 1 -- 172.21.15.55:0/2582725075 >> 172.21.15.55:0/2582725075 conn(0x7f01040b3980 msgr2=0x7f01040b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7fb3ecc09640 1 --2- 172.21.15.55:0/905535134 >> v2:172.21.15.55:3300/0 conn(0x7fb3e0053680 0x7fb3e0053a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:14.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f094974b640 1 -- 172.21.15.55:0/2687454048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09440cf670 con 0x7f09440af880 2024-05-20T22:08:14.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f122d5ed640 1 -- 172.21.15.55:0/808568266 >> v2:172.21.15.138:3300/0 conn(0x7f12280d4100 msgr2=0x7f122806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:14.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f122d5ed640 1 --2- 172.21.15.55:0/808568266 >> v2:172.21.15.138:3300/0 conn(0x7f12280d4100 0x7f122806b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f122d5ed640 1 -- 172.21.15.55:0/808568266 >> v1:172.21.15.55:6789/0 conn(0x7f12280c8470 legacy=0x7f12280d39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f122d5ed640 1 -- 172.21.15.55:0/808568266 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12280cfad0 con 0x7f12280cb530 2024-05-20T22:08:14.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f83c6303640 1 -- 172.21.15.55:0/2648439246 shutdown_connections 2024-05-20T22:08:14.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7f83c6303640 1 -- 172.21.15.55:0/2648439246 wait complete. 2024-05-20T22:08:14.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.939+0000 7fb3e77fe640 1 --2- 172.21.15.55:0/905535134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 0x7fb3e004ff50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0097a0 tx=0x7fb3dc02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9e52aa0e1127798 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7f122ddee640 1 --2- 172.21.15.55:0/808568266 >> v2:172.21.15.138:3300/0 conn(0x7f12280d4100 0x7f122806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:14.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7fb3e6ffd640 1 -- 172.21.15.55:0/905535134 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc037070 con 0x7fb3e004fb40 2024-05-20T22:08:14.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7fb3e6ffd640 1 -- 172.21.15.55:0/905535134 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc032440 con 0x7fb3e004fb40 2024-05-20T22:08:14.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7fb3e6ffd640 1 -- 172.21.15.55:0/905535134 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc037070 con 0x7fb3e004fb40 2024-05-20T22:08:14.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7f122d5ed640 1 --2- 172.21.15.55:0/808568266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280cb530 0x7f12281634f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f12180097a0 tx=0x7f121802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c78c33e0709b1d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7f121ffff640 1 -- 172.21.15.55:0/808568266 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218037070 con 0x7f12280cb530 2024-05-20T22:08:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7f121ffff640 1 -- 172.21.15.55:0/808568266 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121802ae40 con 0x7f12280cb530 2024-05-20T22:08:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7f675b4ca640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f674c007780 0x7f674c054bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7f675b4ca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f674c055110 0x7f674c0519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7f675b4ca640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674c008fb0 con 0x7f674c007370 2024-05-20T22:08:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7f675b4ca640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f674c007b90 con 0x7f674c007780 2024-05-20T22:08:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7f675b4ca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f674c0097b0 con 0x7f674c055110 2024-05-20T22:08:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7f121ffff640 1 -- 172.21.15.55:0/808568266 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218037070 con 0x7f12280cb530 2024-05-20T22:08:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7f092b7fe640 1 -- 172.21.15.55:0/2687454048 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093c00bd30 con 0x7f09440af880 2024-05-20T22:08:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7fbbfa63d640 1 Processor -- start 2024-05-20T22:08:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7fbbfa63d640 1 -- start start 2024-05-20T22:08:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7f092b7fe640 1 -- 172.21.15.55:0/2687454048 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c014620 con 0x7f09440af880 2024-05-20T22:08:14.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7f6759cc7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f674c007780 0x7f674c054bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7f6759cc7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f674c007780 0x7f674c054bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50970/0 (socket says 172.21.15.55:50970) 2024-05-20T22:08:14.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7f6759cc7640 1 -- 172.21.15.55:0/1096906830 learned_addr learned my addr 172.21.15.55:0/1096906830 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7fb3ed40a640 1 -- 172.21.15.55:0/905535134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 msgr2=0x7fb3e004ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7fb3ed40a640 1 --2- 172.21.15.55:0/905535134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 0x7fb3e004ff50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0097a0 tx=0x7fb3dc02a110 comp rx=0 tx=0).stop 2024-05-20T22:08:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7f010a43f640 1 -- 172.21.15.55:0/2582725075 shutdown_connections 2024-05-20T22:08:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7f67594c6640 1 -- 172.21.15.55:0/1096906830 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3398754573 0 0) 0x7f6744001110 con 0x7f674c007370 2024-05-20T22:08:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.941+0000 7f67594c6640 1 -- 172.21.15.55:0/1096906830 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6740003520 con 0x7f674c007370 2024-05-20T22:08:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7f6759cc7640 1 -- 172.21.15.55:0/1096906830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f674c055110 msgr2=0x7f674c0519a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T22:08:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7f6759cc7640 1 --2- 172.21.15.55:0/1096906830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f674c055110 0x7f674c0519a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7f6759cc7640 1 -- 172.21.15.55:0/1096906830 >> v1:172.21.15.55:6789/0 conn(0x7f674c007370 legacy=0x7f674c051290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7f6759cc7640 1 -- 172.21.15.55:0/1096906830 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c009230 con 0x7f674c007780 2024-05-20T22:08:14.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7fd0c8808640 1 Processor -- start 2024-05-20T22:08:14.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.940+0000 7fd0c8808640 1 -- start start 2024-05-20T22:08:14.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7f122e5ef640 1 -- 172.21.15.55:0/808568266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280cb530 msgr2=0x7f12281634f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7f122e5ef640 1 --2- 172.21.15.55:0/808568266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280cb530 0x7f12281634f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f12180097a0 tx=0x7f121802a0f0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7fb3ed40a640 1 -- 172.21.15.55:0/905535134 shutdown_connections 2024-05-20T22:08:14.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7fb3ed40a640 1 --2- 172.21.15.55:0/905535134 >> v2:172.21.15.55:3300/0 conn(0x7fb3e0053680 0x7fb3e0053a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7fb3ed40a640 1 --2- 172.21.15.55:0/905535134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 0x7fb3e004ff50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7fb3ed40a640 1 -- 172.21.15.55:0/905535134 >> 172.21.15.55:0/905535134 conn(0x7fb3e000aa70 msgr2=0x7fb3e000ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7f010a43f640 1 -- 172.21.15.55:0/2582725075 wait complete. 2024-05-20T22:08:14.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbfa63d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf406bc10 0x7fbbf41fc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7fd0c8808640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c406f1a0 0x7fd0c4143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7fd0c8808640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0c40af880 0x7fd0c40ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fb3ed40a640 1 -- 172.21.15.55:0/905535134 shutdown_connections 2024-05-20T22:08:14.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fd0c8808640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c415ac40 0x7fd0c4146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fd0c8808640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c415c440 con 0x7fd0c40af880 2024-05-20T22:08:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7f122e5ef640 1 -- 172.21.15.55:0/808568266 shutdown_connections 2024-05-20T22:08:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7f122e5ef640 1 --2- 172.21.15.55:0/808568266 >> v2:172.21.15.138:3300/0 conn(0x7f12280d4100 0x7f122806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7f122e5ef640 1 --2- 172.21.15.55:0/808568266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280cb530 0x7f12281634f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7f122e5ef640 1 -- 172.21.15.55:0/808568266 >> 172.21.15.55:0/808568266 conn(0x7f12280d4920 msgr2=0x7f12280d4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbfa63d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbf406fcb0 0x7fbbf41ea3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbfa63d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 0x7fbbf41eb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbfa63d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf40b4f80 con 0x7fbbf406bc10 2024-05-20T22:08:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbfa63d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf406c980 con 0x7fbbf406fcb0 2024-05-20T22:08:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbfa63d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf40b5100 con 0x7fbbf40b4b70 2024-05-20T22:08:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7f09297fa640 1 -- 172.21.15.55:0/2687454048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09440758b0 con 0x7f09440af880 2024-05-20T22:08:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7f122e5ef640 1 -- 172.21.15.55:0/808568266 shutdown_connections 2024-05-20T22:08:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbf3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf406bc10 0x7fbbf41fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbf3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf406bc10 0x7fbbf41fc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58146/0 (socket says 172.21.15.55:58146) 2024-05-20T22:08:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbf3fff640 1 -- 172.21.15.55:0/1918725466 learned_addr learned my addr 172.21.15.55:0/1918725466 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7f6759cc7640 1 --2- 172.21.15.55:0/1096906830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 0x7f674c054bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6750009870 tx=0x7f675002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f839eddde57b2ced server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbf37fe640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbf406fcb0 0x7fbbf41ea3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fbbf8bda640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 0x7fbbf41eb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7f83c6303640 1 Processor -- start 2024-05-20T22:08:14.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.942+0000 7f83c6303640 1 -- start start 2024-05-20T22:08:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7f122e5ef640 1 -- 172.21.15.55:0/808568266 wait complete. 2024-05-20T22:08:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7f67594c6640 1 -- 172.21.15.55:0/1096906830 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750037070 con 0x7f674c007780 2024-05-20T22:08:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7f67594c6640 1 -- 172.21.15.55:0/1096906830 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6750032440 con 0x7f674c007780 2024-05-20T22:08:14.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.944+0000 7fbbf3fff640 1 -- 172.21.15.55:0/1918725466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 msgr2=0x7fbbf41eb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.944+0000 7fbbf3fff640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 0x7fbbf41eb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.944+0000 7fbbf3fff640 1 -- 172.21.15.55:0/1918725466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbf406fcb0 msgr2=0x7fbbf41ea3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.944+0000 7fbbf3fff640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbf406fcb0 0x7fbbf41ea3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.944+0000 7fbbf3fff640 1 -- 172.21.15.55:0/1918725466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf41f92d0 con 0x7fbbf406bc10 2024-05-20T22:08:14.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.944+0000 7fbbf3fff640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf406bc10 0x7fbbf41fc120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009e50 tx=0x7fbbec004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f675b4ca640 1 -- 172.21.15.55:0/1096906830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 msgr2=0x7f674c054bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f675b4ca640 1 --2- 172.21.15.55:0/1096906830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 0x7f674c054bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6750009870 tx=0x7f675002a250 comp rx=0 tx=0).stop 2024-05-20T22:08:14.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.944+0000 7f092b7fe640 1 -- 172.21.15.55:0/2687454048 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f093c005d10 con 0x7f09440af880 2024-05-20T22:08:14.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.944+0000 7f092b7fe640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f093006bf00 0x7f093006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f092b7fe640 1 -- 172.21.15.55:0/2687454048 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f093c08bb80 con 0x7f09440af880 2024-05-20T22:08:14.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fd0c1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0c40af880 0x7fd0c40ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fd0c259c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c406f1a0 0x7fd0c4143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fd0c1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0c40af880 0x7fd0c40ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58130/0 (socket says 172.21.15.55:58130) 2024-05-20T22:08:14.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fd0c259c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c406f1a0 0x7fd0c4143d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47290/0 (socket says 172.21.15.55:47290) 2024-05-20T22:08:14.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fd0c1d9b640 1 -- 172.21.15.55:0/2631633222 learned_addr learned my addr 172.21.15.55:0/2631633222 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7fd0c2d9d640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c415ac40 0x7fd0c4146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c415c2c0 con 0x7fd0c415ac40 2024-05-20T22:08:14.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c4070050 con 0x7fd0c406f1a0 2024-05-20T22:08:14.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c6303640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b804f440 0x7f83b80e2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c6303640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83b8050390 0x7f83b80dd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c5301640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b804f440 0x7f83b80e2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c5301640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b804f440 0x7f83b80e2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:50998/0 (socket says 172.21.15.55:50998) 2024-05-20T22:08:14.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c6303640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83b80e7de0 0x7f83b80e9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c5301640 1 -- 172.21.15.55:0/2656068729 learned_addr learned my addr 172.21.15.55:0/2656068729 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c4b00640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83b8050390 0x7f83b80dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c6303640 1 -- 172.21.15.55:0/2656068729 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f83b80072a0 con 0x7f83b80e7de0 2024-05-20T22:08:14.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c6303640 1 -- 172.21.15.55:0/2656068729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8006fc0 con 0x7f83b804f440 2024-05-20T22:08:14.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c6303640 1 -- 172.21.15.55:0/2656068729 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f83b8007580 con 0x7f83b8050390 2024-05-20T22:08:14.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f010a43f640 1 Processor -- start 2024-05-20T22:08:14.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7f010a43f640 1 -- start start 2024-05-20T22:08:14.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f09427fc640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f093006bf00 0x7f093006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.943+0000 7fb3ed40a640 1 -- 172.21.15.55:0/905535134 wait complete. 2024-05-20T22:08:14.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7fb3ed40a640 1 Processor -- start 2024-05-20T22:08:14.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c5301640 1 -- 172.21.15.55:0/2656068729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83b8050390 msgr2=0x7f83b80dd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c5301640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83b8050390 0x7f83b80dd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c5301640 1 -- 172.21.15.55:0/2656068729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83b80e7de0 msgr2=0x7f83b80e9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c5301640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83b80e7de0 0x7f83b80e9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.945+0000 7f83c5301640 1 -- 172.21.15.55:0/2656068729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b80dfca0 con 0x7f83b804f440 2024-05-20T22:08:14.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7f83c5301640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b804f440 0x7f83b80e2380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83c002d830 tx=0x7f83c0068be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7f122e5ef640 1 Processor -- start 2024-05-20T22:08:14.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7f122e5ef640 1 -- start start 2024-05-20T22:08:14.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7f83b5ffb640 1 -- 172.21.15.55:0/2656068729 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c008bcc0 con 0x7f83b804f440 2024-05-20T22:08:14.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7f83b5ffb640 1 -- 172.21.15.55:0/2656068729 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c008f030 con 0x7f83b804f440 2024-05-20T22:08:14.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7f83c6303640 1 -- 172.21.15.55:0/2656068729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b80dff80 con 0x7f83b804f440 2024-05-20T22:08:14.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7f09427fc640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f093006bf00 0x7f093006e410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0934034000 tx=0x7f093402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7fd0c2d9d640 1 -- 172.21.15.55:0/2631633222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c406f1a0 msgr2=0x7fd0c4143d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7fd0c2d9d640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c406f1a0 0x7fd0c4143d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7fd0c2d9d640 1 -- 172.21.15.55:0/2631633222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0c40af880 msgr2=0x7fd0c40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7fd0c259c640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c406f1a0 0x7fd0c4143d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-20T22:08:14.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7fd0c2d9d640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0c40af880 0x7fd0c40ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7fd0c2d9d640 1 -- 172.21.15.55:0/2631633222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c4144250 con 0x7fd0c415ac40 2024-05-20T22:08:14.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7fd0c1d9b640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0c40af880 0x7fd0c40ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:14.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7f83b5ffb640 1 -- 172.21.15.55:0/2656068729 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c006c8c0 con 0x7f83b804f440 2024-05-20T22:08:14.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7f83c6303640 1 -- 172.21.15.55:0/2656068729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b80e0540 con 0x7f83b804f440 2024-05-20T22:08:14.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.947+0000 7f675b4ca640 1 -- 172.21.15.55:0/1096906830 shutdown_connections 2024-05-20T22:08:14.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.947+0000 7f675b4ca640 1 --2- 172.21.15.55:0/1096906830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f674c055110 0x7f674c0519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.947+0000 7f675b4ca640 1 --2- 172.21.15.55:0/1096906830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 0x7f674c054bd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.947+0000 7f675b4ca640 1 -- 172.21.15.55:0/1096906830 >> 172.21.15.55:0/1096906830 conn(0x7f674c055e10 msgr2=0x7f674c056220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:14.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.946+0000 7fb3ed40a640 1 -- start start 2024-05-20T22:08:14.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.947+0000 7fd0c2d9d640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c415ac40 0x7fd0c4146ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40722c0 tx=0x7fd0b400eaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f675b4ca640 1 -- 172.21.15.55:0/1096906830 shutdown_connections 2024-05-20T22:08:14.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7fd0b2ffd640 1 -- 172.21.15.55:0/2631633222 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4017070 con 0x7fd0c415ac40 2024-05-20T22:08:14.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7fd0b2ffd640 1 -- 172.21.15.55:0/2631633222 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b4013680 con 0x7fd0c415ac40 2024-05-20T22:08:14.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7fd0b2ffd640 1 -- 172.21.15.55:0/2631633222 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4013a30 con 0x7fd0c415ac40 2024-05-20T22:08:14.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f010a43f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f010415a6f0 0x7f0104157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f010a43f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0104163a50 0x7f010409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f122e5ef640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280c8470 0x7f122814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f122e5ef640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12280cb530 0x7f12280ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f122e5ef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280d4100 0x7f12280cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c415a640 con 0x7fd0c415ac40 2024-05-20T22:08:14.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c40cdb30 con 0x7fd0c415ac40 2024-05-20T22:08:14.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7fb3ed40a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 0x7fb3e0186900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f122e5ef640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f12280d4510 con 0x7f12280cb530 2024-05-20T22:08:14.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f122e5ef640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12280d31f0 con 0x7f12280d4100 2024-05-20T22:08:14.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7fb3ed40a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0053680 0x7fb3e004b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f122e5ef640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f12280d4690 con 0x7f12280c8470 2024-05-20T22:08:14.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f0103fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f010415a6f0 0x7f0104157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f83b5ffb640 1 -- 172.21.15.55:0/2656068729 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83c006ca60 con 0x7f83b804f440 2024-05-20T22:08:14.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f122cdec640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12280cb530 0x7f12280ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7fb3ed40a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e00e7ed0 0x7fb3e004e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f122cdec640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12280cb530 0x7f12280ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58152/0 (socket says 172.21.15.55:58152) 2024-05-20T22:08:14.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f122cdec640 1 -- 172.21.15.55:0/1318506435 learned_addr learned my addr 172.21.15.55:0/1318506435 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7fb3ed40a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e0050830 con 0x7fb3e0053680 2024-05-20T22:08:14.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f0103fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f010415a6f0 0x7f0104157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47328/0 (socket says 172.21.15.55:47328) 2024-05-20T22:08:14.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f0103fff640 1 -- 172.21.15.55:0/3745525524 learned_addr learned my addr 172.21.15.55:0/3745525524 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f010a43f640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01040af5a0 0x7f01040af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.948+0000 7f675b4ca640 1 -- 172.21.15.55:0/1096906830 wait complete. 2024-05-20T22:08:14.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7fb3ed40a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e0053160 con 0x7fb3e00e7ed0 2024-05-20T22:08:14.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7fb3ed40a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e00348e0 con 0x7fb3e004fb40 2024-05-20T22:08:14.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7fb3e77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0053680 0x7fb3e004b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7fb3e77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0053680 0x7fb3e004b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58162/0 (socket says 172.21.15.55:58162) 2024-05-20T22:08:14.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7fb3e77fe640 1 -- 172.21.15.55:0/2520677543 learned_addr learned my addr 172.21.15.55:0/2520677543 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f010406c900 con 0x7f0104163a50 2024-05-20T22:08:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0104073670 con 0x7f01040af5a0 2024-05-20T22:08:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f01040ae090 con 0x7f010415a6f0 2024-05-20T22:08:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f83c6303640 1 -- 172.21.15.55:0/2656068729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b8005570 con 0x7f83b804f440 2024-05-20T22:08:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f01089dc640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01040af5a0 0x7f01040af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f122d5ed640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280c8470 0x7f122814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f83b5ffb640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f83a406be30 0x7f83a406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7fb3e7fff640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 0x7fb3e0186900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7fb3e77fe640 1 -- 172.21.15.55:0/2520677543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 msgr2=0x7fb3e0186900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7fb3e77fe640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 0x7fb3e0186900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f0103fff640 1 -- 172.21.15.55:0/3745525524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01040af5a0 msgr2=0x7f01040af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f0103fff640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01040af5a0 0x7f01040af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7fb3e77fe640 1 -- 172.21.15.55:0/2520677543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e00e7ed0 msgr2=0x7fb3e004e790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T22:08:14.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.949+0000 7f122ddee640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280d4100 0x7f12280cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f0103fff640 1 -- 172.21.15.55:0/3745525524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0104163a50 msgr2=0x7f010409a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f0103fff640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0104163a50 0x7f010409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f0103fff640 1 -- 172.21.15.55:0/3745525524 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01040afef0 con 0x7f010415a6f0 2024-05-20T22:08:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7fb3e77fe640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e00e7ed0 0x7fb3e004e790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f83c4b00640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f83a406be30 0x7f83a406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f83b5ffb640 1 -- 172.21.15.55:0/2656068729 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f83c01061a0 con 0x7f83b804f440 2024-05-20T22:08:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7fb3e77fe640 1 -- 172.21.15.55:0/2520677543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e004b1e0 con 0x7fb3e0053680 2024-05-20T22:08:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f0103fff640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f010415a6f0 0x7f0104157580 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009f10 tx=0x7f00fc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f122cdec640 1 -- 172.21.15.55:0/1318506435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280c8470 msgr2=0x7f122814e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f122cdec640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280c8470 0x7f122814e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f122cdec640 1 -- 172.21.15.55:0/1318506435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280d4100 msgr2=0x7f12280cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:14.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f122cdec640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280d4100 0x7f12280cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:14.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f122cdec640 1 -- 172.21.15.55:0/1318506435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122814eb00 con 0x7f12280cb530 2024-05-20T22:08:14.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f0100ff9640 1 -- 172.21.15.55:0/3745525524 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc041020 con 0x7f010415a6f0 2024-05-20T22:08:14.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7fb3e77fe640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0053680 0x7fb3e004b970 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0098c0 tx=0x7fb3dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f0100ff9640 1 -- 172.21.15.55:0/3745525524 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00fc008790 con 0x7f010415a6f0 2024-05-20T22:08:14.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.951+0000 7f122cdec640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12280cb530 0x7f12280ca820 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f122400c3c0 tx=0x7f122400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.951+0000 7f675b4ca640 1 Processor -- start 2024-05-20T22:08:14.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.951+0000 7f675b4ca640 1 -- start start 2024-05-20T22:08:14.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.951+0000 7f83c4b00640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f83a406be30 0x7f83a406e340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f83bc034000 tx=0x7f83bc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:14.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.951+0000 7fd0b2ffd640 1 -- 172.21.15.55:0/2631633222 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0b401a070 con 0x7fd0c415ac40 2024-05-20T22:08:14.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.950+0000 7f0100ff9640 1 -- 172.21.15.55:0/3745525524 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc036040 con 0x7f010415a6f0 2024-05-20T22:08:14.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.951+0000 7fd0b2ffd640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0a406beb0 0x7fd0a406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:14.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.951+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010415dd70 con 0x7f010415a6f0 2024-05-20T22:08:14.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.951+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f010409b830 con 0x7f010415a6f0 2024-05-20T22:08:14.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.952+0000 7fd0c259c640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0a406beb0 0x7fd0a406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:14.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.952+0000 7fd0b2ffd640 1 -- 172.21.15.55:0/2631633222 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd0b408ae10 con 0x7fd0c415ac40 2024-05-20T22:08:14.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.952+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0104069090 con 0x7f010415a6f0 2024-05-20T22:08:15.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.952+0000 7fd0c259c640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0a406beb0 0x7fd0a406e3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8034000 tx=0x7fd0b802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:15.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f0100ff9640 1 -- 172.21.15.55:0/3745525524 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00fc044020 con 0x7f010415a6f0 2024-05-20T22:08:15.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c415a3d0 con 0x7fd0c415ac40 2024-05-20T22:08:15.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f675b4ca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f674c007370 0x7f674c17df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:15.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f675b4ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 0x7f674c0361a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:15.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f675b4ca640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f674c055110 0x7f674c0366e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:15.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f675b4ca640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0494a0 con 0x7f674c055110 2024-05-20T22:08:15.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f675b4ca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f674c050d70 con 0x7f674c007780 2024-05-20T22:08:15.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f675b4ca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f674c051ee0 con 0x7f674c007370 2024-05-20T22:08:15.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f675a4c8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f674c007370 0x7f674c17df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:15.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f6759cc7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 0x7f674c0361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:15.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f0100ff9640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f00e406be30 0x7f00e406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:15.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f6759cc7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 0x7f674c0361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:51052/0 (socket says 172.21.15.55:51052) 2024-05-20T22:08:15.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f6759cc7640 1 -- 172.21.15.55:0/2553082954 learned_addr learned my addr 172.21.15.55:0/2553082954 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:15.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f675acc9640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f674c055110 0x7f674c0366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:15.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f0100ff9640 1 -- 172.21.15.55:0/3745525524 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f00fc0abf80 con 0x7f010415a6f0 2024-05-20T22:08:15.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f01037fe640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f00e406be30 0x7f00e406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:15.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7fb3e4ff9640 1 -- 172.21.15.55:0/2520677543 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc042070 con 0x7fb3e0053680 2024-05-20T22:08:15.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.953+0000 7f092b7fe640 1 -- 172.21.15.55:0/2687454048 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f093c022030 con 0x7f09440af880 2024-05-20T22:08:15.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7fb3ed40a640 1 -- 172.21.15.55:0/2520677543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e003a140 con 0x7fb3e0053680 2024-05-20T22:08:15.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7fb3ed40a640 1 -- 172.21.15.55:0/2520677543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e003a700 con 0x7fb3e0053680 2024-05-20T22:08:15.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7fbbf0ff9640 1 -- 172.21.15.55:0/1918725466 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec036050 con 0x7fbbf406bc10 2024-05-20T22:08:15.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7fbbf0ff9640 1 -- 172.21.15.55:0/1918725466 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbec042070 con 0x7fbbf406bc10 2024-05-20T22:08:15.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7fbbf0ff9640 1 -- 172.21.15.55:0/1918725466 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec009510 con 0x7fbbf406bc10 2024-05-20T22:08:15.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7fb3e4ff9640 1 -- 172.21.15.55:0/2520677543 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc03b610 con 0x7fb3e0053680 2024-05-20T22:08:15.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7fb3e4ff9640 1 -- 172.21.15.55:0/2520677543 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc02ae60 con 0x7fb3e0053680 2024-05-20T22:08:15.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7fbbfa63d640 1 -- 172.21.15.55:0/1918725466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf41fc660 con 0x7fbbf406bc10 2024-05-20T22:08:15.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7fbbfa63d640 1 -- 172.21.15.55:0/1918725466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf41eca70 con 0x7fbbf406bc10 2024-05-20T22:08:15.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f121dffb640 1 -- 172.21.15.55:0/1318506435 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224017070 con 0x7f12280cb530 2024-05-20T22:08:15.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f121dffb640 1 -- 172.21.15.55:0/1318506435 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1224005ce0 con 0x7f12280cb530 2024-05-20T22:08:15.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f01037fe640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f00e406be30 0x7f00e406e340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f00f4006fd0 tx=0x7f00f4006bc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:15.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f122e5ef640 1 -- 172.21.15.55:0/1318506435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122814bcc0 con 0x7f12280cb530 2024-05-20T22:08:15.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f122e5ef640 1 -- 172.21.15.55:0/1318506435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122814c280 con 0x7f12280cb530 2024-05-20T22:08:15.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f6759cc7640 1 -- 172.21.15.55:0/2553082954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f674c007370 msgr2=0x7f674c17df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f6759cc7640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f674c007370 0x7f674c17df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f6759cc7640 1 -- 172.21.15.55:0/2553082954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f674c055110 msgr2=0x7f674c0366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f6759cc7640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f674c055110 0x7f674c0366e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f121dffb640 1 -- 172.21.15.55:0/1318506435 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240072d0 con 0x7f12280cb530 2024-05-20T22:08:15.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.954+0000 7f6759cc7640 1 -- 172.21.15.55:0/2553082954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c036f60 con 0x7f674c007780 2024-05-20T22:08:15.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.955+0000 7f6759cc7640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 0x7f674c0361a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f675002a840 tx=0x7f67500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:15.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.955+0000 7fbbfa63d640 1 -- 172.21.15.55:0/1918725466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf40758b0 con 0x7fbbf406bc10 2024-05-20T22:08:15.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.955+0000 7fb3ed40a640 1 -- 172.21.15.55:0/2520677543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c8005290 con 0x7fb3e0053680 2024-05-20T22:08:15.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.956+0000 7f83b5ffb640 1 -- 172.21.15.55:0/2656068729 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f83b80507a0 con 0x7f83b804f440 2024-05-20T22:08:15.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.956+0000 7f122e5ef640 1 -- 172.21.15.55:0/1318506435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12280758b0 con 0x7f12280cb530 2024-05-20T22:08:15.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.958+0000 7f121dffb640 1 -- 172.21.15.55:0/1318506435 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f122401a070 con 0x7f12280cb530 2024-05-20T22:08:15.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.957+0000 7fb3e4ff9640 1 -- 172.21.15.55:0/2520677543 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3dc045070 con 0x7fb3e0053680 2024-05-20T22:08:15.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.958+0000 7fb3e4ff9640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3cc06be60 0x7fb3cc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:15.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.958+0000 7fb3e7fff640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3cc06be60 0x7fb3cc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:15.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.958+0000 7fb3e4ff9640 1 -- 172.21.15.55:0/2520677543 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb3dc0ab8a0 con 0x7fb3e0053680 2024-05-20T22:08:15.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.958+0000 7f121dffb640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f120c06be80 0x7f120c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:15.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.959+0000 7f121dffb640 1 -- 172.21.15.55:0/1318506435 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1224056980 con 0x7f12280cb530 2024-05-20T22:08:15.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.959+0000 7f0100ff9640 1 -- 172.21.15.55:0/3745525524 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f00fc04b030 con 0x7f010415a6f0 2024-05-20T22:08:15.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.959+0000 7fb3e7fff640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3cc06be60 0x7fb3cc06e370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8074b60 tx=0x7fb3e806f040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:15.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.959+0000 7f122d5ed640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f120c06be80 0x7f120c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:15.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.960+0000 7f122d5ed640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f120c06be80 0x7f120c06e390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1218034000 tx=0x7f121802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:15.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.960+0000 7f674affd640 1 -- 172.21.15.55:0/2553082954 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750032e60 con 0x7f674c007780 2024-05-20T22:08:15.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.960+0000 7f674affd640 1 -- 172.21.15.55:0/2553082954 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6750042070 con 0x7f674c007780 2024-05-20T22:08:15.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.960+0000 7f674affd640 1 -- 172.21.15.55:0/2553082954 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750037040 con 0x7f674c007780 2024-05-20T22:08:15.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.960+0000 7f675b4ca640 1 -- 172.21.15.55:0/2553082954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674c0e3cb0 con 0x7f674c007780 2024-05-20T22:08:15.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.960+0000 7fd0b2ffd640 1 -- 172.21.15.55:0/2631633222 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd0b4058e10 con 0x7fd0c415ac40 2024-05-20T22:08:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.960+0000 7f675b4ca640 1 -- 172.21.15.55:0/2553082954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f674c0e4270 con 0x7f674c007780 2024-05-20T22:08:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.963+0000 7f674affd640 1 -- 172.21.15.55:0/2553082954 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6750032440 con 0x7f674c007780 2024-05-20T22:08:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.964+0000 7f674affd640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f673806be80 0x7f673806e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.964+0000 7f674affd640 1 -- 172.21.15.55:0/2553082954 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f67500ab960 con 0x7f674c007780 2024-05-20T22:08:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.964+0000 7f675a4c8640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f673806be80 0x7f673806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.964+0000 7fb3e4ff9640 1 -- 172.21.15.55:0/2520677543 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb3dc079f80 con 0x7fb3e0053680 2024-05-20T22:08:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.964+0000 7f675b4ca640 1 -- 172.21.15.55:0/2553082954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674c036cf0 con 0x7f674c007780 2024-05-20T22:08:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.965+0000 7f675a4c8640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f673806be80 0x7f673806e390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f67440012b0 tx=0x7f6744009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.966+0000 7fbbf0ff9640 1 -- 172.21.15.55:0/1918725466 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbec0039c0 con 0x7fbbf406bc10 2024-05-20T22:08:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.967+0000 7fbbf0ff9640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbbe006be80 0x7fbbe006e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.967+0000 7fbbf37fe640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbbe006be80 0x7fbbe006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:15.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.968+0000 7fbbf37fe640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbbe006be80 0x7fbbe006e390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbbf415f0f0 tx=0x7fbbe4007e00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:15.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.970+0000 7f674affd640 1 -- 172.21.15.55:0/2553082954 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6750079960 con 0x7f674c007780 2024-05-20T22:08:15.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.972+0000 7fbbf0ff9640 1 -- 172.21.15.55:0/1918725466 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbbec078b60 con 0x7fbbf406bc10 2024-05-20T22:08:15.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.972+0000 7f121dffb640 1 -- 172.21.15.55:0/1318506435 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f122814eb00 con 0x7f12280cb530 2024-05-20T22:08:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:14.982+0000 7fbbf0ff9640 1 -- 172.21.15.55:0/1918725466 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbbec04c020 con 0x7fbbf406bc10 2024-05-20T22:08:15.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.384+0000 7f09297fa640 1 -- 172.21.15.55:0/2687454048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f09440cfc90 con 0x7f09440af880 2024-05-20T22:08:15.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.385+0000 7f83c6303640 1 -- 172.21.15.55:0/2656068729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f83b80e6a20 con 0x7f83b804f440 2024-05-20T22:08:15.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.385+0000 7f092b7fe640 1 -- 172.21.15.55:0/2687454048 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f093c05a140 con 0x7f09440af880 2024-05-20T22:08:15.386 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-20T22:08:15.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.385+0000 7f83b5ffb640 1 -- 172.21.15.55:0/2656068729 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f83c00d46e0 con 0x7f83b804f440 2024-05-20T22:08:15.387 INFO:teuthology.orchestra.run.smithi055.stdout:55834574857 2024-05-20T22:08:15.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.388+0000 7f09297fa640 1 -- 172.21.15.55:0/2687454048 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f093006bf00 msgr2=0x7f093006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.388+0000 7f09297fa640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f093006bf00 0x7f093006e410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0934034000 tx=0x7f093402e000 comp rx=0 tx=0).stop 2024-05-20T22:08:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.388+0000 7f09297fa640 1 -- 172.21.15.55:0/2687454048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09440af880 msgr2=0x7f0944152530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.389+0000 7f09297fa640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09440af880 0x7f0944152530 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f093c00ef00 tx=0x7f093c00b480 comp rx=0 tx=0).stop 2024-05-20T22:08:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.389+0000 7f09297fa640 1 -- 172.21.15.55:0/2687454048 shutdown_connections 2024-05-20T22:08:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.389+0000 7f09297fa640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f093006bf00 0x7f093006e410 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.389+0000 7f09297fa640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09440ce660 0x7f0944160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.389+0000 7f09297fa640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09440b5420 0x7f0944160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.389+0000 7f09297fa640 1 --2- 172.21.15.55:0/2687454048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09440af880 0x7f0944152530 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.389+0000 7f09297fa640 1 -- 172.21.15.55:0/2687454048 >> 172.21.15.55:0/2687454048 conn(0x7f09440732e0 msgr2=0x7f094415cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:15.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.389+0000 7f83ab7fe640 1 -- 172.21.15.55:0/2656068729 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f83a406be30 msgr2=0x7f83a406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.389+0000 7f83ab7fe640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f83a406be30 0x7f83a406e340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f83bc034000 tx=0x7f83bc02e000 comp rx=0 tx=0).stop 2024-05-20T22:08:15.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f09297fa640 1 -- 172.21.15.55:0/2687454048 shutdown_connections 2024-05-20T22:08:15.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f83ab7fe640 1 -- 172.21.15.55:0/2656068729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b804f440 msgr2=0x7f83b80e2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f83ab7fe640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b804f440 0x7f83b80e2380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83c002d830 tx=0x7f83c0068be0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f09297fa640 1 -- 172.21.15.55:0/2687454048 wait complete. 2024-05-20T22:08:15.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f83ab7fe640 1 -- 172.21.15.55:0/2656068729 shutdown_connections 2024-05-20T22:08:15.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f83ab7fe640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f83a406be30 0x7f83a406e340 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f83ab7fe640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83b80e7de0 0x7f83b80e9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f83ab7fe640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83b8050390 0x7f83b80dd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f83ab7fe640 1 --2- 172.21.15.55:0/2656068729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b804f440 0x7f83b80e2380 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0104166860 con 0x7f010415a6f0 2024-05-20T22:08:15.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f83ab7fe640 1 -- 172.21.15.55:0/2656068729 >> 172.21.15.55:0/2656068729 conn(0x7f83b8054cf0 msgr2=0x7f83b8005990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:15.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f83ab7fe640 1 -- 172.21.15.55:0/2656068729 shutdown_connections 2024-05-20T22:08:15.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.390+0000 7f83ab7fe640 1 -- 172.21.15.55:0/2656068729 wait complete. 2024-05-20T22:08:15.394 INFO:teuthology.orchestra.run.smithi055.stdout:55834574857 2024-05-20T22:08:15.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.391+0000 7f0100ff9640 1 -- 172.21.15.55:0/3745525524 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f01040afef0 con 0x7f010415a6f0 2024-05-20T22:08:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.394+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f00e406be30 msgr2=0x7f00e406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.394+0000 7f010a43f640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f00e406be30 0x7f00e406e340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f00f4006fd0 tx=0x7f00f4006bc0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.394+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f010415a6f0 msgr2=0x7f0104157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.394+0000 7f010a43f640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f010415a6f0 0x7f0104157580 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009f10 tx=0x7f00fc004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.395+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 shutdown_connections 2024-05-20T22:08:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.395+0000 7f010a43f640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f00e406be30 0x7f00e406e340 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.395+0000 7f010a43f640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01040af5a0 0x7f01040af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.395+0000 7f010a43f640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0104163a50 0x7f010409a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.395+0000 7f010a43f640 1 --2- 172.21.15.55:0/3745525524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f010415a6f0 0x7f0104157580 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.395+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 >> 172.21.15.55:0/3745525524 conn(0x7f01040b3980 msgr2=0x7f01040b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:15.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.395+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 shutdown_connections 2024-05-20T22:08:15.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.395+0000 7f010a43f640 1 -- 172.21.15.55:0/3745525524 wait complete. 2024-05-20T22:08:15.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.397+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd0c40758b0 con 0x7fd0c415ac40 2024-05-20T22:08:15.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.398+0000 7fd0b2ffd640 1 -- 172.21.15.55:0/2631633222 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd0b405d110 con 0x7fd0c415ac40 2024-05-20T22:08:15.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.398+0000 7fb3ed40a640 1 -- 172.21.15.55:0/2520677543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb3c8005850 con 0x7fb3e0053680 2024-05-20T22:08:15.399 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-20T22:08:15.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.399+0000 7fb3e4ff9640 1 -- 172.21.15.55:0/2520677543 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb3dc079880 con 0x7fb3e0053680 2024-05-20T22:08:15.400 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-20T22:08:15.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.400+0000 7f675b4ca640 1 -- 172.21.15.55:0/2553082954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f674c0059c0 con 0x7f674c007780 2024-05-20T22:08:15.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.401+0000 7f674affd640 1 -- 172.21.15.55:0/2553082954 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f675007dc60 con 0x7f674c007780 2024-05-20T22:08:15.402 INFO:teuthology.orchestra.run.smithi055.stdout:55834574856 2024-05-20T22:08:15.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.402+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0a406beb0 msgr2=0x7fd0a406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.402+0000 7fd0c8808640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0a406beb0 0x7fd0a406e3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8034000 tx=0x7fd0b802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:15.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.402+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c415ac40 msgr2=0x7fd0c4146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.402+0000 7fd0c8808640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c415ac40 0x7fd0c4146ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40722c0 tx=0x7fd0b400eaa0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.402+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 shutdown_connections 2024-05-20T22:08:15.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.402+0000 7fd0c8808640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0a406beb0 0x7fd0a406e3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.402+0000 7fd0c8808640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c415ac40 0x7fd0c4146ae0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.402+0000 7fd0c8808640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0c40af880 0x7fd0c40ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.402+0000 7fd0c8808640 1 --2- 172.21.15.55:0/2631633222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0c406f1a0 0x7fd0c4143d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.402+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 >> 172.21.15.55:0/2631633222 conn(0x7fd0c40732e0 msgr2=0x7fd0c415cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:15.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.403+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 shutdown_connections 2024-05-20T22:08:15.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.403+0000 7fb3ed40a640 1 -- 172.21.15.55:0/2520677543 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3cc06be60 msgr2=0x7fb3cc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.403+0000 7fd0c8808640 1 -- 172.21.15.55:0/2631633222 wait complete. 2024-05-20T22:08:15.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.403+0000 7fb3ed40a640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3cc06be60 0x7fb3cc06e370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8074b60 tx=0x7fb3e806f040 comp rx=0 tx=0).stop 2024-05-20T22:08:15.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.403+0000 7fb3ed40a640 1 -- 172.21.15.55:0/2520677543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0053680 msgr2=0x7fb3e004b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.403+0000 7fb3ed40a640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0053680 0x7fb3e004b970 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0098c0 tx=0x7fb3dc0046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.404+0000 7fb3ed40a640 1 -- 172.21.15.55:0/2520677543 shutdown_connections 2024-05-20T22:08:15.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.404+0000 7fb3ed40a640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3cc06be60 0x7fb3cc06e370 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.404+0000 7fb3ed40a640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3e00e7ed0 0x7fb3e004e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.404+0000 7fb3ed40a640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0053680 0x7fb3e004b970 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.404+0000 7fb3ed40a640 1 --2- 172.21.15.55:0/2520677543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3e004fb40 0x7fb3e0186900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.404+0000 7fb3ed40a640 1 -- 172.21.15.55:0/2520677543 >> 172.21.15.55:0/2520677543 conn(0x7fb3e000aa70 msgr2=0x7fb3e0007980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:15.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.404+0000 7fb3ed40a640 1 -- 172.21.15.55:0/2520677543 shutdown_connections 2024-05-20T22:08:15.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.404+0000 7fb3ed40a640 1 -- 172.21.15.55:0/2520677543 wait complete. 2024-05-20T22:08:15.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 -- 172.21.15.55:0/2553082954 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f673806be80 msgr2=0x7f673806e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f673806be80 0x7f673806e390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f67440012b0 tx=0x7f6744009040 comp rx=0 tx=0).stop 2024-05-20T22:08:15.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 -- 172.21.15.55:0/2553082954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 msgr2=0x7f674c0361a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 0x7f674c0361a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f675002a840 tx=0x7f67500046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 -- 172.21.15.55:0/2553082954 shutdown_connections 2024-05-20T22:08:15.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f122e5ef640 1 -- 172.21.15.55:0/1318506435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f12280ccba0 con 0x7f12280cb530 2024-05-20T22:08:15.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f673806be80 0x7f673806e390 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f674c055110 0x7f674c0366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f674c007780 0x7f674c0361a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 --2- 172.21.15.55:0/2553082954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f674c007370 0x7f674c17df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 -- 172.21.15.55:0/2553082954 >> 172.21.15.55:0/2553082954 conn(0x7f674c055e10 msgr2=0x7f674c04bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:15.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.405+0000 7f675b4ca640 1 -- 172.21.15.55:0/2553082954 shutdown_connections 2024-05-20T22:08:15.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.406+0000 7f675b4ca640 1 -- 172.21.15.55:0/2553082954 wait complete. 2024-05-20T22:08:15.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.407+0000 7f121dffb640 1 -- 172.21.15.55:0/1318506435 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f122814eb00 con 0x7f12280cb530 2024-05-20T22:08:15.411 INFO:teuthology.orchestra.run.smithi055.stdout:55834574857 2024-05-20T22:08:15.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.410+0000 7f122e5ef640 1 -- 172.21.15.55:0/1318506435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f120c06be80 msgr2=0x7f120c06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f120c06be80 0x7f120c06e390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1218034000 tx=0x7f121802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:15.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 -- 172.21.15.55:0/1318506435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12280cb530 msgr2=0x7f12280ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12280cb530 0x7f12280ca820 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f122400c3c0 tx=0x7f122400c880 comp rx=0 tx=0).stop 2024-05-20T22:08:15.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 -- 172.21.15.55:0/1318506435 shutdown_connections 2024-05-20T22:08:15.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f120c06be80 0x7f120c06e390 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12280d4100 0x7f12280cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12280cb530 0x7f12280ca820 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 --2- 172.21.15.55:0/1318506435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12280c8470 0x7f122814e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 -- 172.21.15.55:0/1318506435 >> 172.21.15.55:0/1318506435 conn(0x7f12280d4920 msgr2=0x7f1228162690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:15.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 -- 172.21.15.55:0/1318506435 shutdown_connections 2024-05-20T22:08:15.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.411+0000 7f122e5ef640 1 -- 172.21.15.55:0/1318506435 wait complete. 2024-05-20T22:08:15.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-05-20T22:08:15.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.413+0000 7fbbfa63d640 1 -- 172.21.15.55:0/1918725466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbbf406edf0 con 0x7fbbf406bc10 2024-05-20T22:08:15.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-05-20T22:08:15.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.416+0000 7fbbf0ff9640 1 -- 172.21.15.55:0/1918725466 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbbec07aac0 con 0x7fbbf406bc10 2024-05-20T22:08:15.418 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-20T22:08:15.421 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-05-20T22:08:15.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.419+0000 7fbbfa63d640 1 -- 172.21.15.55:0/1918725466 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbbe006be80 msgr2=0x7fbbe006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.419+0000 7fbbfa63d640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbbe006be80 0x7fbbe006e390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbbf415f0f0 tx=0x7fbbe4007e00 comp rx=0 tx=0).stop 2024-05-20T22:08:15.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.419+0000 7fbbfa63d640 1 -- 172.21.15.55:0/1918725466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf406bc10 msgr2=0x7fbbf41fc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:15.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.419+0000 7fbbfa63d640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf406bc10 0x7fbbf41fc120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009e50 tx=0x7fbbec004a70 comp rx=0 tx=0).stop 2024-05-20T22:08:15.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.419+0000 7fbbfa63d640 1 -- 172.21.15.55:0/1918725466 shutdown_connections 2024-05-20T22:08:15.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.419+0000 7fbbfa63d640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbbe006be80 0x7fbbe006e390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.419+0000 7fbbfa63d640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbbf40b4b70 0x7fbbf41eb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.419+0000 7fbbfa63d640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbf406fcb0 0x7fbbf41ea3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.419+0000 7fbbfa63d640 1 --2- 172.21.15.55:0/1918725466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf406bc10 0x7fbbf41fc120 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:15.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.420+0000 7fbbfa63d640 1 -- 172.21.15.55:0/1918725466 >> 172.21.15.55:0/1918725466 conn(0x7fbbf40d06f0 msgr2=0x7fbbf40b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:15.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.420+0000 7fbbfa63d640 1 -- 172.21.15.55:0/1918725466 shutdown_connections 2024-05-20T22:08:15.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:15.420+0000 7fbbfa63d640 1 -- 172.21.15.55:0/1918725466 wait complete. 2024-05-20T22:08:15.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2024-05-20T22:08:15.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2024-05-20T22:08:15.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2024-05-20T22:08:15.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-05-20T22:08:15.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2024-05-20T22:08:16.417 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-20T22:08:16.418 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-20T22:08:16.421 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-20T22:08:16.428 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-20T22:08:16.428 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-20T22:08:16.429 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-20T22:08:16.433 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-20T22:08:16.436 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-20T22:08:16.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.616+0000 7fc54b7f5640 1 Processor -- start 2024-05-20T22:08:16.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.616+0000 7f309302b640 1 Processor -- start 2024-05-20T22:08:16.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.616+0000 7fc54b7f5640 1 -- start start 2024-05-20T22:08:16.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.616+0000 7f309302b640 1 -- start start 2024-05-20T22:08:16.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.617+0000 7f25791db640 1 Processor -- start 2024-05-20T22:08:16.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.619+0000 7fc54b7f5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc54415e550 0x7fc54415e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.619+0000 7fc54b7f5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc544161140 0x7fc54406c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.619+0000 7fc54b7f5640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc544157870 con 0x7fc54406bfe0 2024-05-20T22:08:16.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.617+0000 7f25791db640 1 -- start start 2024-05-20T22:08:16.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.619+0000 7fc54b7f5640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fc54406c930 con 0x7fc54415e550 2024-05-20T22:08:16.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.620+0000 7fc54b7f5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc544073310 con 0x7fc544161140 2024-05-20T22:08:16.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.620+0000 7fc549591640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc54415e550 0x7fc54415e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.620+0000 7fc549d92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc544161140 0x7fc54406c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.620+0000 7fc549591640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc54415e550 0x7fc54415e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58180/0 (socket says 172.21.15.55:58180) 2024-05-20T22:08:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.620+0000 7fc549d92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc544161140 0x7fc54406c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47356/0 (socket says 172.21.15.55:47356) 2024-05-20T22:08:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.620+0000 7fc549591640 1 -- 172.21.15.55:0/3160099456 learned_addr learned my addr 172.21.15.55:0/3160099456 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.620+0000 7f25791db640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f256c053de0 0x7f256c0e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.620+0000 7f25791db640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c034060 0x7f256c0e0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f25791db640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f256c050460 con 0x7f256c055810 2024-05-20T22:08:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f25791db640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f256c0e3850 con 0x7f256c053de0 2024-05-20T22:08:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f25791db640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f256c0e3520 con 0x7f256c034060 2024-05-20T22:08:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f2573fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f256c053de0 0x7f256c0e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f2573fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f256c053de0 0x7f256c0e3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58196/0 (socket says 172.21.15.55:58196) 2024-05-20T22:08:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f2573fff640 1 -- 172.21.15.55:0/3327173051 learned_addr learned my addr 172.21.15.55:0/3327173051 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f25789da640 1 --2- 172.21.15.55:0/3327173051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c034060 0x7f256c0e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.619+0000 7f309302b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084055810 0x7f3084033950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.619+0000 7f309302b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3084033e90 0x7f3084006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.619+0000 7f309302b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30840342a0 con 0x7f3084053de0 2024-05-20T22:08:16.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.619+0000 7f309302b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f30840e3850 con 0x7f3084033e90 2024-05-20T22:08:16.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.619+0000 7f309302b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f30840e3520 con 0x7f3084055810 2024-05-20T22:08:16.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.620+0000 7f3092029640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3084053de0 0x7f30840e3ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46444/0 (socket says 172.21.15.55:46444) 2024-05-20T22:08:16.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.620+0000 7f3092029640 1 -- 172.21.15.55:0/2948420846 learned_addr learned my addr 172.21.15.55:0/2948420846 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:16.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f2572ffd640 1 -- 172.21.15.55:0/3327173051 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2675845308 0 0) 0x7f256c050460 con 0x7f256c055810 2024-05-20T22:08:16.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f2572ffd640 1 -- 172.21.15.55:0/3327173051 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2564003520 con 0x7f256c055810 2024-05-20T22:08:16.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.622+0000 7f3091828640 1 --2- 172.21.15.55:0/2948420846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084055810 0x7f3084033950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f25789da640 1 -- 172.21.15.55:0/3327173051 >> v2:172.21.15.55:3300/0 conn(0x7f256c053de0 msgr2=0x7f256c0e3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:16.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.622+0000 7fc53bfff640 1 -- 172.21.15.55:0/3160099456 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 671181192 0 0) 0x7fc544157870 con 0x7fc54406bfe0 2024-05-20T22:08:16.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f25789da640 1 --2- 172.21.15.55:0/3327173051 >> v2:172.21.15.55:3300/0 conn(0x7f256c053de0 0x7f256c0e3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f25789da640 1 -- 172.21.15.55:0/3327173051 >> v1:172.21.15.138:6789/0 conn(0x7f256c055810 legacy=0x7f256c033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.621+0000 7f25789da640 1 -- 172.21.15.55:0/3327173051 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c050700 con 0x7f256c034060 2024-05-20T22:08:16.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.622+0000 7fc549d92640 1 -- 172.21.15.55:0/3160099456 >> v2:172.21.15.55:3300/0 conn(0x7fc54415e550 msgr2=0x7fc54415e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:16.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.622+0000 7fc549d92640 1 --2- 172.21.15.55:0/3160099456 >> v2:172.21.15.55:3300/0 conn(0x7fc54415e550 0x7fc54415e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.622+0000 7fc549d92640 1 -- 172.21.15.55:0/3160099456 >> v1:172.21.15.55:6789/0 conn(0x7fc54406bfe0 legacy=0x7fc544160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.622+0000 7fc549d92640 1 -- 172.21.15.55:0/3160099456 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5440735f0 con 0x7fc544161140 2024-05-20T22:08:16.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.623+0000 7f25789da640 1 --2- 172.21.15.55:0/3327173051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c034060 0x7f256c0e0350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f257406a560 tx=0x7f25740a5e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff2b17454f612277 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.623+0000 7f2572ffd640 1 -- 172.21.15.55:0/3327173051 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740b3070 con 0x7f256c034060 2024-05-20T22:08:16.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.623+0000 7f2572ffd640 1 -- 172.21.15.55:0/3327173051 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25740a6a00 con 0x7f256c034060 2024-05-20T22:08:16.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.623+0000 7f3091828640 1 -- 172.21.15.55:0/2948420846 >> v2:172.21.15.55:3300/0 conn(0x7f3084033e90 msgr2=0x7f3084006090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:16.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.623+0000 7f3091828640 1 --2- 172.21.15.55:0/2948420846 >> v2:172.21.15.55:3300/0 conn(0x7f3084033e90 0x7f3084006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.623+0000 7f3091828640 1 -- 172.21.15.55:0/2948420846 >> v1:172.21.15.55:6789/0 conn(0x7f3084053de0 legacy=0x7f30840e3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.623+0000 7f3091828640 1 -- 172.21.15.55:0/2948420846 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3084050700 con 0x7f3084055810 2024-05-20T22:08:16.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.623+0000 7f2572ffd640 1 -- 172.21.15.55:0/3327173051 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740ae440 con 0x7f256c034060 2024-05-20T22:08:16.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.624+0000 7fc549d92640 1 --2- 172.21.15.55:0/3160099456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc544161140 0x7fc54406c3f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc5400097a0 tx=0x7fc54002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e736072b7b6d642 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7fc53bfff640 1 -- 172.21.15.55:0/3160099456 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540037070 con 0x7fc544161140 2024-05-20T22:08:16.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7fc53bfff640 1 -- 172.21.15.55:0/3160099456 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc540032440 con 0x7fc544161140 2024-05-20T22:08:16.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f25791db640 1 -- 172.21.15.55:0/3327173051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c034060 msgr2=0x7f256c0e0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f25791db640 1 --2- 172.21.15.55:0/3327173051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c034060 0x7f256c0e0350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f257406a560 tx=0x7f25740a5e00 comp rx=0 tx=0).stop 2024-05-20T22:08:16.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f3091828640 1 --2- 172.21.15.55:0/2948420846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084055810 0x7f3084033950 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30880097a0 tx=0x7f308802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b99964c9271d5d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f25791db640 1 -- 172.21.15.55:0/3327173051 shutdown_connections 2024-05-20T22:08:16.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f25791db640 1 --2- 172.21.15.55:0/3327173051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c034060 0x7f256c0e0350 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f25791db640 1 --2- 172.21.15.55:0/3327173051 >> v2:172.21.15.55:3300/0 conn(0x7f256c053de0 0x7f256c0e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f25791db640 1 -- 172.21.15.55:0/3327173051 >> 172.21.15.55:0/3327173051 conn(0x7f256c054cf0 msgr2=0x7f256c055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:16.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7fc53bfff640 1 -- 172.21.15.55:0/3160099456 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540032870 con 0x7fc544161140 2024-05-20T22:08:16.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f25791db640 1 -- 172.21.15.55:0/3327173051 shutdown_connections 2024-05-20T22:08:16.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f3091027640 1 -- 172.21.15.55:0/2948420846 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088037070 con 0x7f3084055810 2024-05-20T22:08:16.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f3091027640 1 -- 172.21.15.55:0/2948420846 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308802adb0 con 0x7f3084055810 2024-05-20T22:08:16.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.625+0000 7f25791db640 1 -- 172.21.15.55:0/3327173051 wait complete. 2024-05-20T22:08:16.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.627+0000 7f309302b640 1 -- 172.21.15.55:0/2948420846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084055810 msgr2=0x7f3084033950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.627+0000 7f309302b640 1 --2- 172.21.15.55:0/2948420846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084055810 0x7f3084033950 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30880097a0 tx=0x7f308802a060 comp rx=0 tx=0).stop 2024-05-20T22:08:16.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.626+0000 7fc54b7f5640 1 -- 172.21.15.55:0/3160099456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc544161140 msgr2=0x7fc54406c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.626+0000 7fc54b7f5640 1 --2- 172.21.15.55:0/3160099456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc544161140 0x7fc54406c3f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc5400097a0 tx=0x7fc54002a1c0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.626+0000 7fc54b7f5640 1 -- 172.21.15.55:0/3160099456 shutdown_connections 2024-05-20T22:08:16.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.626+0000 7fc54b7f5640 1 --2- 172.21.15.55:0/3160099456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc544161140 0x7fc54406c3f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.626+0000 7fc54b7f5640 1 --2- 172.21.15.55:0/3160099456 >> v2:172.21.15.55:3300/0 conn(0x7fc54415e550 0x7fc54415e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.626+0000 7fc54b7f5640 1 -- 172.21.15.55:0/3160099456 >> 172.21.15.55:0/3160099456 conn(0x7fc5440b3980 msgr2=0x7fc5440b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:16.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.626+0000 7fc54b7f5640 1 -- 172.21.15.55:0/3160099456 shutdown_connections 2024-05-20T22:08:16.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.626+0000 7fc54b7f5640 1 -- 172.21.15.55:0/3160099456 wait complete. 2024-05-20T22:08:16.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.628+0000 7f309302b640 1 -- 172.21.15.55:0/2948420846 shutdown_connections 2024-05-20T22:08:16.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.628+0000 7f309302b640 1 --2- 172.21.15.55:0/2948420846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084055810 0x7f3084033950 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.628+0000 7f309302b640 1 --2- 172.21.15.55:0/2948420846 >> v2:172.21.15.55:3300/0 conn(0x7f3084033e90 0x7f3084006090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.628+0000 7f309302b640 1 -- 172.21.15.55:0/2948420846 >> 172.21.15.55:0/2948420846 conn(0x7f3084054cf0 msgr2=0x7f3084055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:16.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.628+0000 7f25791db640 1 Processor -- start 2024-05-20T22:08:16.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.628+0000 7f309302b640 1 -- 172.21.15.55:0/2948420846 shutdown_connections 2024-05-20T22:08:16.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.629+0000 7f25791db640 1 -- start start 2024-05-20T22:08:16.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.630+0000 7fc54b7f5640 1 Processor -- start 2024-05-20T22:08:16.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.630+0000 7fc54b7f5640 1 -- start start 2024-05-20T22:08:16.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.631+0000 7f25791db640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f256c053de0 0x7f256c0dd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.628+0000 7f309302b640 1 -- 172.21.15.55:0/2948420846 wait complete. 2024-05-20T22:08:16.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.631+0000 7f309302b640 1 Processor -- start 2024-05-20T22:08:16.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.632+0000 7f25791db640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256c055810 0x7f256c17ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.632+0000 7f2573fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f256c053de0 0x7f256c0dd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.632+0000 7f2573fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f256c053de0 0x7f256c0dd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58216/0 (socket says 172.21.15.55:58216) 2024-05-20T22:08:16.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.632+0000 7f25791db640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c04c7f0 0x7f256c049870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.632+0000 7f25737fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256c055810 0x7f256c17ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.632+0000 7f25737fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256c055810 0x7f256c17ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:51064/0 (socket says 172.21.15.55:51064) 2024-05-20T22:08:16.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.632+0000 7f25737fe640 1 -- 172.21.15.55:0/4054080149 learned_addr learned my addr 172.21.15.55:0/4054080149 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.632+0000 7f25791db640 1 -- 172.21.15.55:0/4054080149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f256c050cc0 con 0x7f256c053de0 2024-05-20T22:08:16.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.633+0000 7f25791db640 1 -- 172.21.15.55:0/4054080149 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f256c0e4220 con 0x7f256c055810 2024-05-20T22:08:16.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.633+0000 7fc54b7f5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54406bfe0 0x7fc544149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.633+0000 7f25789da640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c04c7f0 0x7f256c049870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.633+0000 7f25791db640 1 -- 172.21.15.55:0/4054080149 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f256c0e0890 con 0x7f256c04c7f0 2024-05-20T22:08:16.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.633+0000 7fc549591640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54406bfe0 0x7fc544149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.633+0000 7fc549591640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54406bfe0 0x7fc544149fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58224/0 (socket says 172.21.15.55:58224) 2024-05-20T22:08:16.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.633+0000 7fc549591640 1 -- 172.21.15.55:0/708652368 learned_addr learned my addr 172.21.15.55:0/708652368 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.633+0000 7fc54b7f5640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54415e550 0x7fc544150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.633+0000 7fc54b7f5640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544153710 0x7fc544153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7fc548d90640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54415e550 0x7fc544150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.631+0000 7f309302b640 1 -- start start 2024-05-20T22:08:16.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f309302b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084033e90 0x7f30840f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7fc549d92640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544153710 0x7fc544153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f309302b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3084053de0 0x7f30840f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f309302b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3084055810 0x7f30840f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f309302b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30840065d0 con 0x7f3084053de0 2024-05-20T22:08:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f309302b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3084050cc0 con 0x7f3084055810 2024-05-20T22:08:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f309302b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3084006750 con 0x7f3084033e90 2024-05-20T22:08:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7fc54b7f5640 1 -- 172.21.15.55:0/708652368 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc54406bac0 con 0x7fc54406bfe0 2024-05-20T22:08:16.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f3092029640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084033e90 0x7f30840f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f3091828640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3084053de0 0x7f30840f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f3091828640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3084053de0 0x7f30840f7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58234/0 (socket says 172.21.15.55:58234) 2024-05-20T22:08:16.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f3091828640 1 -- 172.21.15.55:0/469648855 learned_addr learned my addr 172.21.15.55:0/469648855 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f25789da640 1 -- 172.21.15.55:0/4054080149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256c055810 msgr2=0x7f256c17ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f25789da640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256c055810 0x7f256c17ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f25789da640 1 -- 172.21.15.55:0/4054080149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f256c053de0 msgr2=0x7f256c0dd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f25789da640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f256c053de0 0x7f256c0dd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f25789da640 1 -- 172.21.15.55:0/4054080149 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c17cf60 con 0x7f256c04c7f0 2024-05-20T22:08:16.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.634+0000 7f2573fff640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f256c053de0 0x7f256c0dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:16.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7f309282a640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3084055810 0x7f30840f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7fc54b7f5640 1 -- 172.21.15.55:0/708652368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc54406b7e0 con 0x7fc544153710 2024-05-20T22:08:16.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7f3091828640 1 -- 172.21.15.55:0/469648855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084033e90 msgr2=0x7f30840f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7f3091828640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084033e90 0x7f30840f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7f3091828640 1 -- 172.21.15.55:0/469648855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3084055810 msgr2=0x7f30840f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7f3091828640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3084055810 0x7f30840f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7f3091828640 1 -- 172.21.15.55:0/469648855 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30840f7280 con 0x7f3084053de0 2024-05-20T22:08:16.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7f25789da640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c04c7f0 0x7f256c049870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2574051020 tx=0x7f25740750a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7f2570ff9640 1 -- 172.21.15.55:0/4054080149 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740b3050 con 0x7f256c04c7f0 2024-05-20T22:08:16.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7f25791db640 1 -- 172.21.15.55:0/4054080149 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256c17d470 con 0x7f256c04c7f0 2024-05-20T22:08:16.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7f25791db640 1 -- 172.21.15.55:0/4054080149 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f256c04bff0 con 0x7f256c04c7f0 2024-05-20T22:08:16.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.636+0000 7f2570ff9640 1 -- 172.21.15.55:0/4054080149 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25740b4040 con 0x7f256c04c7f0 2024-05-20T22:08:16.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.636+0000 7f3091828640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3084053de0 0x7f30840f7ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f30880098c0 tx=0x7f30880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.637+0000 7f3082ffd640 1 -- 172.21.15.55:0/469648855 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088042020 con 0x7f3084053de0 2024-05-20T22:08:16.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.637+0000 7f2570ff9640 1 -- 172.21.15.55:0/4054080149 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740aedd0 con 0x7f256c04c7f0 2024-05-20T22:08:16.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.637+0000 7f309302b640 1 -- 172.21.15.55:0/469648855 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30840f6fa0 con 0x7f3084053de0 2024-05-20T22:08:16.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.637+0000 7f309302b640 1 -- 172.21.15.55:0/469648855 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30840dcfd0 con 0x7f3084053de0 2024-05-20T22:08:16.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7fc54b7f5640 1 -- 172.21.15.55:0/708652368 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc544072690 con 0x7fc54415e550 2024-05-20T22:08:16.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7fc548d90640 1 -- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544153710 msgr2=0x7fc544153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7fc548d90640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544153710 0x7fc544153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7fc548d90640 1 -- 172.21.15.55:0/708652368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54406bfe0 msgr2=0x7fc544149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7fc548d90640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54406bfe0 0x7fc544149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.635+0000 7fc548d90640 1 -- 172.21.15.55:0/708652368 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc544154060 con 0x7fc54415e550 2024-05-20T22:08:16.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.636+0000 7fc548d90640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54415e550 0x7fc544150240 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c780 tx=0x7fc53c00cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.636+0000 7fc539ffb640 1 -- 172.21.15.55:0/708652368 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c003b90 con 0x7fc54415e550 2024-05-20T22:08:16.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.636+0000 7fc539ffb640 1 -- 172.21.15.55:0/708652368 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc53c003d30 con 0x7fc54415e550 2024-05-20T22:08:16.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.638+0000 7fc539ffb640 1 -- 172.21.15.55:0/708652368 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c005310 con 0x7fc54415e550 2024-05-20T22:08:16.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.638+0000 7fc549d92640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544153710 0x7fc544153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:16.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.638+0000 7f3082ffd640 1 -- 172.21.15.55:0/469648855 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308802adb0 con 0x7f3084053de0 2024-05-20T22:08:16.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.638+0000 7fc54b7f5640 1 -- 172.21.15.55:0/708652368 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc544150780 con 0x7fc54415e550 2024-05-20T22:08:16.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.639+0000 7fc54b7f5640 1 -- 172.21.15.55:0/708652368 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54414e0a0 con 0x7fc54415e550 2024-05-20T22:08:16.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.639+0000 7f25791db640 1 -- 172.21.15.55:0/4054080149 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2554005290 con 0x7f256c04c7f0 2024-05-20T22:08:16.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.640+0000 7f3082ffd640 1 -- 172.21.15.55:0/469648855 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088042020 con 0x7f3084053de0 2024-05-20T22:08:16.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.640+0000 7fc54b7f5640 1 -- 172.21.15.55:0/708652368 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc544069090 con 0x7fc54415e550 2024-05-20T22:08:16.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.641+0000 7fc539ffb640 1 -- 172.21.15.55:0/708652368 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc53c0054b0 con 0x7fc54415e550 2024-05-20T22:08:16.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.642+0000 7f309302b640 1 -- 172.21.15.55:0/469648855 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3074005290 con 0x7f3084053de0 2024-05-20T22:08:16.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.642+0000 7fc539ffb640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc52807cec0 0x7fc52807f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.642+0000 7fc539ffb640 1 -- 172.21.15.55:0/708652368 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc53c08ae90 con 0x7fc54415e550 2024-05-20T22:08:16.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.642+0000 7f2570ff9640 1 -- 172.21.15.55:0/4054080149 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25740c1070 con 0x7f256c04c7f0 2024-05-20T22:08:16.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.643+0000 7f3082ffd640 1 -- 172.21.15.55:0/469648855 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3088045020 con 0x7f3084053de0 2024-05-20T22:08:16.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.643+0000 7fc549591640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc52807cec0 0x7fc52807f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.643+0000 7fc549591640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc52807cec0 0x7fc52807f3d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc534009b20 tx=0x7fc534008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.643+0000 7f2570ff9640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f255806bf30 0x7f255806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.643+0000 7f2570ff9640 1 -- 172.21.15.55:0/4054080149 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2574126540 con 0x7f256c04c7f0 2024-05-20T22:08:16.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.644+0000 7f3082ffd640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f307006c230 0x7f307006e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.644+0000 7f2573fff640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f255806bf30 0x7f255806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.644+0000 7f3082ffd640 1 -- 172.21.15.55:0/469648855 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f30880307b0 con 0x7f3084053de0 2024-05-20T22:08:16.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.647+0000 7f3092029640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f307006c230 0x7f307006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.647+0000 7fc539ffb640 1 -- 172.21.15.55:0/708652368 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc544154060 con 0x7fc54415e550 2024-05-20T22:08:16.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.647+0000 7f2573fff640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f255806bf30 0x7f255806e440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f256c00d6d0 tx=0x7f2560009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.648+0000 7f3092029640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f307006c230 0x7f307006e740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f308c074b60 tx=0x7f308c068500 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.651+0000 7f2570ff9640 1 -- 172.21.15.55:0/4054080149 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f25740f44b0 con 0x7f256c04c7f0 2024-05-20T22:08:16.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.655+0000 7f3082ffd640 1 -- 172.21.15.55:0/469648855 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f308804c070 con 0x7f3084053de0 2024-05-20T22:08:16.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.675+0000 7f3f0ddff640 1 Processor -- start 2024-05-20T22:08:16.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.676+0000 7f3f0ddff640 1 -- start start 2024-05-20T22:08:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.678+0000 7f3f0ddff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f00053de0 0x7f3f000e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.678+0000 7f3f0ddff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3f00055810 0x7f3f00033950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.678+0000 7f3f0ddff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3f00033e90 0x7f3f00006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.678+0000 7f3f0ddff640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f3f000e3850 con 0x7f3f00033e90 2024-05-20T22:08:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.678+0000 7f3f0ddff640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3f000342a0 con 0x7f3f00055810 2024-05-20T22:08:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.678+0000 7f3f0ddff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f000e3520 con 0x7f3f00053de0 2024-05-20T22:08:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.678+0000 7f3f0d5fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3f00033e90 0x7f3f00006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.678+0000 7f3f0d5fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3f00033e90 0x7f3f00006090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58242/0 (socket says 172.21.15.55:58242) 2024-05-20T22:08:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.678+0000 7f3f0d5fe640 1 -- 172.21.15.55:0/736720708 learned_addr learned my addr 172.21.15.55:0/736720708 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.679+0000 7f3f0d5fe640 1 -- 172.21.15.55:0/736720708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f00053de0 msgr2=0x7f3f000e3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.679+0000 7f3f0d5fe640 1 --2- 172.21.15.55:0/736720708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f00053de0 0x7f3f000e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.679+0000 7f3f0d5fe640 1 -- 172.21.15.55:0/736720708 >> v2:172.21.15.138:3300/0 conn(0x7f3f00055810 msgr2=0x7f3f00033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.679+0000 7f3f0d5fe640 1 --2- 172.21.15.55:0/736720708 >> v2:172.21.15.138:3300/0 conn(0x7f3f00055810 0x7f3f00033950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.679+0000 7f3f0d5fe640 1 -- 172.21.15.55:0/736720708 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f00050700 con 0x7f3f00033e90 2024-05-20T22:08:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.680+0000 7f3f0d5fe640 1 --2- 172.21.15.55:0/736720708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 0x7f3f00006090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f00034450 tx=0x7f3f080a5ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f6d84ef40c53a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.681+0000 7f3f077fe640 1 -- 172.21.15.55:0/736720708 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f080b3070 con 0x7f3f00033e90 2024-05-20T22:08:16.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.681+0000 7f3f077fe640 1 -- 172.21.15.55:0/736720708 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f080a6ae0 con 0x7f3f00033e90 2024-05-20T22:08:16.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.681+0000 7f3f077fe640 1 -- 172.21.15.55:0/736720708 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f080ae440 con 0x7f3f00033e90 2024-05-20T22:08:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.682+0000 7f3f0ddff640 1 -- 172.21.15.55:0/736720708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 msgr2=0x7f3f00006090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.682+0000 7f3f0ddff640 1 --2- 172.21.15.55:0/736720708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 0x7f3f00006090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f00034450 tx=0x7f3f080a5ee0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.683+0000 7f3f0ddff640 1 -- 172.21.15.55:0/736720708 shutdown_connections 2024-05-20T22:08:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.683+0000 7f3f0ddff640 1 --2- 172.21.15.55:0/736720708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 0x7f3f00006090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.683+0000 7f3f0ddff640 1 --2- 172.21.15.55:0/736720708 >> v2:172.21.15.138:3300/0 conn(0x7f3f00055810 0x7f3f00033950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.683+0000 7f3f0ddff640 1 --2- 172.21.15.55:0/736720708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f00053de0 0x7f3f000e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.683+0000 7f3f0ddff640 1 -- 172.21.15.55:0/736720708 >> 172.21.15.55:0/736720708 conn(0x7f3f00054cf0 msgr2=0x7f3f00055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.682+0000 7f267d757640 1 Processor -- start 2024-05-20T22:08:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.682+0000 7f267d757640 1 -- start start 2024-05-20T22:08:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.683+0000 7f3f0ddff640 1 -- 172.21.15.55:0/736720708 shutdown_connections 2024-05-20T22:08:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.683+0000 7f3f0ddff640 1 -- 172.21.15.55:0/736720708 wait complete. 2024-05-20T22:08:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.684+0000 7f7c29d3b640 1 Processor -- start 2024-05-20T22:08:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f7c29d3b640 1 -- start start 2024-05-20T22:08:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f267d757640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26700e7cf0 0x7f26700514c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f267d757640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2670051a00 0x7f2670051e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f267d757640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 0x7f2670052af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f267d757640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f2670052350 con 0x7f26700e7cf0 2024-05-20T22:08:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f267d757640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f267003c1f0 con 0x7f2670051a00 2024-05-20T22:08:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f267d757640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2670054a40 con 0x7f26700526e0 2024-05-20T22:08:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f2676ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26700e7cf0 0x7f26700514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f2676ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26700e7cf0 0x7f26700514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58248/0 (socket says 172.21.15.55:58248) 2024-05-20T22:08:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f2676ffd640 1 -- 172.21.15.55:0/4095028573 learned_addr learned my addr 172.21.15.55:0/4095028573 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f26777fe640 1 --2- 172.21.15.55:0/4095028573 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 0x7f2670052af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.685+0000 7f26767fc640 1 --2- 172.21.15.55:0/4095028573 >> v2:172.21.15.138:3300/0 conn(0x7f2670051a00 0x7f2670051e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.686+0000 7f26777fe640 1 -- 172.21.15.55:0/4095028573 >> v2:172.21.15.138:3300/0 conn(0x7f2670051a00 msgr2=0x7f2670051e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.686+0000 7f26777fe640 1 --2- 172.21.15.55:0/4095028573 >> v2:172.21.15.138:3300/0 conn(0x7f2670051a00 0x7f2670051e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.686+0000 7f26777fe640 1 -- 172.21.15.55:0/4095028573 >> v2:172.21.15.55:3300/0 conn(0x7f26700e7cf0 msgr2=0x7f26700514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.686+0000 7f26777fe640 1 --2- 172.21.15.55:0/4095028573 >> v2:172.21.15.55:3300/0 conn(0x7f26700e7cf0 0x7f26700514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.686+0000 7f26777fe640 1 -- 172.21.15.55:0/4095028573 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26700e8100 con 0x7f26700526e0 2024-05-20T22:08:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.686+0000 7f3f0ddff640 1 Processor -- start 2024-05-20T22:08:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.686+0000 7f3f0ddff640 1 -- start start 2024-05-20T22:08:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.687+0000 7f26777fe640 1 --2- 172.21.15.55:0/4095028573 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 0x7f2670052af0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2678069e80 tx=0x7f26780a5e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb2f5bf3d7c13dba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.688+0000 7f2675ffb640 1 -- 172.21.15.55:0/4095028573 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780b3070 con 0x7f26700526e0 2024-05-20T22:08:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.688+0000 7f2675ffb640 1 -- 172.21.15.55:0/4095028573 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26780a6a60 con 0x7f26700526e0 2024-05-20T22:08:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.688+0000 7f2675ffb640 1 -- 172.21.15.55:0/4095028573 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780ae440 con 0x7f26700526e0 2024-05-20T22:08:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.687+0000 7fba8dbe0640 1 Processor -- start 2024-05-20T22:08:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.687+0000 7fba8dbe0640 1 -- start start 2024-05-20T22:08:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.687+0000 7f7c29d3b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c053de0 0x7f7c1c0e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.687+0000 7f7c29d3b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c1c050460 con 0x7f7c1c033950 2024-05-20T22:08:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.687+0000 7f7c29d3b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c1c0e3520 con 0x7f7c1c0e0350 2024-05-20T22:08:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.687+0000 7f7c29d3b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1c0e3850 con 0x7f7c1c053de0 2024-05-20T22:08:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.687+0000 7f7c23fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7c1c033950 0x7f7c1c033d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46466/0 (socket says 172.21.15.55:46466) 2024-05-20T22:08:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.687+0000 7f7c23fff640 1 -- 172.21.15.55:0/1309061508 learned_addr learned my addr 172.21.15.55:0/1309061508 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.687+0000 7f7c28d39640 1 --2- 172.21.15.55:0/1309061508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c053de0 0x7f7c1c0e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.688+0000 7f7c237fe640 1 -- 172.21.15.55:0/1309061508 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4069121585 0 0) 0x7f7c1c050460 con 0x7f7c1c033950 2024-05-20T22:08:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.688+0000 7f7c237fe640 1 -- 172.21.15.55:0/1309061508 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c14003520 con 0x7f7c1c033950 2024-05-20T22:08:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.688+0000 7f7c28d39640 1 -- 172.21.15.55:0/1309061508 >> v1:172.21.15.138:6789/0 conn(0x7f7c1c0e0350 legacy=0x7f7c1c0e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.688+0000 7f7c28d39640 1 -- 172.21.15.55:0/1309061508 >> v1:172.21.15.55:6789/0 conn(0x7f7c1c033950 legacy=0x7f7c1c033d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.688+0000 7f7c28d39640 1 -- 172.21.15.55:0/1309061508 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c050700 con 0x7f7c1c053de0 2024-05-20T22:08:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.689+0000 7f1b7739d640 1 Processor -- start 2024-05-20T22:08:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.689+0000 7f3f0ddff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 0x7f3f0004cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.689+0000 7f1b7739d640 1 -- start start 2024-05-20T22:08:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.689+0000 7f3f0ddff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f00053de0 0x7f3f0004d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.689+0000 7fba8dbe0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 0x7fba800541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.689+0000 7fba8dbe0640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba8003c6c0 0x7fba800f1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.689+0000 7fba8dbe0640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba80057380 con 0x7fba80058700 2024-05-20T22:08:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.689+0000 7fba8dbe0640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fba80058270 con 0x7fba8003c6c0 2024-05-20T22:08:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.689+0000 7fba8dbe0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fba80057660 con 0x7fba80056d30 2024-05-20T22:08:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7fba877fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 0x7fba800541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7fba877fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 0x7fba800541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47446/0 (socket says 172.21.15.55:47446) 2024-05-20T22:08:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7fba877fe640 1 -- 172.21.15.55:0/3908579514 learned_addr learned my addr 172.21.15.55:0/3908579514 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7fba87fff640 1 --2- 172.21.15.55:0/3908579514 >> v2:172.21.15.55:3300/0 conn(0x7fba8003c6c0 0x7fba800f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f267d757640 1 -- 172.21.15.55:0/4095028573 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 msgr2=0x7f2670052af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f267d757640 1 --2- 172.21.15.55:0/4095028573 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 0x7f2670052af0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2678069e80 tx=0x7f26780a5e60 comp rx=0 tx=0).stop 2024-05-20T22:08:16.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f267d757640 1 -- 172.21.15.55:0/4095028573 shutdown_connections 2024-05-20T22:08:16.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f267d757640 1 --2- 172.21.15.55:0/4095028573 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 0x7f2670052af0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f267d757640 1 --2- 172.21.15.55:0/4095028573 >> v2:172.21.15.138:3300/0 conn(0x7f2670051a00 0x7f2670051e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f267d757640 1 --2- 172.21.15.55:0/4095028573 >> v2:172.21.15.55:3300/0 conn(0x7f26700e7cf0 0x7f26700514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f267d757640 1 -- 172.21.15.55:0/4095028573 >> 172.21.15.55:0/4095028573 conn(0x7f267000c3f0 msgr2=0x7f267000c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:16.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7fba877fe640 1 -- 172.21.15.55:0/3908579514 >> v2:172.21.15.55:3300/0 conn(0x7fba8003c6c0 msgr2=0x7fba800f1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:16.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7fba877fe640 1 --2- 172.21.15.55:0/3908579514 >> v2:172.21.15.55:3300/0 conn(0x7fba8003c6c0 0x7fba800f1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7fba877fe640 1 -- 172.21.15.55:0/3908579514 >> v1:172.21.15.55:6789/0 conn(0x7fba80058700 legacy=0x7fba8003bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:16.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7fba877fe640 1 -- 172.21.15.55:0/3908579514 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba80057940 con 0x7fba80056d30 2024-05-20T22:08:16.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.689+0000 7f3f0ddff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f00055810 0x7f3f0004d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f3f0cdfd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 0x7f3f0004cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f3f07fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f00053de0 0x7f3f0004d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f3f07fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f00053de0 0x7f3f0004d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:51118/0 (socket says 172.21.15.55:51118) 2024-05-20T22:08:16.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f3f0cdfd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 0x7f3f0004cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58262/0 (socket says 172.21.15.55:58262) 2024-05-20T22:08:16.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f3f07fff640 1 -- 172.21.15.55:0/759231019 learned_addr learned my addr 172.21.15.55:0/759231019 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f267d757640 1 -- 172.21.15.55:0/4095028573 shutdown_connections 2024-05-20T22:08:16.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f267d757640 1 -- 172.21.15.55:0/4095028573 wait complete. 2024-05-20T22:08:16.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f3f0d5fe640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f00055810 0x7f3f0004d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00050cc0 con 0x7f3f00033e90 2024-05-20T22:08:16.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000e4220 con 0x7f3f00053de0 2024-05-20T22:08:16.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.690+0000 7f7c28d39640 1 --2- 172.21.15.55:0/1309061508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c053de0 0x7f7c1c0e3ce0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7c180097d0 tx=0x7f7c1802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa68009e7a78479 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f7c237fe640 1 -- 172.21.15.55:0/1309061508 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18037070 con 0x7f7c1c053de0 2024-05-20T22:08:16.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f7c237fe640 1 -- 172.21.15.55:0/1309061508 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c18032440 con 0x7f7c1c053de0 2024-05-20T22:08:16.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f7c237fe640 1 -- 172.21.15.55:0/1309061508 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18032850 con 0x7f7c1c053de0 2024-05-20T22:08:16.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f000065d0 con 0x7f3f00055810 2024-05-20T22:08:16.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.691+0000 7fba877fe640 1 --2- 172.21.15.55:0/3908579514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 0x7fba800541c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba800583f0 tx=0x7fba7c02a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26f43f323ef4164 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7fba867fc640 1 -- 172.21.15.55:0/3908579514 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c037070 con 0x7fba80056d30 2024-05-20T22:08:16.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7fba867fc640 1 -- 172.21.15.55:0/3908579514 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba7c032440 con 0x7fba80056d30 2024-05-20T22:08:16.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f3f0cdfd640 1 -- 172.21.15.55:0/759231019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f00055810 msgr2=0x7f3f0004d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f3f0cdfd640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f00055810 0x7f3f0004d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f7c29d3b640 1 -- 172.21.15.55:0/1309061508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c053de0 msgr2=0x7f7c1c0e3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f7c29d3b640 1 --2- 172.21.15.55:0/1309061508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c053de0 0x7f7c1c0e3ce0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7c180097d0 tx=0x7f7c1802a1c0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f7c29d3b640 1 -- 172.21.15.55:0/1309061508 shutdown_connections 2024-05-20T22:08:16.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f7c29d3b640 1 --2- 172.21.15.55:0/1309061508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c053de0 0x7f7c1c0e3ce0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7c180097d0 tx=0x7f7c1802a1c0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f7c29d3b640 1 -- 172.21.15.55:0/1309061508 >> 172.21.15.55:0/1309061508 conn(0x7f7c1c054cf0 msgr2=0x7f7c1c055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f1b7739d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b680540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f1b7739d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1b68007460 0x7f1b6803d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f3f0cdfd640 1 -- 172.21.15.55:0/759231019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f00053de0 msgr2=0x7f3f0004d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f3f0cdfd640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f00053de0 0x7f3f0004d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f7c29d3b640 1 -- 172.21.15.55:0/1309061508 shutdown_connections 2024-05-20T22:08:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f3f0cdfd640 1 -- 172.21.15.55:0/759231019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0002c660 con 0x7f3f00033e90 2024-05-20T22:08:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f3f07fff640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f00053de0 0x7f3f0004d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:16.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7fba867fc640 1 -- 172.21.15.55:0/3908579514 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c032810 con 0x7fba80056d30 2024-05-20T22:08:16.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f3f0cdfd640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 0x7f3f0004cbd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400b440 tx=0x7f3ef400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7fba8dbe0640 1 -- 172.21.15.55:0/3908579514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 msgr2=0x7fba800541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7fba8dbe0640 1 --2- 172.21.15.55:0/3908579514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 0x7fba800541c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba800583f0 tx=0x7fba7c02a200 comp rx=0 tx=0).stop 2024-05-20T22:08:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f7c29d3b640 1 -- 172.21.15.55:0/1309061508 wait complete. 2024-05-20T22:08:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7fba8dbe0640 1 -- 172.21.15.55:0/3908579514 shutdown_connections 2024-05-20T22:08:16.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f1b7739d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b680549f0 con 0x7f1b680545e0 2024-05-20T22:08:16.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f1b7739d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1b680e3ca0 con 0x7f1b68007460 2024-05-20T22:08:16.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f1b7739d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1b68007990 con 0x7f1b680e3890 2024-05-20T22:08:16.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f1b7639b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b680540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f1b7639b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b680540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47472/0 (socket says 172.21.15.55:47472) 2024-05-20T22:08:16.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7fba8dbe0640 1 --2- 172.21.15.55:0/3908579514 >> v2:172.21.15.55:3300/0 conn(0x7fba8003c6c0 0x7fba800f1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f1b7639b640 1 -- 172.21.15.55:0/4082950489 learned_addr learned my addr 172.21.15.55:0/4082950489 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.692+0000 7f1b76b9c640 1 --2- 172.21.15.55:0/4082950489 >> v2:172.21.15.138:3300/0 conn(0x7f1b68007460 0x7f1b6803d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7fba8dbe0640 1 --2- 172.21.15.55:0/3908579514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 0x7fba800541c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7fba8dbe0640 1 -- 172.21.15.55:0/3908579514 >> 172.21.15.55:0/3908579514 conn(0x7fba8003ac50 msgr2=0x7fba8003b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7fba8dbe0640 1 -- 172.21.15.55:0/3908579514 shutdown_connections 2024-05-20T22:08:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f1b7639b640 1 -- 172.21.15.55:0/4082950489 >> v2:172.21.15.138:3300/0 conn(0x7f1b68007460 msgr2=0x7f1b6803d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f1b7639b640 1 --2- 172.21.15.55:0/4082950489 >> v2:172.21.15.138:3300/0 conn(0x7f1b68007460 0x7f1b6803d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f1b7639b640 1 -- 172.21.15.55:0/4082950489 >> v1:172.21.15.138:6789/0 conn(0x7f1b680545e0 legacy=0x7f1b68006d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T22:08:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.693+0000 7f1b7639b640 1 -- 172.21.15.55:0/4082950489 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b68007c70 con 0x7f1b680e3890 2024-05-20T22:08:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7f1b7639b640 1 --2- 172.21.15.55:0/4082950489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b680540a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009870 tx=0x7f1b6c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28cbc69c118b743 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7fba8dbe0640 1 -- 172.21.15.55:0/3908579514 wait complete. 2024-05-20T22:08:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7f3f057fa640 1 -- 172.21.15.55:0/759231019 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4011830 con 0x7f3f00033e90 2024-05-20T22:08:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7f267d757640 1 Processor -- start 2024-05-20T22:08:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7f3f057fa640 1 -- 172.21.15.55:0/759231019 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef4016070 con 0x7f3f00033e90 2024-05-20T22:08:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.694+0000 7f267d757640 1 -- start start 2024-05-20T22:08:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.695+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0004bb30 con 0x7f3f00033e90 2024-05-20T22:08:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.696+0000 7f7c29d3b640 1 Processor -- start 2024-05-20T22:08:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.696+0000 7f7c29d3b640 1 -- start start 2024-05-20T22:08:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.696+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f0004c0f0 con 0x7f3f00033e90 2024-05-20T22:08:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.696+0000 7f3f057fa640 1 -- 172.21.15.55:0/759231019 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4011c10 con 0x7f3f00033e90 2024-05-20T22:08:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.697+0000 7fba8dbe0640 1 Processor -- start 2024-05-20T22:08:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.697+0000 7fba8dbe0640 1 -- start start 2024-05-20T22:08:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.697+0000 7f267d757640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2670051a00 0x7f267004ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.698+0000 7f2676ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2670051a00 0x7f267004ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.698+0000 7f2676ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2670051a00 0x7f267004ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58274/0 (socket says 172.21.15.55:58274) 2024-05-20T22:08:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.698+0000 7f2676ffd640 1 -- 172.21.15.55:0/4268926060 learned_addr learned my addr 172.21.15.55:0/4268926060 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.698+0000 7f7c29d3b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1c033950 0x7f7c1c0e1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.698+0000 7f267d757640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 0x7f267004a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.698+0000 7f7c29d3b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c0e0350 0x7f7c1c0e2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f7c28d39640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1c033950 0x7f7c1c0e1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f7c28d39640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1c033950 0x7f7c1c0e1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58280/0 (socket says 172.21.15.55:58280) 2024-05-20T22:08:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f7c28d39640 1 -- 172.21.15.55:0/3781776549 learned_addr learned my addr 172.21.15.55:0/3781776549 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f7c29d3b640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c0dd280 0x7f7c1c034db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c050cc0 con 0x7f7c1c033950 2024-05-20T22:08:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c0e4220 con 0x7f7c1c0dd280 2024-05-20T22:08:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1c055810 con 0x7f7c1c0e0350 2024-05-20T22:08:16.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f267d757640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26700e7cf0 0x7f26700e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f26767fc640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 0x7f267004a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f267d757640 1 -- 172.21.15.55:0/4268926060 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f267003ca00 con 0x7f2670051a00 2024-05-20T22:08:16.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f26777fe640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26700e7cf0 0x7f26700e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f267d757640 1 -- 172.21.15.55:0/4268926060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f267003c720 con 0x7f26700e7cf0 2024-05-20T22:08:16.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f7c23fff640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c0e0350 0x7f7c1c0e2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f7c2953a640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c0dd280 0x7f7c1c034db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f267d757640 1 -- 172.21.15.55:0/4268926060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f267004db10 con 0x7f26700526e0 2024-05-20T22:08:16.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.697+0000 7f1b75399640 1 -- 172.21.15.55:0/4082950489 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c037070 con 0x7f1b680e3890 2024-05-20T22:08:16.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.698+0000 7f1b75399640 1 -- 172.21.15.55:0/4082950489 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c032440 con 0x7f1b680e3890 2024-05-20T22:08:16.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.698+0000 7f1b75399640 1 -- 172.21.15.55:0/4082950489 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c032810 con 0x7f1b680e3890 2024-05-20T22:08:16.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7fba8dbe0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba8003c6c0 0x7fba800d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7fba8dbe0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 0x7fba800ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7fba8dbe0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba80058700 0x7fba800e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7f26767fc640 1 -- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26700e7cf0 msgr2=0x7f26700e0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7f26767fc640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26700e7cf0 0x7f26700e0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7f26767fc640 1 -- 172.21.15.55:0/4268926060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2670051a00 msgr2=0x7f267004ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7f26767fc640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2670051a00 0x7f267004ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7f26767fc640 1 -- 172.21.15.55:0/4268926060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267004f7a0 con 0x7f26700526e0 2024-05-20T22:08:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7f26767fc640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 0x7f267004a9a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f266c00e8a0 tx=0x7f266c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7fba8dbe0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fba80058b10 con 0x7fba80058700 2024-05-20T22:08:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f265f7fe640 1 -- 172.21.15.55:0/4268926060 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c017070 con 0x7f26700526e0 2024-05-20T22:08:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f265f7fe640 1 -- 172.21.15.55:0/4268926060 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266c00c940 con 0x7f26700526e0 2024-05-20T22:08:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f265f7fe640 1 -- 172.21.15.55:0/4268926060 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c009e60 con 0x7f26700526e0 2024-05-20T22:08:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f267d757640 1 -- 172.21.15.55:0/4268926060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26700dfdf0 con 0x7f26700526e0 2024-05-20T22:08:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f267d757640 1 -- 172.21.15.55:0/4268926060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f26700e03b0 con 0x7f26700526e0 2024-05-20T22:08:16.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f7c23fff640 1 -- 172.21.15.55:0/3781776549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c0dd280 msgr2=0x7f7c1c034db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f7c23fff640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c0dd280 0x7f7c1c034db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f7c23fff640 1 -- 172.21.15.55:0/3781776549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1c033950 msgr2=0x7f7c1c0e1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f7c23fff640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1c033950 0x7f7c1c0e1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f7c23fff640 1 -- 172.21.15.55:0/3781776549 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c17e370 con 0x7f7c1c0e0350 2024-05-20T22:08:16.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f7c23fff640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c0e0350 0x7f7c1c0e2000 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c050630 tx=0x7f7c24074560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7fba877fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba8003c6c0 0x7fba800d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7fba877fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba8003c6c0 0x7fba800d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:51150/0 (socket says 172.21.15.55:51150) 2024-05-20T22:08:16.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7fba877fe640 1 -- 172.21.15.55:0/1956768266 learned_addr learned my addr 172.21.15.55:0/1956768266 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7fba87fff640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba80058700 0x7fba800e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7fba8dbe0640 1 -- 172.21.15.55:0/1956768266 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba80057f00 con 0x7fba8003c6c0 2024-05-20T22:08:16.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7fba8dbe0640 1 -- 172.21.15.55:0/1956768266 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fba80058c90 con 0x7fba80056d30 2024-05-20T22:08:16.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7f7c217fa640 1 -- 172.21.15.55:0/3781776549 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2406d310 con 0x7f7c1c0e0350 2024-05-20T22:08:16.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7f7c217fa640 1 -- 172.21.15.55:0/3781776549 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2406da70 con 0x7f7c1c0e0350 2024-05-20T22:08:16.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7f7c217fa640 1 -- 172.21.15.55:0/3781776549 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2408b7f0 con 0x7f7c1c0e0350 2024-05-20T22:08:16.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1c04e100 con 0x7f7c1c0e0350 2024-05-20T22:08:16.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1c04e6c0 con 0x7f7c1c0e0350 2024-05-20T22:08:16.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.699+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee8005290 con 0x7f3f00033e90 2024-05-20T22:08:16.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7f3f057fa640 1 -- 172.21.15.55:0/759231019 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ef401b460 con 0x7f3f00033e90 2024-05-20T22:08:16.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.700+0000 7f3f057fa640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3eec06be80 0x7f3eec06e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.701+0000 7f3f07fff640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3eec06be80 0x7f3eec06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7fba86ffd640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 0x7fba800ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7f3f07fff640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3eec06be80 0x7f3eec06e390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f000dee20 tx=0x7f3efc0090e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7fba87fff640 1 -- 172.21.15.55:0/1956768266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 msgr2=0x7fba800ea0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7fba87fff640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 0x7fba800ea0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7f1b7739d640 1 -- 172.21.15.55:0/4082950489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 msgr2=0x7f1b680540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7f1b7739d640 1 --2- 172.21.15.55:0/4082950489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b680540a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009870 tx=0x7f1b6c02a210 comp rx=0 tx=0).stop 2024-05-20T22:08:16.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7fba87fff640 1 -- 172.21.15.55:0/1956768266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba8003c6c0 msgr2=0x7fba800d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7fba87fff640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba8003c6c0 0x7fba800d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.702+0000 7fba87fff640 1 -- 172.21.15.55:0/1956768266 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba800d1900 con 0x7fba80058700 2024-05-20T22:08:16.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.703+0000 7fba87fff640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba80058700 0x7fba800e6e00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fba8806a560 tx=0x7fba88074560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.703+0000 7f3f057fa640 1 -- 172.21.15.55:0/759231019 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3ef4056d10 con 0x7f3f00033e90 2024-05-20T22:08:16.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.703+0000 7f265d7fa640 1 -- 172.21.15.55:0/4268926060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2650005290 con 0x7f26700526e0 2024-05-20T22:08:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.704+0000 7f1b7739d640 1 -- 172.21.15.55:0/4082950489 shutdown_connections 2024-05-20T22:08:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.704+0000 7f1b7739d640 1 --2- 172.21.15.55:0/4082950489 >> v2:172.21.15.138:3300/0 conn(0x7f1b68007460 0x7f1b6803d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.704+0000 7f1b7739d640 1 --2- 172.21.15.55:0/4082950489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b680540a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.704+0000 7f1b7739d640 1 -- 172.21.15.55:0/4082950489 >> 172.21.15.55:0/4082950489 conn(0x7f1b6803ac50 msgr2=0x7f1b6803b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.704+0000 7f1b7739d640 1 -- 172.21.15.55:0/4082950489 shutdown_connections 2024-05-20T22:08:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.704+0000 7f1b7739d640 1 -- 172.21.15.55:0/4082950489 wait complete. 2024-05-20T22:08:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.704+0000 7f265f7fe640 1 -- 172.21.15.55:0/4268926060 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f266c01a070 con 0x7f26700526e0 2024-05-20T22:08:16.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.705+0000 7f7c217fa640 1 -- 172.21.15.55:0/3781776549 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c2408b990 con 0x7f7c1c0e0350 2024-05-20T22:08:16.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.706+0000 7f265f7fe640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266406bf00 0x7f266406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.706+0000 7f7c217fa640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c0806be80 0x7f7c0806e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.706+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c1c005570 con 0x7f7c1c0e0350 2024-05-20T22:08:16.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.706+0000 7f7c217fa640 1 -- 172.21.15.55:0/3781776549 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7c241062c0 con 0x7f7c1c0e0350 2024-05-20T22:08:16.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.706+0000 7f2676ffd640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266406bf00 0x7f266406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.707+0000 7f265f7fe640 1 -- 172.21.15.55:0/4268926060 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f266c08b330 con 0x7f26700526e0 2024-05-20T22:08:16.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.707+0000 7f1b7739d640 1 Processor -- start 2024-05-20T22:08:16.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.707+0000 7f1b7739d640 1 -- start start 2024-05-20T22:08:16.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.707+0000 7f7c28d39640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c0806be80 0x7f7c0806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.707+0000 7f2676ffd640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266406bf00 0x7f266406e410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2668009ba0 tx=0x7f2668009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.708+0000 7f7c28d39640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c0806be80 0x7f7c0806e390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c1802a200 tx=0x7f7c1802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.709+0000 7f1b7739d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b68007460 0x7f1b6804d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.709+0000 7f1b7739d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b680545e0 0x7f1b68052d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.709+0000 7f1b7739d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b6804dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.709+0000 7f1b7739d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6803cf10 con 0x7f1b680545e0 2024-05-20T22:08:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.709+0000 7f1b7639b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b68007460 0x7f1b6804d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.709+0000 7f1b75b9a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b680545e0 0x7f1b68052d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.709+0000 7f1b7639b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b68007460 0x7f1b6804d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:51164/0 (socket says 172.21.15.55:51164) 2024-05-20T22:08:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.709+0000 7f1b7639b640 1 -- 172.21.15.55:0/983496441 learned_addr learned my addr 172.21.15.55:0/983496441 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:16.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.710+0000 7f265f7fe640 1 -- 172.21.15.55:0/4268926060 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f266c0592a0 con 0x7f26700526e0 2024-05-20T22:08:16.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.710+0000 7f1b76b9c640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b6804dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.710+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6803cc30 con 0x7f1b68007460 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.710+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6803d950 con 0x7f1b680e3890 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.710+0000 7f1b7639b640 1 -- 172.21.15.55:0/983496441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 msgr2=0x7f1b6804dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.710+0000 7f1b7639b640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b6804dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.710+0000 7f1b7639b640 1 -- 172.21.15.55:0/983496441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b680545e0 msgr2=0x7f1b68052d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.710+0000 7f1b7639b640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b680545e0 0x7f1b68052d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.710+0000 7f1b7639b640 1 -- 172.21.15.55:0/983496441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6804da60 con 0x7f1b68007460 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.711+0000 7f1b76b9c640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b6804dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.711+0000 7f1b7639b640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b68007460 0x7f1b6804d520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009990 tx=0x7f1b6c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.711+0000 7f1b66ffd640 1 -- 172.21.15.55:0/983496441 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c043070 con 0x7f1b68007460 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.711+0000 7f1b66ffd640 1 -- 172.21.15.55:0/983496441 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c038040 con 0x7f1b68007460 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.711+0000 7f1b66ffd640 1 -- 172.21.15.55:0/983496441 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c037050 con 0x7f1b68007460 2024-05-20T22:08:16.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.711+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b680532c0 con 0x7f1b68007460 2024-05-20T22:08:16.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.711+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b680522a0 con 0x7f1b68007460 2024-05-20T22:08:16.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.713+0000 7fba73fff640 1 -- 172.21.15.55:0/1956768266 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8808cd50 con 0x7fba80058700 2024-05-20T22:08:16.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.713+0000 7fba8dbe0640 1 -- 172.21.15.55:0/1956768266 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba800e5800 con 0x7fba80058700 2024-05-20T22:08:16.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.713+0000 7fba73fff640 1 -- 172.21.15.55:0/1956768266 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8806c680 con 0x7fba80058700 2024-05-20T22:08:16.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.714+0000 7fba73fff640 1 -- 172.21.15.55:0/1956768266 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8809a060 con 0x7fba80058700 2024-05-20T22:08:16.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.715+0000 7fba8dbe0640 1 -- 172.21.15.55:0/1956768266 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba800e5dc0 con 0x7fba80058700 2024-05-20T22:08:16.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.713+0000 7f3f057fa640 1 -- 172.21.15.55:0/759231019 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3ef4058c70 con 0x7f3f00033e90 2024-05-20T22:08:16.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.716+0000 7f7c217fa640 1 -- 172.21.15.55:0/3781776549 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7c240d4330 con 0x7f7c1c0e0350 2024-05-20T22:08:16.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.716+0000 7fba73fff640 1 -- 172.21.15.55:0/1956768266 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba8806c830 con 0x7fba80058700 2024-05-20T22:08:16.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.714+0000 7f1b66ffd640 1 -- 172.21.15.55:0/983496441 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b6c03cab0 con 0x7f1b68007460 2024-05-20T22:08:16.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.715+0000 7f1b66ffd640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1b5406be30 0x7f1b5406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.715+0000 7f1b66ffd640 1 -- 172.21.15.55:0/983496441 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1b6c0ab170 con 0x7f1b68007460 2024-05-20T22:08:16.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.715+0000 7f1b75b9a640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1b5406be30 0x7f1b5406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.716+0000 7fba73fff640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba6c06c260 0x7fba6c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:16.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.716+0000 7f1b75b9a640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1b5406be30 0x7f1b5406e340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1b680dfca0 tx=0x7f1b60009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.717+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b68056dc0 con 0x7f1b68007460 2024-05-20T22:08:16.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.717+0000 7fba877fe640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba6c06c260 0x7fba6c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:16.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.717+0000 7fba73fff640 1 -- 172.21.15.55:0/1956768266 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fba88074690 con 0x7fba80058700 2024-05-20T22:08:16.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.717+0000 7fba71ffb640 1 -- 172.21.15.55:0/1956768266 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba800059c0 con 0x7fba80058700 2024-05-20T22:08:16.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.718+0000 7fba877fe640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba6c06c260 0x7fba6c06e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009630 tx=0x7fba7c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:16.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.723+0000 7f1b66ffd640 1 -- 172.21.15.55:0/983496441 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1b680549f0 con 0x7f1b68007460 2024-05-20T22:08:16.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:16.723+0000 7fba73fff640 1 -- 172.21.15.55:0/1956768266 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fba880d3040 con 0x7fba80058700 2024-05-20T22:08:17.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.068+0000 7fc54b7f5640 1 -- 172.21.15.55:0/708652368 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc5440ae8a0 con 0x7fc54415e550 2024-05-20T22:08:17.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.069+0000 7fc539ffb640 1 -- 172.21.15.55:0/708652368 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc544154060 con 0x7fc54415e550 2024-05-20T22:08:17.070 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-20T22:08:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.072+0000 7f309302b640 1 -- 172.21.15.55:0/469648855 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3074005580 con 0x7f3084053de0 2024-05-20T22:08:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.072+0000 7f3082ffd640 1 -- 172.21.15.55:0/469648855 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f30840f7280 con 0x7f3084053de0 2024-05-20T22:08:17.073 INFO:teuthology.orchestra.run.smithi055.stdout:55834574859 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 -- 172.21.15.55:0/708652368 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc52807cec0 msgr2=0x7fc52807f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc52807cec0 0x7fc52807f3d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc534009b20 tx=0x7fc534008040 comp rx=0 tx=0).stop 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 -- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54415e550 msgr2=0x7fc544150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54415e550 0x7fc544150240 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c780 tx=0x7fc53c00cb80 comp rx=0 tx=0).stop 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 -- 172.21.15.55:0/708652368 shutdown_connections 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc52807cec0 0x7fc52807f3d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544153710 0x7fc544153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54415e550 0x7fc544150240 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 --2- 172.21.15.55:0/708652368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54406bfe0 0x7fc544149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 -- 172.21.15.55:0/708652368 >> 172.21.15.55:0/708652368 conn(0x7fc5440b3980 msgr2=0x7fc5440aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 -- 172.21.15.55:0/708652368 shutdown_connections 2024-05-20T22:08:17.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.073+0000 7fc5277fe640 1 -- 172.21.15.55:0/708652368 wait complete. 2024-05-20T22:08:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.074+0000 7f25791db640 1 -- 172.21.15.55:0/4054080149 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2554005580 con 0x7f256c04c7f0 2024-05-20T22:08:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.075+0000 7f2570ff9640 1 -- 172.21.15.55:0/4054080149 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f25740f87b0 con 0x7f256c04c7f0 2024-05-20T22:08:17.076 INFO:teuthology.orchestra.run.smithi055.stdout:55834574859 2024-05-20T22:08:17.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.076+0000 7f309302b640 1 -- 172.21.15.55:0/469648855 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f307006c230 msgr2=0x7f307006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.076+0000 7f309302b640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f307006c230 0x7f307006e740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f308c074b60 tx=0x7f308c068500 comp rx=0 tx=0).stop 2024-05-20T22:08:17.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.077+0000 7f309302b640 1 -- 172.21.15.55:0/469648855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3084053de0 msgr2=0x7f30840f7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.077+0000 7f309302b640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3084053de0 0x7f30840f7ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f30880098c0 tx=0x7f30880046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.077+0000 7f309302b640 1 -- 172.21.15.55:0/469648855 shutdown_connections 2024-05-20T22:08:17.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.077+0000 7f309302b640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f307006c230 0x7f307006e740 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.077+0000 7f309302b640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3084055810 0x7f30840f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.077+0000 7f309302b640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3084053de0 0x7f30840f7ab0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.077+0000 7f309302b640 1 --2- 172.21.15.55:0/469648855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3084033e90 0x7f30840f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.077+0000 7f309302b640 1 -- 172.21.15.55:0/469648855 >> 172.21.15.55:0/469648855 conn(0x7f3084054cf0 msgr2=0x7f30840deb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.078+0000 7f309302b640 1 -- 172.21.15.55:0/469648855 shutdown_connections 2024-05-20T22:08:17.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.078+0000 7f309302b640 1 -- 172.21.15.55:0/469648855 wait complete. 2024-05-20T22:08:17.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.078+0000 7f255e7fc640 1 -- 172.21.15.55:0/4054080149 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f255806bf30 msgr2=0x7f255806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.078+0000 7f255e7fc640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f255806bf30 0x7f255806e440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f256c00d6d0 tx=0x7f2560009240 comp rx=0 tx=0).stop 2024-05-20T22:08:17.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.078+0000 7f255e7fc640 1 -- 172.21.15.55:0/4054080149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c04c7f0 msgr2=0x7f256c049870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.078+0000 7f255e7fc640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c04c7f0 0x7f256c049870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2574051020 tx=0x7f25740750a0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.078+0000 7f255e7fc640 1 -- 172.21.15.55:0/4054080149 shutdown_connections 2024-05-20T22:08:17.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.079+0000 7f255e7fc640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f255806bf30 0x7f255806e440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.079+0000 7f255e7fc640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f256c04c7f0 0x7f256c049870 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.079+0000 7f255e7fc640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256c055810 0x7f256c17ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.079+0000 7f255e7fc640 1 --2- 172.21.15.55:0/4054080149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f256c053de0 0x7f256c0dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.079+0000 7f255e7fc640 1 -- 172.21.15.55:0/4054080149 >> 172.21.15.55:0/4054080149 conn(0x7f256c054cf0 msgr2=0x7f256c034750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.079+0000 7f255e7fc640 1 -- 172.21.15.55:0/4054080149 shutdown_connections 2024-05-20T22:08:17.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.079+0000 7f255e7fc640 1 -- 172.21.15.55:0/4054080149 wait complete. 2024-05-20T22:08:17.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.3 2024-05-20T22:08:17.095 DEBUG:teuthology.parallel:result is None 2024-05-20T22:08:17.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.5 2024-05-20T22:08:17.099 DEBUG:teuthology.parallel:result is None 2024-05-20T22:08:17.100 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.1 2024-05-20T22:08:17.100 DEBUG:teuthology.parallel:result is None 2024-05-20T22:08:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.112+0000 7f265d7fa640 1 -- 172.21.15.55:0/4268926060 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2650005580 con 0x7f26700526e0 2024-05-20T22:08:17.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.113+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7c1c0485c0 con 0x7f7c1c0e0350 2024-05-20T22:08:17.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.113+0000 7f265f7fe640 1 -- 172.21.15.55:0/4268926060 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f266c05d5a0 con 0x7f26700526e0 2024-05-20T22:08:17.114 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-20T22:08:17.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.113+0000 7f7c217fa640 1 -- 172.21.15.55:0/3781776549 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7c240d8630 con 0x7f7c1c0e0350 2024-05-20T22:08:17.115 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-20T22:08:17.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3ee8005850 con 0x7f3f00033e90 2024-05-20T22:08:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f265d7fa640 1 -- 172.21.15.55:0/4268926060 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266406bf00 msgr2=0x7f266406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f265d7fa640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266406bf00 0x7f266406e410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2668009ba0 tx=0x7f2668009240 comp rx=0 tx=0).stop 2024-05-20T22:08:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f265d7fa640 1 -- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 msgr2=0x7f267004a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f265d7fa640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 0x7f267004a9a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f266c00e8a0 tx=0x7f266c00ed60 comp rx=0 tx=0).stop 2024-05-20T22:08:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f3f057fa640 1 -- 172.21.15.55:0/759231019 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ef405cf30 con 0x7f3f00033e90 2024-05-20T22:08:17.117 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-20T22:08:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f265d7fa640 1 -- 172.21.15.55:0/4268926060 shutdown_connections 2024-05-20T22:08:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f265d7fa640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f266406bf00 0x7f266406e410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f265d7fa640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26700e7cf0 0x7f26700e0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f265d7fa640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26700526e0 0x7f267004a9a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f265d7fa640 1 --2- 172.21.15.55:0/4268926060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2670051a00 0x7f267004ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.115+0000 7f265d7fa640 1 -- 172.21.15.55:0/4268926060 >> 172.21.15.55:0/4268926060 conn(0x7f267000c3f0 msgr2=0x7f267000c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f265d7fa640 1 -- 172.21.15.55:0/4268926060 shutdown_connections 2024-05-20T22:08:17.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f265d7fa640 1 -- 172.21.15.55:0/4268926060 wait complete. 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c0806be80 msgr2=0x7f7c0806e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c0806be80 0x7f7c0806e390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c1802a200 tx=0x7f7c1802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c0e0350 msgr2=0x7f7c1c0e2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c0e0350 0x7f7c1c0e2000 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c050630 tx=0x7f7c24074560 comp rx=0 tx=0).stop 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 shutdown_connections 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c0806be80 0x7f7c0806e390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c0dd280 0x7f7c1c034db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c1c0e0350 0x7f7c1c0e2000 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 --2- 172.21.15.55:0/3781776549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1c033950 0x7f7c1c0e1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 >> 172.21.15.55:0/3781776549 conn(0x7f7c1c054cf0 msgr2=0x7f7c1c006bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 shutdown_connections 2024-05-20T22:08:17.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.116+0000 7f7c29d3b640 1 -- 172.21.15.55:0/3781776549 wait complete. 2024-05-20T22:08:17.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3eec06be80 msgr2=0x7f3eec06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3eec06be80 0x7f3eec06e390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f000dee20 tx=0x7f3efc0090e0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 msgr2=0x7f3f0004cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 0x7f3f0004cbd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400b440 tx=0x7f3ef400b900 comp rx=0 tx=0).stop 2024-05-20T22:08:17.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 shutdown_connections 2024-05-20T22:08:17.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3eec06be80 0x7f3eec06e390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f00055810 0x7f3f0004d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f00053de0 0x7f3f0004d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 --2- 172.21.15.55:0/759231019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f00033e90 0x7f3f0004cbd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 >> 172.21.15.55:0/759231019 conn(0x7f3f00054cf0 msgr2=0x7f3f000e1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 shutdown_connections 2024-05-20T22:08:17.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.118+0000 7f3f0ddff640 1 -- 172.21.15.55:0/759231019 wait complete. 2024-05-20T22:08:17.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.0 2024-05-20T22:08:17.132 DEBUG:teuthology.parallel:result is None 2024-05-20T22:08:17.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.6 2024-05-20T22:08:17.135 DEBUG:teuthology.parallel:result is None 2024-05-20T22:08:17.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.4 2024-05-20T22:08:17.136 DEBUG:teuthology.parallel:result is None 2024-05-20T22:08:17.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.162+0000 7fba71ffb640 1 -- 172.21.15.55:0/1956768266 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fba80053ae0 con 0x7fba80058700 2024-05-20T22:08:17.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.162+0000 7fba73fff640 1 -- 172.21.15.55:0/1956768266 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fba880d7340 con 0x7fba80058700 2024-05-20T22:08:17.163 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-20T22:08:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.164+0000 7fba71ffb640 1 -- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba6c06c260 msgr2=0x7fba6c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.164+0000 7fba71ffb640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba6c06c260 0x7fba6c06e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009630 tx=0x7fba7c02e000 comp rx=0 tx=0).stop 2024-05-20T22:08:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.164+0000 7fba71ffb640 1 -- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba80058700 msgr2=0x7fba800e6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.164+0000 7fba71ffb640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba80058700 0x7fba800e6e00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fba8806a560 tx=0x7fba88074560 comp rx=0 tx=0).stop 2024-05-20T22:08:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.164+0000 7fba71ffb640 1 -- 172.21.15.55:0/1956768266 shutdown_connections 2024-05-20T22:08:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.164+0000 7fba71ffb640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fba6c06c260 0x7fba6c06e770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.164+0000 7fba71ffb640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba80058700 0x7fba800e6e00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.164+0000 7fba71ffb640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba80056d30 0x7fba800ea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.164+0000 7fba71ffb640 1 --2- 172.21.15.55:0/1956768266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba8003c6c0 0x7fba800d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.164+0000 7fba71ffb640 1 -- 172.21.15.55:0/1956768266 >> 172.21.15.55:0/1956768266 conn(0x7fba8003ac50 msgr2=0x7fba800050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.165+0000 7fba71ffb640 1 -- 172.21.15.55:0/1956768266 shutdown_connections 2024-05-20T22:08:17.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.165+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1b68005490 con 0x7f1b68007460 2024-05-20T22:08:17.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.165+0000 7fba71ffb640 1 -- 172.21.15.55:0/1956768266 wait complete. 2024-05-20T22:08:17.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.165+0000 7f1b66ffd640 1 -- 172.21.15.55:0/983496441 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b6c079060 con 0x7f1b68007460 2024-05-20T22:08:17.168 INFO:teuthology.orchestra.run.smithi055.stdout:55834574859 2024-05-20T22:08:17.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1b5406be30 msgr2=0x7f1b5406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1b5406be30 0x7f1b5406e340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1b680dfca0 tx=0x7f1b60009060 comp rx=0 tx=0).stop 2024-05-20T22:08:17.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b68007460 msgr2=0x7f1b6804d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b68007460 0x7f1b6804d520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009990 tx=0x7f1b6c0046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 shutdown_connections 2024-05-20T22:08:17.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1b5406be30 0x7f1b5406e340 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b680e3890 0x7f1b6804dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b680545e0 0x7f1b68052d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 --2- 172.21.15.55:0/983496441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b68007460 0x7f1b6804d520 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 >> 172.21.15.55:0/983496441 conn(0x7f1b6803ac50 msgr2=0x7f1b6803a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 shutdown_connections 2024-05-20T22:08:17.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.167+0000 7f1b7739d640 1 -- 172.21.15.55:0/983496441 wait complete. 2024-05-20T22:08:17.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.2 2024-05-20T22:08:17.178 DEBUG:teuthology.parallel:result is None 2024-05-20T22:08:17.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.7 2024-05-20T22:08:17.180 DEBUG:teuthology.parallel:result is None 2024-05-20T22:08:17.180 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-20T22:08:17.180 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:08:17.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.333+0000 7fe588afe640 1 Processor -- start 2024-05-20T22:08:17.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.333+0000 7fe588afe640 1 -- start start 2024-05-20T22:08:17.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.335+0000 7fe588afe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe5840b8140 0x7fe5840d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.335+0000 7fe588afe640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5840b7c90 con 0x7fe5840d1190 2024-05-20T22:08:17.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.335+0000 7fe588afe640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5840d1740 con 0x7fe5840ce520 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.335+0000 7fe588afe640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe5840ce930 con 0x7fe5840b8140 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.335+0000 7fe581d9b640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe5840d1190 0x7fe5840b60b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46490/0 (socket says 172.21.15.55:46490) 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.335+0000 7fe581d9b640 1 -- 172.21.15.55:0/168979989 learned_addr learned my addr 172.21.15.55:0/168979989 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.335+0000 7fe58259c640 1 --2- 172.21.15.55:0/168979989 >> v2:172.21.15.138:3300/0 conn(0x7fe5840b8140 0x7fe5840d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.335+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1628534184 0 0) 0x7fe5840b7c90 con 0x7fe5840d1190 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.335+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe570003520 con 0x7fe5840d1190 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3423551557 0 0) 0x7fe570003520 con 0x7fe5840d1190 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 >> v2:172.21.15.138:3300/0 conn(0x7fe5840b8140 msgr2=0x7fe5840d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 --2- 172.21.15.55:0/168979989 >> v2:172.21.15.138:3300/0 conn(0x7fe5840b8140 0x7fe5840d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 >> v1:172.21.15.138:6789/0 conn(0x7fe5840ce520 legacy=0x7fe5840b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58409bc10 con 0x7fe5840d1190 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fe56c001670 con 0x7fe5840d1190 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 >> v1:172.21.15.55:6789/0 conn(0x7fe5840d1190 legacy=0x7fe5840b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5700039b0 0x7fe570003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe570004800 0x7fe570004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5700051c0 0x7fe57000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe5840ce930 con 0x7fe570004800 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe57000c160 con 0x7fe5700051c0 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe57000c360 con 0x7fe5700039b0 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe581d9b640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe570004800 0x7fe570004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe58259c640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5700039b0 0x7fe570003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe582d9d640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5700051c0 0x7fe57000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe581d9b640 1 -- 172.21.15.55:0/168979989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5700039b0 msgr2=0x7fe570003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe581d9b640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5700039b0 0x7fe570003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe581d9b640 1 -- 172.21.15.55:0/168979989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5700051c0 msgr2=0x7fe57000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:17.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe581d9b640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5700051c0 0x7fe57000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe581d9b640 1 -- 172.21.15.55:0/168979989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe570003520 con 0x7fe570004800 2024-05-20T22:08:17.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.336+0000 7fe582d9d640 1 -- 172.21.15.55:0/168979989 reap_dead start 2024-05-20T22:08:17.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.337+0000 7fe581d9b640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe570004800 0x7fe570004c10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe584074430 tx=0x7fe570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1024224b2875dc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.337+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c007d30 con 0x7fe570004800 2024-05-20T22:08:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.337+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56c005ad0 con 0x7fe570004800 2024-05-20T22:08:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.337+0000 7fe58159a640 1 -- 172.21.15.55:0/168979989 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c007070 con 0x7fe570004800 2024-05-20T22:08:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.338+0000 7fe588afe640 1 -- 172.21.15.55:0/168979989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe570004800 msgr2=0x7fe570004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.338+0000 7fe588afe640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe570004800 0x7fe570004c10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe584074430 tx=0x7fe570005830 comp rx=0 tx=0).stop 2024-05-20T22:08:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.338+0000 7fe588afe640 1 -- 172.21.15.55:0/168979989 shutdown_connections 2024-05-20T22:08:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.338+0000 7fe588afe640 1 --2- 172.21.15.55:0/168979989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe570004800 0x7fe570004c10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.338+0000 7fe588afe640 1 -- 172.21.15.55:0/168979989 >> 172.21.15.55:0/168979989 conn(0x7fe5840b4340 msgr2=0x7fe5840b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.338+0000 7fe588afe640 1 -- 172.21.15.55:0/168979989 shutdown_connections 2024-05-20T22:08:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.338+0000 7fe588afe640 1 -- 172.21.15.55:0/168979989 wait complete. 2024-05-20T22:08:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.340+0000 7fe588afe640 1 Processor -- start 2024-05-20T22:08:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.340+0000 7fe588afe640 1 -- start start 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.341+0000 7fe588afe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe570004800 0x7fe5840cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.341+0000 7fe588afe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5841d68d0 0x7fe5840c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.341+0000 7fe588afe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5841d6ce0 0x7fe58409dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.341+0000 7fe588afe640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe5840d06a0 con 0x7fe5841d68d0 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.341+0000 7fe588afe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5840d0520 con 0x7fe5841d6ce0 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.341+0000 7fe588afe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe5840b8140 con 0x7fe570004800 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.341+0000 7fe581d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5841d68d0 0x7fe5840c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe581d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5841d68d0 0x7fe5840c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58308/0 (socket says 172.21.15.55:58308) 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe581d9b640 1 -- 172.21.15.55:0/975781493 learned_addr learned my addr 172.21.15.55:0/975781493 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe58259c640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe570004800 0x7fe5840cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe582d9d640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5841d6ce0 0x7fe58409dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe581d9b640 1 -- 172.21.15.55:0/975781493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe570004800 msgr2=0x7fe5840cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe581d9b640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe570004800 0x7fe5840cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe581d9b640 1 -- 172.21.15.55:0/975781493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5841d6ce0 msgr2=0x7fe58409dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe581d9b640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5841d6ce0 0x7fe58409dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe581d9b640 1 -- 172.21.15.55:0/975781493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5840ca6d0 con 0x7fe5841d68d0 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe581d9b640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5841d68d0 0x7fe5840c9b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c2d0 tx=0x7fe56c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe57affd640 1 -- 172.21.15.55:0/975781493 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c037040 con 0x7fe5841d68d0 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe57affd640 1 -- 172.21.15.55:0/975781493 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56c001630 con 0x7fe5841d68d0 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe588afe640 1 -- 172.21.15.55:0/975781493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5840ca9b0 con 0x7fe5841d68d0 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe57affd640 1 -- 172.21.15.55:0/975781493 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c004a00 con 0x7fe5841d68d0 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.342+0000 7fe588afe640 1 -- 172.21.15.55:0/975781493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5840caf70 con 0x7fe5841d68d0 2024-05-20T22:08:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.343+0000 7fe588afe640 1 -- 172.21.15.55:0/975781493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe584069090 con 0x7fe5841d68d0 2024-05-20T22:08:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.344+0000 7fe57affd640 1 -- 172.21.15.55:0/975781493 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe56c004ba0 con 0x7fe5841d68d0 2024-05-20T22:08:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.344+0000 7fe57affd640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe56406be30 0x7fe56406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.345+0000 7fe58259c640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe56406be30 0x7fe56406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.345+0000 7fe57affd640 1 -- 172.21.15.55:0/975781493 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe56c0ab020 con 0x7fe5841d68d0 2024-05-20T22:08:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.345+0000 7fe58259c640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe56406be30 0x7fe56406e340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5840739f0 tx=0x7fe57400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:17.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.349+0000 7fe57affd640 1 -- 172.21.15.55:0/975781493 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe56c079770 con 0x7fe5841d68d0 2024-05-20T22:08:17.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.529+0000 7fe588afe640 1 -- 172.21.15.55:0/975781493 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5840b71f0 con 0x7fe56406be30 2024-05-20T22:08:17.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.537+0000 7fe57affd640 1 -- 172.21.15.55:0/975781493 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332538 (secure 0 0 0) 0x7fe5840b71f0 con 0x7fe56406be30 2024-05-20T22:08:17.545 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:08:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 -- 172.21.15.55:0/975781493 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe56406be30 msgr2=0x7fe56406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe56406be30 0x7fe56406e340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5840739f0 tx=0x7fe57400b040 comp rx=0 tx=0).stop 2024-05-20T22:08:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 -- 172.21.15.55:0/975781493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5841d68d0 msgr2=0x7fe5840c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5841d68d0 0x7fe5840c9b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c2d0 tx=0x7fe56c001a40 comp rx=0 tx=0).stop 2024-05-20T22:08:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 -- 172.21.15.55:0/975781493 shutdown_connections 2024-05-20T22:08:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe56406be30 0x7fe56406e340 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5841d6ce0 0x7fe58409dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5841d68d0 0x7fe5840c9b80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 --2- 172.21.15.55:0/975781493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe570004800 0x7fe5840cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 -- 172.21.15.55:0/975781493 >> 172.21.15.55:0/975781493 conn(0x7fe5840b4340 msgr2=0x7fe584070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 -- 172.21.15.55:0/975781493 shutdown_connections 2024-05-20T22:08:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.541+0000 7fe588afe640 1 -- 172.21.15.55:0/975781493 wait complete. 2024-05-20T22:08:17.554 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-05-20T22:08:16.332123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236020,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738700,"statfs":{"total":773094113280,"available":772852428800,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002582"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581341+0000","last_change":"2024-05-20T22:07:52.664054+0000","last_active":"2024-05-20T22:07:59.581341+0000","last_peered":"2024-05-20T22:07:59.581341+0000","last_clean":"2024-05-20T22:07:59.581341+0000","last_became_active":"2024-05-20T22:07:52.663784+0000","last_became_peered":"2024-05-20T22:07:52.663784+0000","last_unstale":"2024-05-20T22:07:59.581341+0000","last_undegraded":"2024-05-20T22:07:59.581341+0000","last_fullsized":"2024-05-20T22:07:59.581341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:37.756571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092411+0000","last_change":"2024-05-20T22:07:53.678918+0000","last_active":"2024-05-20T22:08:00.092411+0000","last_peered":"2024-05-20T22:08:00.092411+0000","last_clean":"2024-05-20T22:08:00.092411+0000","last_became_active":"2024-05-20T22:07:53.678467+0000","last_became_peered":"2024-05-20T22:07:53.678467+0000","last_unstale":"2024-05-20T22:08:00.092411+0000","last_undegraded":"2024-05-20T22:08:00.092411+0000","last_fullsized":"2024-05-20T22:08:00.092411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092012+0000","last_change":"2024-05-20T22:07:57.732174+0000","last_active":"2024-05-20T22:08:00.092012+0000","last_peered":"2024-05-20T22:08:00.092012+0000","last_clean":"2024-05-20T22:08:00.092012+0000","last_became_active":"2024-05-20T22:07:57.731124+0000","last_became_peered":"2024-05-20T22:07:57.731124+0000","last_unstale":"2024-05-20T22:08:00.092012+0000","last_undegraded":"2024-05-20T22:08:00.092012+0000","last_fullsized":"2024-05-20T22:08:00.092012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:11.233995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092785+0000","last_change":"2024-05-20T22:07:52.664457+0000","last_active":"2024-05-20T22:08:00.092785+0000","last_peered":"2024-05-20T22:08:00.092785+0000","last_clean":"2024-05-20T22:08:00.092785+0000","last_became_active":"2024-05-20T22:07:52.664121+0000","last_became_peered":"2024-05-20T22:07:52.664121+0000","last_unstale":"2024-05-20T22:08:00.092785+0000","last_undegraded":"2024-05-20T22:08:00.092785+0000","last_fullsized":"2024-05-20T22:08:00.092785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581131+0000","last_change":"2024-05-20T22:07:53.669570+0000","last_active":"2024-05-20T22:07:59.581131+0000","last_peered":"2024-05-20T22:07:59.581131+0000","last_clean":"2024-05-20T22:07:59.581131+0000","last_became_active":"2024-05-20T22:07:53.668913+0000","last_became_peered":"2024-05-20T22:07:53.668913+0000","last_unstale":"2024-05-20T22:07:59.581131+0000","last_undegraded":"2024-05-20T22:07:59.581131+0000","last_fullsized":"2024-05-20T22:07:59.581131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092313+0000","last_change":"2024-05-20T22:07:57.728254+0000","last_active":"2024-05-20T22:08:00.092313+0000","last_peered":"2024-05-20T22:08:00.092313+0000","last_clean":"2024-05-20T22:08:00.092313+0000","last_became_active":"2024-05-20T22:07:57.727787+0000","last_became_peered":"2024-05-20T22:07:57.727787+0000","last_unstale":"2024-05-20T22:08:00.092313+0000","last_undegraded":"2024-05-20T22:08:00.092313+0000","last_fullsized":"2024-05-20T22:08:00.092313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:43.588870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706646+0000","last_change":"2024-05-20T22:07:52.657626+0000","last_active":"2024-05-20T22:07:58.706646+0000","last_peered":"2024-05-20T22:07:58.706646+0000","last_clean":"2024-05-20T22:07:58.706646+0000","last_became_active":"2024-05-20T22:07:52.657298+0000","last_became_peered":"2024-05-20T22:07:52.657298+0000","last_unstale":"2024-05-20T22:07:58.706646+0000","last_undegraded":"2024-05-20T22:07:58.706646+0000","last_fullsized":"2024-05-20T22:07:58.706646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:41.518373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092705+0000","last_change":"2024-05-20T22:07:53.669706+0000","last_active":"2024-05-20T22:08:00.092705+0000","last_peered":"2024-05-20T22:08:00.092705+0000","last_clean":"2024-05-20T22:08:00.092705+0000","last_became_active":"2024-05-20T22:07:53.669332+0000","last_became_peered":"2024-05-20T22:07:53.669332+0000","last_unstale":"2024-05-20T22:08:00.092705+0000","last_undegraded":"2024-05-20T22:08:00.092705+0000","last_fullsized":"2024-05-20T22:08:00.092705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092885+0000","last_change":"2024-05-20T22:07:57.734186+0000","last_active":"2024-05-20T22:08:00.092885+0000","last_peered":"2024-05-20T22:08:00.092885+0000","last_clean":"2024-05-20T22:08:00.092885+0000","last_became_active":"2024-05-20T22:07:57.733618+0000","last_became_peered":"2024-05-20T22:07:57.733618+0000","last_unstale":"2024-05-20T22:08:00.092885+0000","last_undegraded":"2024-05-20T22:08:00.092885+0000","last_fullsized":"2024-05-20T22:08:00.092885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:46.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706028+0000","last_change":"2024-05-20T22:07:52.660033+0000","last_active":"2024-05-20T22:07:58.706028+0000","last_peered":"2024-05-20T22:07:58.706028+0000","last_clean":"2024-05-20T22:07:58.706028+0000","last_became_active":"2024-05-20T22:07:52.659733+0000","last_became_peered":"2024-05-20T22:07:52.659733+0000","last_unstale":"2024-05-20T22:07:58.706028+0000","last_undegraded":"2024-05-20T22:07:58.706028+0000","last_fullsized":"2024-05-20T22:07:58.706028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:41.160812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093319+0000","last_change":"2024-05-20T22:07:53.669432+0000","last_active":"2024-05-20T22:08:00.093319+0000","last_peered":"2024-05-20T22:08:00.093319+0000","last_clean":"2024-05-20T22:08:00.093319+0000","last_became_active":"2024-05-20T22:07:53.668235+0000","last_became_peered":"2024-05-20T22:07:53.668235+0000","last_unstale":"2024-05-20T22:08:00.093319+0000","last_undegraded":"2024-05-20T22:08:00.093319+0000","last_fullsized":"2024-05-20T22:08:00.093319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:42.044571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092545+0000","last_change":"2024-05-20T22:07:57.734065+0000","last_active":"2024-05-20T22:08:00.092545+0000","last_peered":"2024-05-20T22:08:00.092545+0000","last_clean":"2024-05-20T22:08:00.092545+0000","last_became_active":"2024-05-20T22:07:57.733537+0000","last_became_peered":"2024-05-20T22:07:57.733537+0000","last_unstale":"2024-05-20T22:08:00.092545+0000","last_undegraded":"2024-05-20T22:08:00.092545+0000","last_fullsized":"2024-05-20T22:08:00.092545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:16.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705724+0000","last_change":"2024-05-20T22:07:53.675279+0000","last_active":"2024-05-20T22:07:58.705724+0000","last_peered":"2024-05-20T22:07:58.705724+0000","last_clean":"2024-05-20T22:07:58.705724+0000","last_became_active":"2024-05-20T22:07:53.674924+0000","last_became_peered":"2024-05-20T22:07:53.674924+0000","last_unstale":"2024-05-20T22:07:58.705724+0000","last_undegraded":"2024-05-20T22:07:58.705724+0000","last_fullsized":"2024-05-20T22:07:58.705724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705726+0000","last_change":"2024-05-20T22:07:52.655127+0000","last_active":"2024-05-20T22:07:58.705726+0000","last_peered":"2024-05-20T22:07:58.705726+0000","last_clean":"2024-05-20T22:07:58.705726+0000","last_became_active":"2024-05-20T22:07:52.654807+0000","last_became_peered":"2024-05-20T22:07:52.654807+0000","last_unstale":"2024-05-20T22:07:58.705726+0000","last_undegraded":"2024-05-20T22:07:58.705726+0000","last_fullsized":"2024-05-20T22:07:58.705726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093607+0000","last_change":"2024-05-20T22:07:57.732155+0000","last_active":"2024-05-20T22:08:00.093607+0000","last_peered":"2024-05-20T22:08:00.093607+0000","last_clean":"2024-05-20T22:08:00.093607+0000","last_became_active":"2024-05-20T22:07:57.731370+0000","last_became_peered":"2024-05-20T22:07:57.731370+0000","last_unstale":"2024-05-20T22:08:00.093607+0000","last_undegraded":"2024-05-20T22:08:00.093607+0000","last_fullsized":"2024-05-20T22:08:00.093607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:30.698775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092498+0000","last_change":"2024-05-20T22:07:53.677185+0000","last_active":"2024-05-20T22:08:00.092498+0000","last_peered":"2024-05-20T22:08:00.092498+0000","last_clean":"2024-05-20T22:08:00.092498+0000","last_became_active":"2024-05-20T22:07:53.676661+0000","last_became_peered":"2024-05-20T22:07:53.676661+0000","last_unstale":"2024-05-20T22:08:00.092498+0000","last_undegraded":"2024-05-20T22:08:00.092498+0000","last_fullsized":"2024-05-20T22:08:00.092498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093251+0000","last_change":"2024-05-20T22:07:52.668583+0000","last_active":"2024-05-20T22:08:00.093251+0000","last_peered":"2024-05-20T22:08:00.093251+0000","last_clean":"2024-05-20T22:08:00.093251+0000","last_became_active":"2024-05-20T22:07:52.667850+0000","last_became_peered":"2024-05-20T22:07:52.667850+0000","last_unstale":"2024-05-20T22:08:00.093251+0000","last_undegraded":"2024-05-20T22:08:00.093251+0000","last_fullsized":"2024-05-20T22:08:00.093251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092843+0000","last_change":"2024-05-20T22:07:57.730407+0000","last_active":"2024-05-20T22:08:00.092843+0000","last_peered":"2024-05-20T22:08:00.092843+0000","last_clean":"2024-05-20T22:08:00.092843+0000","last_became_active":"2024-05-20T22:07:57.729918+0000","last_became_peered":"2024-05-20T22:07:57.729918+0000","last_unstale":"2024-05-20T22:08:00.092843+0000","last_undegraded":"2024-05-20T22:08:00.092843+0000","last_fullsized":"2024-05-20T22:08:00.092843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:44.596866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092522+0000","last_change":"2024-05-20T22:07:53.675389+0000","last_active":"2024-05-20T22:08:00.092522+0000","last_peered":"2024-05-20T22:08:00.092522+0000","last_clean":"2024-05-20T22:08:00.092522+0000","last_became_active":"2024-05-20T22:07:53.675027+0000","last_became_peered":"2024-05-20T22:07:53.675027+0000","last_unstale":"2024-05-20T22:08:00.092522+0000","last_undegraded":"2024-05-20T22:08:00.092522+0000","last_fullsized":"2024-05-20T22:08:00.092522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092234+0000","last_change":"2024-05-20T22:07:52.662236+0000","last_active":"2024-05-20T22:08:00.092234+0000","last_peered":"2024-05-20T22:08:00.092234+0000","last_clean":"2024-05-20T22:08:00.092234+0000","last_became_active":"2024-05-20T22:07:52.660352+0000","last_became_peered":"2024-05-20T22:07:52.660352+0000","last_unstale":"2024-05-20T22:08:00.092234+0000","last_undegraded":"2024-05-20T22:08:00.092234+0000","last_fullsized":"2024-05-20T22:08:00.092234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:41.518373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093269+0000","last_change":"2024-05-20T22:07:57.736022+0000","last_active":"2024-05-20T22:08:00.093269+0000","last_peered":"2024-05-20T22:08:00.093269+0000","last_clean":"2024-05-20T22:08:00.093269+0000","last_became_active":"2024-05-20T22:07:57.735570+0000","last_became_peered":"2024-05-20T22:07:57.735570+0000","last_unstale":"2024-05-20T22:08:00.093269+0000","last_undegraded":"2024-05-20T22:08:00.093269+0000","last_fullsized":"2024-05-20T22:08:00.093269+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:53.866913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092952+0000","last_change":"2024-05-20T22:07:53.662746+0000","last_active":"2024-05-20T22:08:00.092952+0000","last_peered":"2024-05-20T22:08:00.092952+0000","last_clean":"2024-05-20T22:08:00.092952+0000","last_became_active":"2024-05-20T22:07:53.662458+0000","last_became_peered":"2024-05-20T22:07:53.662458+0000","last_unstale":"2024-05-20T22:08:00.092952+0000","last_undegraded":"2024-05-20T22:08:00.092952+0000","last_fullsized":"2024-05-20T22:08:00.092952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:18.996690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705082+0000","last_change":"2024-05-20T22:07:52.656158+0000","last_active":"2024-05-20T22:07:58.705082+0000","last_peered":"2024-05-20T22:07:58.705082+0000","last_clean":"2024-05-20T22:07:58.705082+0000","last_became_active":"2024-05-20T22:07:52.655448+0000","last_became_peered":"2024-05-20T22:07:52.655448+0000","last_unstale":"2024-05-20T22:07:58.705082+0000","last_undegraded":"2024-05-20T22:07:58.705082+0000","last_fullsized":"2024-05-20T22:07:58.705082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093755+0000","last_change":"2024-05-20T22:07:57.734374+0000","last_active":"2024-05-20T22:08:00.093755+0000","last_peered":"2024-05-20T22:08:00.093755+0000","last_clean":"2024-05-20T22:08:00.093755+0000","last_became_active":"2024-05-20T22:07:57.733737+0000","last_became_peered":"2024-05-20T22:07:57.733737+0000","last_unstale":"2024-05-20T22:08:00.093755+0000","last_undegraded":"2024-05-20T22:08:00.093755+0000","last_fullsized":"2024-05-20T22:08:00.093755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581291+0000","last_change":"2024-05-20T22:07:53.670700+0000","last_active":"2024-05-20T22:07:59.581291+0000","last_peered":"2024-05-20T22:07:59.581291+0000","last_clean":"2024-05-20T22:07:59.581291+0000","last_became_active":"2024-05-20T22:07:53.670361+0000","last_became_peered":"2024-05-20T22:07:53.670361+0000","last_unstale":"2024-05-20T22:07:59.581291+0000","last_undegraded":"2024-05-20T22:07:59.581291+0000","last_fullsized":"2024-05-20T22:07:59.581291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706857+0000","last_change":"2024-05-20T22:07:52.658539+0000","last_active":"2024-05-20T22:07:58.706857+0000","last_peered":"2024-05-20T22:07:58.706857+0000","last_clean":"2024-05-20T22:07:58.706857+0000","last_became_active":"2024-05-20T22:07:52.658232+0000","last_became_peered":"2024-05-20T22:07:52.658232+0000","last_unstale":"2024-05-20T22:07:58.706857+0000","last_undegraded":"2024-05-20T22:07:58.706857+0000","last_fullsized":"2024-05-20T22:07:58.706857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092305+0000","last_change":"2024-05-20T22:07:57.726798+0000","last_active":"2024-05-20T22:08:00.092305+0000","last_peered":"2024-05-20T22:08:00.092305+0000","last_clean":"2024-05-20T22:08:00.092305+0000","last_became_active":"2024-05-20T22:07:57.726321+0000","last_became_peered":"2024-05-20T22:07:57.726321+0000","last_unstale":"2024-05-20T22:08:00.092305+0000","last_undegraded":"2024-05-20T22:08:00.092305+0000","last_fullsized":"2024-05-20T22:08:00.092305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:00:00.091936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093912+0000","last_change":"2024-05-20T22:07:53.673830+0000","last_active":"2024-05-20T22:08:00.093912+0000","last_peered":"2024-05-20T22:08:00.093912+0000","last_clean":"2024-05-20T22:08:00.093912+0000","last_became_active":"2024-05-20T22:07:53.673509+0000","last_became_peered":"2024-05-20T22:07:53.673509+0000","last_unstale":"2024-05-20T22:08:00.093912+0000","last_undegraded":"2024-05-20T22:08:00.093912+0000","last_fullsized":"2024-05-20T22:08:00.093912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092565+0000","last_change":"2024-05-20T22:07:52.661954+0000","last_active":"2024-05-20T22:08:00.092565+0000","last_peered":"2024-05-20T22:08:00.092565+0000","last_clean":"2024-05-20T22:08:00.092565+0000","last_became_active":"2024-05-20T22:07:52.661304+0000","last_became_peered":"2024-05-20T22:07:52.661304+0000","last_unstale":"2024-05-20T22:08:00.092565+0000","last_undegraded":"2024-05-20T22:08:00.092565+0000","last_fullsized":"2024-05-20T22:08:00.092565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.091968+0000","last_change":"2024-05-20T22:07:57.731395+0000","last_active":"2024-05-20T22:08:00.091968+0000","last_peered":"2024-05-20T22:08:00.091968+0000","last_clean":"2024-05-20T22:08:00.091968+0000","last_became_active":"2024-05-20T22:07:57.730839+0000","last_became_peered":"2024-05-20T22:07:57.730839+0000","last_unstale":"2024-05-20T22:08:00.091968+0000","last_undegraded":"2024-05-20T22:08:00.091968+0000","last_fullsized":"2024-05-20T22:08:00.091968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:44:34.494247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092791+0000","last_change":"2024-05-20T22:07:53.673439+0000","last_active":"2024-05-20T22:08:00.092791+0000","last_peered":"2024-05-20T22:08:00.092791+0000","last_clean":"2024-05-20T22:08:00.092791+0000","last_became_active":"2024-05-20T22:07:53.672746+0000","last_became_peered":"2024-05-20T22:07:53.672746+0000","last_unstale":"2024-05-20T22:08:00.092791+0000","last_undegraded":"2024-05-20T22:08:00.092791+0000","last_fullsized":"2024-05-20T22:08:00.092791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706775+0000","last_change":"2024-05-20T22:07:52.662817+0000","last_active":"2024-05-20T22:07:58.706775+0000","last_peered":"2024-05-20T22:07:58.706775+0000","last_clean":"2024-05-20T22:07:58.706775+0000","last_became_active":"2024-05-20T22:07:52.662463+0000","last_became_peered":"2024-05-20T22:07:52.662463+0000","last_unstale":"2024-05-20T22:07:58.706775+0000","last_undegraded":"2024-05-20T22:07:58.706775+0000","last_fullsized":"2024-05-20T22:07:58.706775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:07.396536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580916+0000","last_change":"2024-05-20T22:07:57.730858+0000","last_active":"2024-05-20T22:07:59.580916+0000","last_peered":"2024-05-20T22:07:59.580916+0000","last_clean":"2024-05-20T22:07:59.580916+0000","last_became_active":"2024-05-20T22:07:57.730370+0000","last_became_peered":"2024-05-20T22:07:57.730370+0000","last_unstale":"2024-05-20T22:07:59.580916+0000","last_undegraded":"2024-05-20T22:07:59.580916+0000","last_fullsized":"2024-05-20T22:07:59.580916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093484+0000","last_change":"2024-05-20T22:07:53.673073+0000","last_active":"2024-05-20T22:08:00.093484+0000","last_peered":"2024-05-20T22:08:00.093484+0000","last_clean":"2024-05-20T22:08:00.093484+0000","last_became_active":"2024-05-20T22:07:53.672744+0000","last_became_peered":"2024-05-20T22:07:53.672744+0000","last_unstale":"2024-05-20T22:08:00.093484+0000","last_undegraded":"2024-05-20T22:08:00.093484+0000","last_fullsized":"2024-05-20T22:08:00.093484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:49.823373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093340+0000","last_change":"2024-05-20T22:07:52.668045+0000","last_active":"2024-05-20T22:08:00.093340+0000","last_peered":"2024-05-20T22:08:00.093340+0000","last_clean":"2024-05-20T22:08:00.093340+0000","last_became_active":"2024-05-20T22:07:52.666855+0000","last_became_peered":"2024-05-20T22:07:52.666855+0000","last_unstale":"2024-05-20T22:08:00.093340+0000","last_undegraded":"2024-05-20T22:08:00.093340+0000","last_fullsized":"2024-05-20T22:08:00.093340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092837+0000","last_change":"2024-05-20T22:07:57.740710+0000","last_active":"2024-05-20T22:08:00.092837+0000","last_peered":"2024-05-20T22:08:00.092837+0000","last_clean":"2024-05-20T22:08:00.092837+0000","last_became_active":"2024-05-20T22:07:57.737102+0000","last_became_peered":"2024-05-20T22:07:57.737102+0000","last_unstale":"2024-05-20T22:08:00.092837+0000","last_undegraded":"2024-05-20T22:08:00.092837+0000","last_fullsized":"2024-05-20T22:08:00.092837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092905+0000","last_change":"2024-05-20T22:07:53.667120+0000","last_active":"2024-05-20T22:08:00.092905+0000","last_peered":"2024-05-20T22:08:00.092905+0000","last_clean":"2024-05-20T22:08:00.092905+0000","last_became_active":"2024-05-20T22:07:53.666809+0000","last_became_peered":"2024-05-20T22:07:53.666809+0000","last_unstale":"2024-05-20T22:08:00.092905+0000","last_undegraded":"2024-05-20T22:08:00.092905+0000","last_fullsized":"2024-05-20T22:08:00.092905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:08.402858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093723+0000","last_change":"2024-05-20T22:07:52.664415+0000","last_active":"2024-05-20T22:08:00.093723+0000","last_peered":"2024-05-20T22:08:00.093723+0000","last_clean":"2024-05-20T22:08:00.093723+0000","last_became_active":"2024-05-20T22:07:52.664085+0000","last_became_peered":"2024-05-20T22:07:52.664085+0000","last_unstale":"2024-05-20T22:08:00.093723+0000","last_undegraded":"2024-05-20T22:08:00.093723+0000","last_fullsized":"2024-05-20T22:08:00.093723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092731+0000","last_change":"2024-05-20T22:07:57.736468+0000","last_active":"2024-05-20T22:08:00.092731+0000","last_peered":"2024-05-20T22:08:00.092731+0000","last_clean":"2024-05-20T22:08:00.092731+0000","last_became_active":"2024-05-20T22:07:57.736003+0000","last_became_peered":"2024-05-20T22:07:57.736003+0000","last_unstale":"2024-05-20T22:08:00.092731+0000","last_undegraded":"2024-05-20T22:08:00.092731+0000","last_fullsized":"2024-05-20T22:08:00.092731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.091885+0000","last_change":"2024-05-20T22:07:53.674097+0000","last_active":"2024-05-20T22:08:00.091885+0000","last_peered":"2024-05-20T22:08:00.091885+0000","last_clean":"2024-05-20T22:08:00.091885+0000","last_became_active":"2024-05-20T22:07:53.673766+0000","last_became_peered":"2024-05-20T22:07:53.673766+0000","last_unstale":"2024-05-20T22:08:00.091885+0000","last_undegraded":"2024-05-20T22:08:00.091885+0000","last_fullsized":"2024-05-20T22:08:00.091885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706155+0000","last_change":"2024-05-20T22:07:52.662684+0000","last_active":"2024-05-20T22:07:58.706155+0000","last_peered":"2024-05-20T22:07:58.706155+0000","last_clean":"2024-05-20T22:07:58.706155+0000","last_became_active":"2024-05-20T22:07:52.662220+0000","last_became_peered":"2024-05-20T22:07:52.662220+0000","last_unstale":"2024-05-20T22:07:58.706155+0000","last_undegraded":"2024-05-20T22:07:58.706155+0000","last_fullsized":"2024-05-20T22:07:58.706155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093363+0000","last_change":"2024-05-20T22:07:57.740613+0000","last_active":"2024-05-20T22:08:00.093363+0000","last_peered":"2024-05-20T22:08:00.093363+0000","last_clean":"2024-05-20T22:08:00.093363+0000","last_became_active":"2024-05-20T22:07:57.740165+0000","last_became_peered":"2024-05-20T22:07:57.740165+0000","last_unstale":"2024-05-20T22:08:00.093363+0000","last_undegraded":"2024-05-20T22:08:00.093363+0000","last_fullsized":"2024-05-20T22:08:00.093363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:03:25.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580372+0000","last_change":"2024-05-20T22:07:53.656427+0000","last_active":"2024-05-20T22:07:59.580372+0000","last_peered":"2024-05-20T22:07:59.580372+0000","last_clean":"2024-05-20T22:07:59.580372+0000","last_became_active":"2024-05-20T22:07:53.656081+0000","last_became_peered":"2024-05-20T22:07:53.656081+0000","last_unstale":"2024-05-20T22:07:59.580372+0000","last_undegraded":"2024-05-20T22:07:59.580372+0000","last_fullsized":"2024-05-20T22:07:59.580372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:58:02.316451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705143+0000","last_change":"2024-05-20T22:07:52.656503+0000","last_active":"2024-05-20T22:07:58.705143+0000","last_peered":"2024-05-20T22:07:58.705143+0000","last_clean":"2024-05-20T22:07:58.705143+0000","last_became_active":"2024-05-20T22:07:52.654665+0000","last_became_peered":"2024-05-20T22:07:52.654665+0000","last_unstale":"2024-05-20T22:07:58.705143+0000","last_undegraded":"2024-05-20T22:07:58.705143+0000","last_fullsized":"2024-05-20T22:07:58.705143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581267+0000","last_change":"2024-05-20T22:07:57.731186+0000","last_active":"2024-05-20T22:07:59.581267+0000","last_peered":"2024-05-20T22:07:59.581267+0000","last_clean":"2024-05-20T22:07:59.581267+0000","last_became_active":"2024-05-20T22:07:57.730239+0000","last_became_peered":"2024-05-20T22:07:57.730239+0000","last_unstale":"2024-05-20T22:07:59.581267+0000","last_undegraded":"2024-05-20T22:07:59.581267+0000","last_fullsized":"2024-05-20T22:07:59.581267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:29.190558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093069+0000","last_change":"2024-05-20T22:07:53.667981+0000","last_active":"2024-05-20T22:08:00.093069+0000","last_peered":"2024-05-20T22:08:00.093069+0000","last_clean":"2024-05-20T22:08:00.093069+0000","last_became_active":"2024-05-20T22:07:53.667320+0000","last_became_peered":"2024-05-20T22:07:53.667320+0000","last_unstale":"2024-05-20T22:08:00.093069+0000","last_undegraded":"2024-05-20T22:08:00.093069+0000","last_fullsized":"2024-05-20T22:08:00.093069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093094+0000","last_change":"2024-05-20T22:07:52.657838+0000","last_active":"2024-05-20T22:08:00.093094+0000","last_peered":"2024-05-20T22:08:00.093094+0000","last_clean":"2024-05-20T22:08:00.093094+0000","last_became_active":"2024-05-20T22:07:52.657530+0000","last_became_peered":"2024-05-20T22:07:52.657530+0000","last_unstale":"2024-05-20T22:08:00.093094+0000","last_undegraded":"2024-05-20T22:08:00.093094+0000","last_fullsized":"2024-05-20T22:08:00.093094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706234+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:07:58.706234+0000","last_peered":"2024-05-20T22:07:58.706234+0000","last_clean":"2024-05-20T22:07:58.706234+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:07:58.706234+0000","last_undegraded":"2024-05-20T22:07:58.706234+0000","last_fullsized":"2024-05-20T22:07:58.706234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706485+0000","last_change":"2024-05-20T22:07:57.738452+0000","last_active":"2024-05-20T22:07:58.706485+0000","last_peered":"2024-05-20T22:07:58.706485+0000","last_clean":"2024-05-20T22:07:58.706485+0000","last_became_active":"2024-05-20T22:07:57.737967+0000","last_became_peered":"2024-05-20T22:07:57.737967+0000","last_unstale":"2024-05-20T22:07:58.706485+0000","last_undegraded":"2024-05-20T22:07:58.706485+0000","last_fullsized":"2024-05-20T22:07:58.706485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706425+0000","last_change":"2024-05-20T22:07:53.677833+0000","last_active":"2024-05-20T22:07:58.706425+0000","last_peered":"2024-05-20T22:07:58.706425+0000","last_clean":"2024-05-20T22:07:58.706425+0000","last_became_active":"2024-05-20T22:07:53.677513+0000","last_became_peered":"2024-05-20T22:07:53.677513+0000","last_unstale":"2024-05-20T22:07:58.706425+0000","last_undegraded":"2024-05-20T22:07:58.706425+0000","last_fullsized":"2024-05-20T22:07:58.706425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706468+0000","last_change":"2024-05-20T22:07:52.659290+0000","last_active":"2024-05-20T22:07:58.706468+0000","last_peered":"2024-05-20T22:07:58.706468+0000","last_clean":"2024-05-20T22:07:58.706468+0000","last_became_active":"2024-05-20T22:07:52.658939+0000","last_became_peered":"2024-05-20T22:07:52.658939+0000","last_unstale":"2024-05-20T22:07:58.706468+0000","last_undegraded":"2024-05-20T22:07:58.706468+0000","last_fullsized":"2024-05-20T22:07:58.706468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:30.832431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705627+0000","last_change":"2024-05-20T22:07:57.732095+0000","last_active":"2024-05-20T22:07:58.705627+0000","last_peered":"2024-05-20T22:07:58.705627+0000","last_clean":"2024-05-20T22:07:58.705627+0000","last_became_active":"2024-05-20T22:07:57.731276+0000","last_became_peered":"2024-05-20T22:07:57.731276+0000","last_unstale":"2024-05-20T22:07:58.705627+0000","last_undegraded":"2024-05-20T22:07:58.705627+0000","last_fullsized":"2024-05-20T22:07:58.705627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:30.698775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092825+0000","last_change":"2024-05-20T22:07:53.669593+0000","last_active":"2024-05-20T22:08:00.092825+0000","last_peered":"2024-05-20T22:08:00.092825+0000","last_clean":"2024-05-20T22:08:00.092825+0000","last_became_active":"2024-05-20T22:07:53.668982+0000","last_became_peered":"2024-05-20T22:07:53.668982+0000","last_unstale":"2024-05-20T22:08:00.092825+0000","last_undegraded":"2024-05-20T22:08:00.092825+0000","last_fullsized":"2024-05-20T22:08:00.092825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:26.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092921+0000","last_change":"2024-05-20T22:07:52.666183+0000","last_active":"2024-05-20T22:08:00.092921+0000","last_peered":"2024-05-20T22:08:00.092921+0000","last_clean":"2024-05-20T22:08:00.092921+0000","last_became_active":"2024-05-20T22:07:52.662670+0000","last_became_peered":"2024-05-20T22:07:52.662670+0000","last_unstale":"2024-05-20T22:08:00.092921+0000","last_undegraded":"2024-05-20T22:08:00.092921+0000","last_fullsized":"2024-05-20T22:08:00.092921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581046+0000","last_change":"2024-05-20T22:07:57.734443+0000","last_active":"2024-05-20T22:07:59.581046+0000","last_peered":"2024-05-20T22:07:59.581046+0000","last_clean":"2024-05-20T22:07:59.581046+0000","last_became_active":"2024-05-20T22:07:57.734021+0000","last_became_peered":"2024-05-20T22:07:57.734021+0000","last_unstale":"2024-05-20T22:07:59.581046+0000","last_undegraded":"2024-05-20T22:07:59.581046+0000","last_fullsized":"2024-05-20T22:07:59.581046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:30.960467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707194+0000","last_change":"2024-05-20T22:07:53.666815+0000","last_active":"2024-05-20T22:07:58.707194+0000","last_peered":"2024-05-20T22:07:58.707194+0000","last_clean":"2024-05-20T22:07:58.707194+0000","last_became_active":"2024-05-20T22:07:53.666480+0000","last_became_peered":"2024-05-20T22:07:53.666480+0000","last_unstale":"2024-05-20T22:07:58.707194+0000","last_undegraded":"2024-05-20T22:07:58.707194+0000","last_fullsized":"2024-05-20T22:07:58.707194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:49.823373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706498+0000","last_change":"2024-05-20T22:07:52.661745+0000","last_active":"2024-05-20T22:07:58.706498+0000","last_peered":"2024-05-20T22:07:58.706498+0000","last_clean":"2024-05-20T22:07:58.706498+0000","last_became_active":"2024-05-20T22:07:52.661304+0000","last_became_peered":"2024-05-20T22:07:52.661304+0000","last_unstale":"2024-05-20T22:07:58.706498+0000","last_undegraded":"2024-05-20T22:07:58.706498+0000","last_fullsized":"2024-05-20T22:07:58.706498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092249+0000","last_change":"2024-05-20T22:07:57.722732+0000","last_active":"2024-05-20T22:08:00.092249+0000","last_peered":"2024-05-20T22:08:00.092249+0000","last_clean":"2024-05-20T22:08:00.092249+0000","last_became_active":"2024-05-20T22:07:57.722287+0000","last_became_peered":"2024-05-20T22:07:57.722287+0000","last_unstale":"2024-05-20T22:08:00.092249+0000","last_undegraded":"2024-05-20T22:08:00.092249+0000","last_fullsized":"2024-05-20T22:08:00.092249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:48.169892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706818+0000","last_change":"2024-05-20T22:07:53.668146+0000","last_active":"2024-05-20T22:07:58.706818+0000","last_peered":"2024-05-20T22:07:58.706818+0000","last_clean":"2024-05-20T22:07:58.706818+0000","last_became_active":"2024-05-20T22:07:53.666969+0000","last_became_peered":"2024-05-20T22:07:53.666969+0000","last_unstale":"2024-05-20T22:07:58.706818+0000","last_undegraded":"2024-05-20T22:07:58.706818+0000","last_fullsized":"2024-05-20T22:07:58.706818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093917+0000","last_change":"2024-05-20T22:07:52.651502+0000","last_active":"2024-05-20T22:08:00.093917+0000","last_peered":"2024-05-20T22:08:00.093917+0000","last_clean":"2024-05-20T22:08:00.093917+0000","last_became_active":"2024-05-20T22:07:52.651166+0000","last_became_peered":"2024-05-20T22:07:52.651166+0000","last_unstale":"2024-05-20T22:08:00.093917+0000","last_undegraded":"2024-05-20T22:08:00.093917+0000","last_fullsized":"2024-05-20T22:08:00.093917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:41:50.914741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092214+0000","last_change":"2024-05-20T22:07:57.736957+0000","last_active":"2024-05-20T22:08:00.092214+0000","last_peered":"2024-05-20T22:08:00.092214+0000","last_clean":"2024-05-20T22:08:00.092214+0000","last_became_active":"2024-05-20T22:07:57.736512+0000","last_became_peered":"2024-05-20T22:07:57.736512+0000","last_unstale":"2024-05-20T22:08:00.092214+0000","last_undegraded":"2024-05-20T22:08:00.092214+0000","last_fullsized":"2024-05-20T22:08:00.092214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:46.233608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705356+0000","last_change":"2024-05-20T22:07:53.655287+0000","last_active":"2024-05-20T22:07:58.705356+0000","last_peered":"2024-05-20T22:07:58.705356+0000","last_clean":"2024-05-20T22:07:58.705356+0000","last_became_active":"2024-05-20T22:07:53.653689+0000","last_became_peered":"2024-05-20T22:07:53.653689+0000","last_unstale":"2024-05-20T22:07:58.705356+0000","last_undegraded":"2024-05-20T22:07:58.705356+0000","last_fullsized":"2024-05-20T22:07:58.705356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:26.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705945+0000","last_change":"2024-05-20T22:07:52.653862+0000","last_active":"2024-05-20T22:07:58.705945+0000","last_peered":"2024-05-20T22:07:58.705945+0000","last_clean":"2024-05-20T22:07:58.705945+0000","last_became_active":"2024-05-20T22:07:52.653527+0000","last_became_peered":"2024-05-20T22:07:52.653527+0000","last_unstale":"2024-05-20T22:07:58.705945+0000","last_undegraded":"2024-05-20T22:07:58.705945+0000","last_fullsized":"2024-05-20T22:07:58.705945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581093+0000","last_change":"2024-05-20T22:07:57.737164+0000","last_active":"2024-05-20T22:07:59.581093+0000","last_peered":"2024-05-20T22:07:59.581093+0000","last_clean":"2024-05-20T22:07:59.581093+0000","last_became_active":"2024-05-20T22:07:57.736733+0000","last_became_peered":"2024-05-20T22:07:57.736733+0000","last_unstale":"2024-05-20T22:07:59.581093+0000","last_undegraded":"2024-05-20T22:07:59.581093+0000","last_fullsized":"2024-05-20T22:07:59.581093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:53.866913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705408+0000","last_change":"2024-05-20T22:07:53.680648+0000","last_active":"2024-05-20T22:07:58.705408+0000","last_peered":"2024-05-20T22:07:58.705408+0000","last_clean":"2024-05-20T22:07:58.705408+0000","last_became_active":"2024-05-20T22:07:53.680316+0000","last_became_peered":"2024-05-20T22:07:53.680316+0000","last_unstale":"2024-05-20T22:07:58.705408+0000","last_undegraded":"2024-05-20T22:07:58.705408+0000","last_fullsized":"2024-05-20T22:07:58.705408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706034+0000","last_change":"2024-05-20T22:07:52.660545+0000","last_active":"2024-05-20T22:07:58.706034+0000","last_peered":"2024-05-20T22:07:58.706034+0000","last_clean":"2024-05-20T22:07:58.706034+0000","last_became_active":"2024-05-20T22:07:52.660191+0000","last_became_peered":"2024-05-20T22:07:52.660191+0000","last_unstale":"2024-05-20T22:07:58.706034+0000","last_undegraded":"2024-05-20T22:07:58.706034+0000","last_fullsized":"2024-05-20T22:07:58.706034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093896+0000","last_change":"2024-05-20T22:07:57.730561+0000","last_active":"2024-05-20T22:08:00.093896+0000","last_peered":"2024-05-20T22:08:00.093896+0000","last_clean":"2024-05-20T22:08:00.093896+0000","last_became_active":"2024-05-20T22:07:57.728162+0000","last_became_peered":"2024-05-20T22:07:57.728162+0000","last_unstale":"2024-05-20T22:08:00.093896+0000","last_undegraded":"2024-05-20T22:08:00.093896+0000","last_fullsized":"2024-05-20T22:08:00.093896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:18.032764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705624+0000","last_change":"2024-05-20T22:07:53.654238+0000","last_active":"2024-05-20T22:07:58.705624+0000","last_peered":"2024-05-20T22:07:58.705624+0000","last_clean":"2024-05-20T22:07:58.705624+0000","last_became_active":"2024-05-20T22:07:53.653629+0000","last_became_peered":"2024-05-20T22:07:53.653629+0000","last_unstale":"2024-05-20T22:07:58.705624+0000","last_undegraded":"2024-05-20T22:07:58.705624+0000","last_fullsized":"2024-05-20T22:07:58.705624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093667+0000","last_change":"2024-05-20T22:07:52.667566+0000","last_active":"2024-05-20T22:08:00.093667+0000","last_peered":"2024-05-20T22:08:00.093667+0000","last_clean":"2024-05-20T22:08:00.093667+0000","last_became_active":"2024-05-20T22:07:52.667214+0000","last_became_peered":"2024-05-20T22:07:52.667214+0000","last_unstale":"2024-05-20T22:08:00.093667+0000","last_undegraded":"2024-05-20T22:08:00.093667+0000","last_fullsized":"2024-05-20T22:08:00.093667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093690+0000","last_change":"2024-05-20T22:07:57.728380+0000","last_active":"2024-05-20T22:08:00.093690+0000","last_peered":"2024-05-20T22:08:00.093690+0000","last_clean":"2024-05-20T22:08:00.093690+0000","last_became_active":"2024-05-20T22:07:57.726788+0000","last_became_peered":"2024-05-20T22:07:57.726788+0000","last_unstale":"2024-05-20T22:08:00.093690+0000","last_undegraded":"2024-05-20T22:08:00.093690+0000","last_fullsized":"2024-05-20T22:08:00.093690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:46:04.066141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580651+0000","last_change":"2024-05-20T22:07:53.659126+0000","last_active":"2024-05-20T22:07:59.580651+0000","last_peered":"2024-05-20T22:07:59.580651+0000","last_clean":"2024-05-20T22:07:59.580651+0000","last_became_active":"2024-05-20T22:07:53.658751+0000","last_became_peered":"2024-05-20T22:07:53.658751+0000","last_unstale":"2024-05-20T22:07:59.580651+0000","last_undegraded":"2024-05-20T22:07:59.580651+0000","last_fullsized":"2024-05-20T22:07:59.580651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581140+0000","last_change":"2024-05-20T22:07:52.659607+0000","last_active":"2024-05-20T22:07:59.581140+0000","last_peered":"2024-05-20T22:07:59.581140+0000","last_clean":"2024-05-20T22:07:59.581140+0000","last_became_active":"2024-05-20T22:07:52.658939+0000","last_became_peered":"2024-05-20T22:07:52.658939+0000","last_unstale":"2024-05-20T22:07:59.581140+0000","last_undegraded":"2024-05-20T22:07:59.581140+0000","last_fullsized":"2024-05-20T22:07:59.581140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:12:51.889148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093500+0000","last_change":"2024-05-20T22:07:57.726495+0000","last_active":"2024-05-20T22:08:00.093500+0000","last_peered":"2024-05-20T22:08:00.093500+0000","last_clean":"2024-05-20T22:08:00.093500+0000","last_became_active":"2024-05-20T22:07:57.726012+0000","last_became_peered":"2024-05-20T22:07:57.726012+0000","last_unstale":"2024-05-20T22:08:00.093500+0000","last_undegraded":"2024-05-20T22:08:00.093500+0000","last_fullsized":"2024-05-20T22:08:00.093500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:44:34.494247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705411+0000","last_change":"2024-05-20T22:07:53.666422+0000","last_active":"2024-05-20T22:07:58.705411+0000","last_peered":"2024-05-20T22:07:58.705411+0000","last_clean":"2024-05-20T22:07:58.705411+0000","last_became_active":"2024-05-20T22:07:53.666109+0000","last_became_peered":"2024-05-20T22:07:53.666109+0000","last_unstale":"2024-05-20T22:07:58.705411+0000","last_undegraded":"2024-05-20T22:07:58.705411+0000","last_fullsized":"2024-05-20T22:07:58.705411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:03.694258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:08.748697+0000","last_change":"2024-05-20T22:07:52.661686+0000","last_active":"2024-05-20T22:08:08.748697+0000","last_peered":"2024-05-20T22:08:08.748697+0000","last_clean":"2024-05-20T22:08:08.748697+0000","last_became_active":"2024-05-20T22:07:52.661192+0000","last_became_peered":"2024-05-20T22:07:52.661192+0000","last_unstale":"2024-05-20T22:08:08.748697+0000","last_undegraded":"2024-05-20T22:08:08.748697+0000","last_fullsized":"2024-05-20T22:08:08.748697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093188+0000","last_change":"2024-05-20T22:07:57.733250+0000","last_active":"2024-05-20T22:08:00.093188+0000","last_peered":"2024-05-20T22:08:00.093188+0000","last_clean":"2024-05-20T22:08:00.093188+0000","last_became_active":"2024-05-20T22:07:57.732096+0000","last_became_peered":"2024-05-20T22:07:57.732096+0000","last_unstale":"2024-05-20T22:08:00.093188+0000","last_undegraded":"2024-05-20T22:08:00.093188+0000","last_fullsized":"2024-05-20T22:08:00.093188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:30.960467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093127+0000","last_change":"2024-05-20T22:07:53.664378+0000","last_active":"2024-05-20T22:08:00.093127+0000","last_peered":"2024-05-20T22:08:00.093127+0000","last_clean":"2024-05-20T22:08:00.093127+0000","last_became_active":"2024-05-20T22:07:53.664061+0000","last_became_peered":"2024-05-20T22:07:53.664061+0000","last_unstale":"2024-05-20T22:08:00.093127+0000","last_undegraded":"2024-05-20T22:08:00.093127+0000","last_fullsized":"2024-05-20T22:08:00.093127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:42.167135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092791+0000","last_change":"2024-05-20T22:07:52.665754+0000","last_active":"2024-05-20T22:08:00.092791+0000","last_peered":"2024-05-20T22:08:00.092791+0000","last_clean":"2024-05-20T22:08:00.092791+0000","last_became_active":"2024-05-20T22:07:52.663248+0000","last_became_peered":"2024-05-20T22:07:52.663248+0000","last_unstale":"2024-05-20T22:08:00.092791+0000","last_undegraded":"2024-05-20T22:08:00.092791+0000","last_fullsized":"2024-05-20T22:08:00.092791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092574+0000","last_change":"2024-05-20T22:07:57.732444+0000","last_active":"2024-05-20T22:08:00.092574+0000","last_peered":"2024-05-20T22:08:00.092574+0000","last_clean":"2024-05-20T22:08:00.092574+0000","last_became_active":"2024-05-20T22:07:57.731063+0000","last_became_peered":"2024-05-20T22:07:57.731063+0000","last_unstale":"2024-05-20T22:08:00.092574+0000","last_undegraded":"2024-05-20T22:08:00.092574+0000","last_fullsized":"2024-05-20T22:08:00.092574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:17.240623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581288+0000","last_change":"2024-05-20T22:07:53.669268+0000","last_active":"2024-05-20T22:07:59.581288+0000","last_peered":"2024-05-20T22:07:59.581288+0000","last_clean":"2024-05-20T22:07:59.581288+0000","last_became_active":"2024-05-20T22:07:53.666127+0000","last_became_peered":"2024-05-20T22:07:53.666127+0000","last_unstale":"2024-05-20T22:07:59.581288+0000","last_undegraded":"2024-05-20T22:07:59.581288+0000","last_fullsized":"2024-05-20T22:07:59.581288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093187+0000","last_change":"2024-05-20T22:07:52.660573+0000","last_active":"2024-05-20T22:08:00.093187+0000","last_peered":"2024-05-20T22:08:00.093187+0000","last_clean":"2024-05-20T22:08:00.093187+0000","last_became_active":"2024-05-20T22:07:52.660227+0000","last_became_peered":"2024-05-20T22:07:52.660227+0000","last_unstale":"2024-05-20T22:08:00.093187+0000","last_undegraded":"2024-05-20T22:08:00.093187+0000","last_fullsized":"2024-05-20T22:08:00.093187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092346+0000","last_change":"2024-05-20T22:07:57.726865+0000","last_active":"2024-05-20T22:08:00.092346+0000","last_peered":"2024-05-20T22:08:00.092346+0000","last_clean":"2024-05-20T22:08:00.092346+0000","last_became_active":"2024-05-20T22:07:57.726411+0000","last_became_peered":"2024-05-20T22:07:57.726411+0000","last_unstale":"2024-05-20T22:08:00.092346+0000","last_undegraded":"2024-05-20T22:08:00.092346+0000","last_fullsized":"2024-05-20T22:08:00.092346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:14.532804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093027+0000","last_change":"2024-05-20T22:07:53.674069+0000","last_active":"2024-05-20T22:08:00.093027+0000","last_peered":"2024-05-20T22:08:00.093027+0000","last_clean":"2024-05-20T22:08:00.093027+0000","last_became_active":"2024-05-20T22:07:53.673668+0000","last_became_peered":"2024-05-20T22:07:53.673668+0000","last_unstale":"2024-05-20T22:08:00.093027+0000","last_undegraded":"2024-05-20T22:08:00.093027+0000","last_fullsized":"2024-05-20T22:08:00.093027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:03.694258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706745+0000","last_change":"2024-05-20T22:07:52.659232+0000","last_active":"2024-05-20T22:07:58.706745+0000","last_peered":"2024-05-20T22:07:58.706745+0000","last_clean":"2024-05-20T22:07:58.706745+0000","last_became_active":"2024-05-20T22:07:52.657718+0000","last_became_peered":"2024-05-20T22:07:52.657718+0000","last_unstale":"2024-05-20T22:07:58.706745+0000","last_undegraded":"2024-05-20T22:07:58.706745+0000","last_fullsized":"2024-05-20T22:07:58.706745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092928+0000","last_change":"2024-05-20T22:07:57.730328+0000","last_active":"2024-05-20T22:08:00.092928+0000","last_peered":"2024-05-20T22:08:00.092928+0000","last_clean":"2024-05-20T22:08:00.092928+0000","last_became_active":"2024-05-20T22:07:57.729093+0000","last_became_peered":"2024-05-20T22:07:57.729093+0000","last_unstale":"2024-05-20T22:08:00.092928+0000","last_undegraded":"2024-05-20T22:08:00.092928+0000","last_fullsized":"2024-05-20T22:08:00.092928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:42:24.385164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581132+0000","last_change":"2024-05-20T22:07:57.738586+0000","last_active":"2024-05-20T22:07:59.581132+0000","last_peered":"2024-05-20T22:07:59.581132+0000","last_clean":"2024-05-20T22:07:59.581132+0000","last_became_active":"2024-05-20T22:07:57.736969+0000","last_became_peered":"2024-05-20T22:07:57.736969+0000","last_unstale":"2024-05-20T22:07:59.581132+0000","last_undegraded":"2024-05-20T22:07:59.581132+0000","last_fullsized":"2024-05-20T22:07:59.581132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:03:25.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092006+0000","last_change":"2024-05-20T22:07:53.677156+0000","last_active":"2024-05-20T22:08:00.092006+0000","last_peered":"2024-05-20T22:08:00.092006+0000","last_clean":"2024-05-20T22:08:00.092006+0000","last_became_active":"2024-05-20T22:07:53.676805+0000","last_became_peered":"2024-05-20T22:07:53.676805+0000","last_unstale":"2024-05-20T22:08:00.092006+0000","last_undegraded":"2024-05-20T22:08:00.092006+0000","last_fullsized":"2024-05-20T22:08:00.092006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092929+0000","last_change":"2024-05-20T22:07:52.659254+0000","last_active":"2024-05-20T22:08:00.092929+0000","last_peered":"2024-05-20T22:08:00.092929+0000","last_clean":"2024-05-20T22:08:00.092929+0000","last_became_active":"2024-05-20T22:07:52.658938+0000","last_became_peered":"2024-05-20T22:07:52.658938+0000","last_unstale":"2024-05-20T22:08:00.092929+0000","last_undegraded":"2024-05-20T22:08:00.092929+0000","last_fullsized":"2024-05-20T22:08:00.092929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706615+0000","last_change":"2024-05-20T22:07:57.731289+0000","last_active":"2024-05-20T22:07:58.706615+0000","last_peered":"2024-05-20T22:07:58.706615+0000","last_clean":"2024-05-20T22:07:58.706615+0000","last_became_active":"2024-05-20T22:07:57.730844+0000","last_became_peered":"2024-05-20T22:07:57.730844+0000","last_unstale":"2024-05-20T22:07:58.706615+0000","last_undegraded":"2024-05-20T22:07:58.706615+0000","last_fullsized":"2024-05-20T22:07:58.706615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092411+0000","last_change":"2024-05-20T22:07:53.676088+0000","last_active":"2024-05-20T22:08:00.092411+0000","last_peered":"2024-05-20T22:08:00.092411+0000","last_clean":"2024-05-20T22:08:00.092411+0000","last_became_active":"2024-05-20T22:07:53.675756+0000","last_became_peered":"2024-05-20T22:07:53.675756+0000","last_unstale":"2024-05-20T22:08:00.092411+0000","last_undegraded":"2024-05-20T22:08:00.092411+0000","last_fullsized":"2024-05-20T22:08:00.092411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093073+0000","last_change":"2024-05-20T22:07:52.660691+0000","last_active":"2024-05-20T22:08:00.093073+0000","last_peered":"2024-05-20T22:08:00.093073+0000","last_clean":"2024-05-20T22:08:00.093073+0000","last_became_active":"2024-05-20T22:07:52.660324+0000","last_became_peered":"2024-05-20T22:07:52.660324+0000","last_unstale":"2024-05-20T22:08:00.093073+0000","last_undegraded":"2024-05-20T22:08:00.093073+0000","last_fullsized":"2024-05-20T22:08:00.093073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:37.756571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706140+0000","last_change":"2024-05-20T22:07:52.660131+0000","last_active":"2024-05-20T22:07:58.706140+0000","last_peered":"2024-05-20T22:07:58.706140+0000","last_clean":"2024-05-20T22:07:58.706140+0000","last_became_active":"2024-05-20T22:07:52.659784+0000","last_became_peered":"2024-05-20T22:07:52.659784+0000","last_unstale":"2024-05-20T22:07:58.706140+0000","last_undegraded":"2024-05-20T22:07:58.706140+0000","last_fullsized":"2024-05-20T22:07:58.706140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581103+0000","last_change":"2024-05-20T22:07:53.659806+0000","last_active":"2024-05-20T22:07:59.581103+0000","last_peered":"2024-05-20T22:07:59.581103+0000","last_clean":"2024-05-20T22:07:59.581103+0000","last_became_active":"2024-05-20T22:07:53.659275+0000","last_became_peered":"2024-05-20T22:07:53.659275+0000","last_unstale":"2024-05-20T22:07:59.581103+0000","last_undegraded":"2024-05-20T22:07:59.581103+0000","last_fullsized":"2024-05-20T22:07:59.581103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706349+0000","last_change":"2024-05-20T22:07:57.732871+0000","last_active":"2024-05-20T22:07:58.706349+0000","last_peered":"2024-05-20T22:07:58.706349+0000","last_clean":"2024-05-20T22:07:58.706349+0000","last_became_active":"2024-05-20T22:07:57.732391+0000","last_became_peered":"2024-05-20T22:07:57.732391+0000","last_unstale":"2024-05-20T22:07:58.706349+0000","last_undegraded":"2024-05-20T22:07:58.706349+0000","last_fullsized":"2024-05-20T22:07:58.706349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:30.967143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092608+0000","last_change":"2024-05-20T22:07:52.665108+0000","last_active":"2024-05-20T22:08:00.092608+0000","last_peered":"2024-05-20T22:08:00.092608+0000","last_clean":"2024-05-20T22:08:00.092608+0000","last_became_active":"2024-05-20T22:07:52.664750+0000","last_became_peered":"2024-05-20T22:07:52.664750+0000","last_unstale":"2024-05-20T22:08:00.092608+0000","last_undegraded":"2024-05-20T22:08:00.092608+0000","last_fullsized":"2024-05-20T22:08:00.092608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:30.832431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092570+0000","last_change":"2024-05-20T22:07:53.671188+0000","last_active":"2024-05-20T22:08:00.092570+0000","last_peered":"2024-05-20T22:08:00.092570+0000","last_clean":"2024-05-20T22:08:00.092570+0000","last_became_active":"2024-05-20T22:07:53.670842+0000","last_became_peered":"2024-05-20T22:07:53.670842+0000","last_unstale":"2024-05-20T22:08:00.092570+0000","last_undegraded":"2024-05-20T22:08:00.092570+0000","last_fullsized":"2024-05-20T22:08:00.092570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:34.677214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093617+0000","last_change":"2024-05-20T22:07:57.725024+0000","last_active":"2024-05-20T22:08:00.093617+0000","last_peered":"2024-05-20T22:08:00.093617+0000","last_clean":"2024-05-20T22:08:00.093617+0000","last_became_active":"2024-05-20T22:07:57.724576+0000","last_became_peered":"2024-05-20T22:07:57.724576+0000","last_unstale":"2024-05-20T22:08:00.093617+0000","last_undegraded":"2024-05-20T22:08:00.093617+0000","last_fullsized":"2024-05-20T22:08:00.093617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:46:27.003055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705082+0000","last_change":"2024-05-20T22:07:53.680796+0000","last_active":"2024-05-20T22:07:58.705082+0000","last_peered":"2024-05-20T22:07:58.705082+0000","last_clean":"2024-05-20T22:07:58.705082+0000","last_became_active":"2024-05-20T22:07:53.680460+0000","last_became_peered":"2024-05-20T22:07:53.680460+0000","last_unstale":"2024-05-20T22:07:58.705082+0000","last_undegraded":"2024-05-20T22:07:58.705082+0000","last_fullsized":"2024-05-20T22:07:58.705082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705955+0000","last_change":"2024-05-20T22:07:53.664643+0000","last_active":"2024-05-20T22:07:58.705955+0000","last_peered":"2024-05-20T22:07:58.705955+0000","last_clean":"2024-05-20T22:07:58.705955+0000","last_became_active":"2024-05-20T22:07:53.664307+0000","last_became_peered":"2024-05-20T22:07:53.664307+0000","last_unstale":"2024-05-20T22:07:58.705955+0000","last_undegraded":"2024-05-20T22:07:58.705955+0000","last_fullsized":"2024-05-20T22:07:58.705955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:08.402858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093600+0000","last_change":"2024-05-20T22:07:53.669161+0000","last_active":"2024-05-20T22:08:00.093600+0000","last_peered":"2024-05-20T22:08:00.093600+0000","last_clean":"2024-05-20T22:08:00.093600+0000","last_became_active":"2024-05-20T22:07:53.668720+0000","last_became_peered":"2024-05-20T22:07:53.668720+0000","last_unstale":"2024-05-20T22:08:00.093600+0000","last_undegraded":"2024-05-20T22:08:00.093600+0000","last_fullsized":"2024-05-20T22:08:00.093600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:42.757306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580972+0000","last_change":"2024-05-20T22:07:53.667728+0000","last_active":"2024-05-20T22:07:59.580972+0000","last_peered":"2024-05-20T22:07:59.580972+0000","last_clean":"2024-05-20T22:07:59.580972+0000","last_became_active":"2024-05-20T22:07:53.665722+0000","last_became_peered":"2024-05-20T22:07:53.665722+0000","last_unstale":"2024-05-20T22:07:59.580972+0000","last_undegraded":"2024-05-20T22:07:59.580972+0000","last_fullsized":"2024-05-20T22:07:59.580972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:48.605408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093771+0000","last_change":"2024-05-20T22:07:53.665673+0000","last_active":"2024-05-20T22:08:00.093771+0000","last_peered":"2024-05-20T22:08:00.093771+0000","last_clean":"2024-05-20T22:08:00.093771+0000","last_became_active":"2024-05-20T22:07:53.665322+0000","last_became_peered":"2024-05-20T22:07:53.665322+0000","last_unstale":"2024-05-20T22:08:00.093771+0000","last_undegraded":"2024-05-20T22:08:00.093771+0000","last_fullsized":"2024-05-20T22:08:00.093771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092400+0000","last_change":"2024-05-20T22:07:53.673539+0000","last_active":"2024-05-20T22:08:00.092400+0000","last_peered":"2024-05-20T22:08:00.092400+0000","last_clean":"2024-05-20T22:08:00.092400+0000","last_became_active":"2024-05-20T22:07:53.673213+0000","last_became_peered":"2024-05-20T22:07:53.673213+0000","last_unstale":"2024-05-20T22:08:00.092400+0000","last_undegraded":"2024-05-20T22:08:00.092400+0000","last_fullsized":"2024-05-20T22:08:00.092400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706633+0000","last_change":"2024-05-20T22:07:53.676094+0000","last_active":"2024-05-20T22:07:58.706633+0000","last_peered":"2024-05-20T22:07:58.706633+0000","last_clean":"2024-05-20T22:07:58.706633+0000","last_became_active":"2024-05-20T22:07:53.675752+0000","last_became_peered":"2024-05-20T22:07:53.675752+0000","last_unstale":"2024-05-20T22:07:58.706633+0000","last_undegraded":"2024-05-20T22:07:58.706633+0000","last_fullsized":"2024-05-20T22:07:58.706633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093799+0000","last_change":"2024-05-20T22:07:53.665953+0000","last_active":"2024-05-20T22:08:00.093799+0000","last_peered":"2024-05-20T22:08:00.093799+0000","last_clean":"2024-05-20T22:08:00.093799+0000","last_became_active":"2024-05-20T22:07:53.665631+0000","last_became_peered":"2024-05-20T22:07:53.665631+0000","last_unstale":"2024-05-20T22:08:00.093799+0000","last_undegraded":"2024-05-20T22:08:00.093799+0000","last_fullsized":"2024-05-20T22:08:00.093799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092802+0000","last_change":"2024-05-20T22:07:53.669986+0000","last_active":"2024-05-20T22:08:00.092802+0000","last_peered":"2024-05-20T22:08:00.092802+0000","last_clean":"2024-05-20T22:08:00.092802+0000","last_became_active":"2024-05-20T22:07:53.669565+0000","last_became_peered":"2024-05-20T22:07:53.669565+0000","last_unstale":"2024-05-20T22:08:00.092802+0000","last_undegraded":"2024-05-20T22:08:00.092802+0000","last_fullsized":"2024-05-20T22:08:00.092802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092993+0000","last_change":"2024-05-20T22:07:53.667290+0000","last_active":"2024-05-20T22:08:00.092993+0000","last_peered":"2024-05-20T22:08:00.092993+0000","last_clean":"2024-05-20T22:08:00.092993+0000","last_became_active":"2024-05-20T22:07:53.666953+0000","last_became_peered":"2024-05-20T22:07:53.666953+0000","last_unstale":"2024-05-20T22:08:00.092993+0000","last_undegraded":"2024-05-20T22:08:00.092993+0000","last_fullsized":"2024-05-20T22:08:00.092993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:48.605408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093536+0000","last_change":"2024-05-20T22:07:53.673713+0000","last_active":"2024-05-20T22:08:00.093536+0000","last_peered":"2024-05-20T22:08:00.093536+0000","last_clean":"2024-05-20T22:08:00.093536+0000","last_became_active":"2024-05-20T22:07:53.673419+0000","last_became_peered":"2024-05-20T22:07:53.673419+0000","last_unstale":"2024-05-20T22:08:00.093536+0000","last_undegraded":"2024-05-20T22:08:00.093536+0000","last_fullsized":"2024-05-20T22:08:00.093536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580805+0000","last_change":"2024-05-20T22:07:53.657177+0000","last_active":"2024-05-20T22:07:59.580805+0000","last_peered":"2024-05-20T22:07:59.580805+0000","last_clean":"2024-05-20T22:07:59.580805+0000","last_became_active":"2024-05-20T22:07:53.656847+0000","last_became_peered":"2024-05-20T22:07:53.656847+0000","last_unstale":"2024-05-20T22:07:59.580805+0000","last_undegraded":"2024-05-20T22:07:59.580805+0000","last_fullsized":"2024-05-20T22:07:59.580805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706361+0000","last_change":"2024-05-20T22:07:53.675562+0000","last_active":"2024-05-20T22:07:58.706361+0000","last_peered":"2024-05-20T22:07:58.706361+0000","last_clean":"2024-05-20T22:07:58.706361+0000","last_became_active":"2024-05-20T22:07:53.673333+0000","last_became_peered":"2024-05-20T22:07:53.673333+0000","last_unstale":"2024-05-20T22:07:58.706361+0000","last_undegraded":"2024-05-20T22:07:58.706361+0000","last_fullsized":"2024-05-20T22:07:58.706361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706689+0000","last_change":"2024-05-20T22:07:53.664701+0000","last_active":"2024-05-20T22:07:58.706689+0000","last_peered":"2024-05-20T22:07:58.706689+0000","last_clean":"2024-05-20T22:07:58.706689+0000","last_became_active":"2024-05-20T22:07:53.663423+0000","last_became_peered":"2024-05-20T22:07:53.663423+0000","last_unstale":"2024-05-20T22:07:58.706689+0000","last_undegraded":"2024-05-20T22:07:58.706689+0000","last_fullsized":"2024-05-20T22:07:58.706689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706441+0000","last_change":"2024-05-20T22:07:53.677915+0000","last_active":"2024-05-20T22:07:58.706441+0000","last_peered":"2024-05-20T22:07:58.706441+0000","last_clean":"2024-05-20T22:07:58.706441+0000","last_became_active":"2024-05-20T22:07:53.677479+0000","last_became_peered":"2024-05-20T22:07:53.677479+0000","last_unstale":"2024-05-20T22:07:58.706441+0000","last_undegraded":"2024-05-20T22:07:58.706441+0000","last_fullsized":"2024-05-20T22:07:58.706441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:42.167135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092947+0000","last_change":"2024-05-20T22:07:53.674336+0000","last_active":"2024-05-20T22:08:00.092947+0000","last_peered":"2024-05-20T22:08:00.092947+0000","last_clean":"2024-05-20T22:08:00.092947+0000","last_became_active":"2024-05-20T22:07:53.673643+0000","last_became_peered":"2024-05-20T22:07:53.673643+0000","last_unstale":"2024-05-20T22:08:00.092947+0000","last_undegraded":"2024-05-20T22:08:00.092947+0000","last_fullsized":"2024-05-20T22:08:00.092947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:26.916927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092025+0000","last_change":"2024-05-20T22:07:53.680694+0000","last_active":"2024-05-20T22:08:00.092025+0000","last_peered":"2024-05-20T22:08:00.092025+0000","last_clean":"2024-05-20T22:08:00.092025+0000","last_became_active":"2024-05-20T22:07:53.680338+0000","last_became_peered":"2024-05-20T22:07:53.680338+0000","last_unstale":"2024-05-20T22:08:00.092025+0000","last_undegraded":"2024-05-20T22:08:00.092025+0000","last_fullsized":"2024-05-20T22:08:00.092025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092124+0000","last_change":"2024-05-20T22:07:53.668909+0000","last_active":"2024-05-20T22:08:00.092124+0000","last_peered":"2024-05-20T22:08:00.092124+0000","last_clean":"2024-05-20T22:08:00.092124+0000","last_became_active":"2024-05-20T22:07:53.668540+0000","last_became_peered":"2024-05-20T22:07:53.668540+0000","last_unstale":"2024-05-20T22:08:00.092124+0000","last_undegraded":"2024-05-20T22:08:00.092124+0000","last_fullsized":"2024-05-20T22:08:00.092124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580610+0000","last_change":"2024-05-20T22:07:53.671760+0000","last_active":"2024-05-20T22:07:59.580610+0000","last_peered":"2024-05-20T22:07:59.580610+0000","last_clean":"2024-05-20T22:07:59.580610+0000","last_became_active":"2024-05-20T22:07:53.671454+0000","last_became_peered":"2024-05-20T22:07:53.671454+0000","last_unstale":"2024-05-20T22:07:59.580610+0000","last_undegraded":"2024-05-20T22:07:59.580610+0000","last_fullsized":"2024-05-20T22:07:59.580610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:56.576260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092384+0000","last_change":"2024-05-20T22:07:53.678862+0000","last_active":"2024-05-20T22:08:00.092384+0000","last_peered":"2024-05-20T22:08:00.092384+0000","last_clean":"2024-05-20T22:08:00.092384+0000","last_became_active":"2024-05-20T22:07:53.678323+0000","last_became_peered":"2024-05-20T22:07:53.678323+0000","last_unstale":"2024-05-20T22:08:00.092384+0000","last_undegraded":"2024-05-20T22:08:00.092384+0000","last_fullsized":"2024-05-20T22:08:00.092384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093008+0000","last_change":"2024-05-20T22:07:53.677127+0000","last_active":"2024-05-20T22:08:00.093008+0000","last_peered":"2024-05-20T22:08:00.093008+0000","last_clean":"2024-05-20T22:08:00.093008+0000","last_became_active":"2024-05-20T22:07:53.676804+0000","last_became_peered":"2024-05-20T22:07:53.676804+0000","last_unstale":"2024-05-20T22:08:00.093008+0000","last_undegraded":"2024-05-20T22:08:00.093008+0000","last_fullsized":"2024-05-20T22:08:00.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:42.524695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706507+0000","last_change":"2024-05-20T22:07:53.677487+0000","last_active":"2024-05-20T22:07:58.706507+0000","last_peered":"2024-05-20T22:07:58.706507+0000","last_clean":"2024-05-20T22:07:58.706507+0000","last_became_active":"2024-05-20T22:07:53.677160+0000","last_became_peered":"2024-05-20T22:07:53.677160+0000","last_unstale":"2024-05-20T22:07:58.706507+0000","last_undegraded":"2024-05-20T22:07:58.706507+0000","last_fullsized":"2024-05-20T22:07:58.706507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706545+0000","last_change":"2024-05-20T22:07:53.675130+0000","last_active":"2024-05-20T22:07:58.706545+0000","last_peered":"2024-05-20T22:07:58.706545+0000","last_clean":"2024-05-20T22:07:58.706545+0000","last_became_active":"2024-05-20T22:07:53.674566+0000","last_became_peered":"2024-05-20T22:07:53.674566+0000","last_unstale":"2024-05-20T22:07:58.706545+0000","last_undegraded":"2024-05-20T22:07:58.706545+0000","last_fullsized":"2024-05-20T22:07:58.706545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707188+0000","last_change":"2024-05-20T22:07:53.668737+0000","last_active":"2024-05-20T22:07:58.707188+0000","last_peered":"2024-05-20T22:07:58.707188+0000","last_clean":"2024-05-20T22:07:58.707188+0000","last_became_active":"2024-05-20T22:07:53.668436+0000","last_became_peered":"2024-05-20T22:07:53.668436+0000","last_unstale":"2024-05-20T22:07:58.707188+0000","last_undegraded":"2024-05-20T22:07:58.707188+0000","last_fullsized":"2024-05-20T22:07:58.707188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:26.916927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092105+0000","last_change":"2024-05-20T22:07:53.667651+0000","last_active":"2024-05-20T22:08:00.092105+0000","last_peered":"2024-05-20T22:08:00.092105+0000","last_clean":"2024-05-20T22:08:00.092105+0000","last_became_active":"2024-05-20T22:07:53.667290+0000","last_became_peered":"2024-05-20T22:07:53.667290+0000","last_unstale":"2024-05-20T22:08:00.092105+0000","last_undegraded":"2024-05-20T22:08:00.092105+0000","last_fullsized":"2024-05-20T22:08:00.092105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705225+0000","last_change":"2024-05-20T22:07:53.655295+0000","last_active":"2024-05-20T22:07:58.705225+0000","last_peered":"2024-05-20T22:07:58.705225+0000","last_clean":"2024-05-20T22:07:58.705225+0000","last_became_active":"2024-05-20T22:07:53.654945+0000","last_became_peered":"2024-05-20T22:07:53.654945+0000","last_unstale":"2024-05-20T22:07:58.705225+0000","last_undegraded":"2024-05-20T22:07:58.705225+0000","last_fullsized":"2024-05-20T22:07:58.705225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:42.757306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093460+0000","last_change":"2024-05-20T22:07:53.681737+0000","last_active":"2024-05-20T22:08:00.093460+0000","last_peered":"2024-05-20T22:08:00.093460+0000","last_clean":"2024-05-20T22:08:00.093460+0000","last_became_active":"2024-05-20T22:07:53.681436+0000","last_became_peered":"2024-05-20T22:07:53.681436+0000","last_unstale":"2024-05-20T22:08:00.093460+0000","last_undegraded":"2024-05-20T22:08:00.093460+0000","last_fullsized":"2024-05-20T22:08:00.093460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705391+0000","last_change":"2024-05-20T22:07:53.662939+0000","last_active":"2024-05-20T22:07:58.705391+0000","last_peered":"2024-05-20T22:07:58.705391+0000","last_clean":"2024-05-20T22:07:58.705391+0000","last_became_active":"2024-05-20T22:07:53.662545+0000","last_became_peered":"2024-05-20T22:07:53.662545+0000","last_unstale":"2024-05-20T22:07:58.705391+0000","last_undegraded":"2024-05-20T22:07:58.705391+0000","last_fullsized":"2024-05-20T22:07:58.705391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092568+0000","last_change":"2024-05-20T22:07:53.677132+0000","last_active":"2024-05-20T22:08:00.092568+0000","last_peered":"2024-05-20T22:08:00.092568+0000","last_clean":"2024-05-20T22:08:00.092568+0000","last_became_active":"2024-05-20T22:07:53.676804+0000","last_became_peered":"2024-05-20T22:07:53.676804+0000","last_unstale":"2024-05-20T22:08:00.092568+0000","last_undegraded":"2024-05-20T22:08:00.092568+0000","last_fullsized":"2024-05-20T22:08:00.092568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580901+0000","last_change":"2024-05-20T22:07:53.669468+0000","last_active":"2024-05-20T22:07:59.580901+0000","last_peered":"2024-05-20T22:07:59.580901+0000","last_clean":"2024-05-20T22:07:59.580901+0000","last_became_active":"2024-05-20T22:07:53.669074+0000","last_became_peered":"2024-05-20T22:07:53.669074+0000","last_unstale":"2024-05-20T22:07:59.580901+0000","last_undegraded":"2024-05-20T22:07:59.580901+0000","last_fullsized":"2024-05-20T22:07:59.580901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:26.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092451+0000","last_change":"2024-05-20T22:07:53.680851+0000","last_active":"2024-05-20T22:08:00.092451+0000","last_peered":"2024-05-20T22:08:00.092451+0000","last_clean":"2024-05-20T22:08:00.092451+0000","last_became_active":"2024-05-20T22:07:53.680468+0000","last_became_peered":"2024-05-20T22:07:53.680468+0000","last_unstale":"2024-05-20T22:08:00.092451+0000","last_undegraded":"2024-05-20T22:08:00.092451+0000","last_fullsized":"2024-05-20T22:08:00.092451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707096+0000","last_change":"2024-05-20T22:07:53.675836+0000","last_active":"2024-05-20T22:07:58.707096+0000","last_peered":"2024-05-20T22:07:58.707096+0000","last_clean":"2024-05-20T22:07:58.707096+0000","last_became_active":"2024-05-20T22:07:53.675544+0000","last_became_peered":"2024-05-20T22:07:53.675544+0000","last_unstale":"2024-05-20T22:07:58.707096+0000","last_undegraded":"2024-05-20T22:07:58.707096+0000","last_fullsized":"2024-05-20T22:07:58.707096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:42.044571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:08:17.560 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:08:17.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.709+0000 7f182092f640 1 Processor -- start 2024-05-20T22:08:17.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.709+0000 7f182092f640 1 -- start start 2024-05-20T22:08:17.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f182092f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181c0b3b30 0x7f181c0b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f182092f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f181c15f100 0x7f181c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f182092f640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f181c0d3dd0 con 0x7f181c06c5c0 2024-05-20T22:08:17.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f182092f640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f181c0d40b0 con 0x7f181c15f100 2024-05-20T22:08:17.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f182092f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f181c0d4390 con 0x7f181c0b3b30 2024-05-20T22:08:17.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f1819d9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f181c15f100 0x7f181c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f181ad9d640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f181c06c5c0 0x7f181c162b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46502/0 (socket says 172.21.15.55:46502) 2024-05-20T22:08:17.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f1819d9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f181c15f100 0x7f181c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58314/0 (socket says 172.21.15.55:58314) 2024-05-20T22:08:17.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f181ad9d640 1 -- 172.21.15.55:0/391823320 learned_addr learned my addr 172.21.15.55:0/391823320 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:17.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f181a59c640 1 --2- 172.21.15.55:0/391823320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181c0b3b30 0x7f181c0b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f1819d9b640 1 -- 172.21.15.55:0/391823320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181c0b3b30 msgr2=0x7f181c0b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:17.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f1819d9b640 1 --2- 172.21.15.55:0/391823320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181c0b3b30 0x7f181c0b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f1819d9b640 1 -- 172.21.15.55:0/391823320 >> v1:172.21.15.55:6789/0 conn(0x7f181c06c5c0 legacy=0x7f181c162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f181959a640 1 -- 172.21.15.55:0/391823320 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2309443710 0 0) 0x7f181c0d3dd0 con 0x7f181c06c5c0 2024-05-20T22:08:17.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.711+0000 7f1819d9b640 1 -- 172.21.15.55:0/391823320 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c06b5a0 con 0x7f181c15f100 2024-05-20T22:08:17.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.712+0000 7f1819d9b640 1 --2- 172.21.15.55:0/391823320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c15f100 0x7f181c06c080 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18100097a0 tx=0x7f181002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe31a922641c5d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:17.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.712+0000 7f181959a640 1 -- 172.21.15.55:0/391823320 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810037070 con 0x7f181c15f100 2024-05-20T22:08:17.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.712+0000 7f181959a640 1 -- 172.21.15.55:0/391823320 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1810032440 con 0x7f181c15f100 2024-05-20T22:08:17.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.712+0000 7f181959a640 1 -- 172.21.15.55:0/391823320 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810032810 con 0x7f181c15f100 2024-05-20T22:08:17.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.713+0000 7f182092f640 1 -- 172.21.15.55:0/391823320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c15f100 msgr2=0x7f181c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.713+0000 7f182092f640 1 --2- 172.21.15.55:0/391823320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c15f100 0x7f181c06c080 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18100097a0 tx=0x7f181002a180 comp rx=0 tx=0).stop 2024-05-20T22:08:17.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.713+0000 7f182092f640 1 -- 172.21.15.55:0/391823320 shutdown_connections 2024-05-20T22:08:17.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.713+0000 7f182092f640 1 --2- 172.21.15.55:0/391823320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c15f100 0x7f181c06c080 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.713+0000 7f182092f640 1 --2- 172.21.15.55:0/391823320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181c0b3b30 0x7f181c0b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.713+0000 7f182092f640 1 -- 172.21.15.55:0/391823320 >> 172.21.15.55:0/391823320 conn(0x7f181c0d51d0 msgr2=0x7f181c0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.713+0000 7f182092f640 1 -- 172.21.15.55:0/391823320 shutdown_connections 2024-05-20T22:08:17.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.713+0000 7f182092f640 1 -- 172.21.15.55:0/391823320 wait complete. 2024-05-20T22:08:17.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.715+0000 7f182092f640 1 Processor -- start 2024-05-20T22:08:17.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.715+0000 7f182092f640 1 -- start start 2024-05-20T22:08:17.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f182092f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c06c5c0 0x7f181c160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f182092f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181c0b3b30 0x7f181c143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f182092f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c15f100 0x7f181c159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f182092f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f181c06c9d0 con 0x7f181c06c5c0 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f182092f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f181c06bb60 con 0x7f181c15f100 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f182092f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f181c06cb50 con 0x7f181c0b3b30 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f181a59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c06c5c0 0x7f181c160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f181a59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c06c5c0 0x7f181c160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58328/0 (socket says 172.21.15.55:58328) 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f181a59c640 1 -- 172.21.15.55:0/2473213344 learned_addr learned my addr 172.21.15.55:0/2473213344 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f1819d9b640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181c0b3b30 0x7f181c143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.716+0000 7f181ad9d640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c15f100 0x7f181c159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f181a59c640 1 -- 172.21.15.55:0/2473213344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181c0b3b30 msgr2=0x7f181c143cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f181a59c640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181c0b3b30 0x7f181c143cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f181a59c640 1 -- 172.21.15.55:0/2473213344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c15f100 msgr2=0x7f181c159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f181a59c640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c15f100 0x7f181c159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f181a59c640 1 -- 172.21.15.55:0/2473213344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c144210 con 0x7f181c06c5c0 2024-05-20T22:08:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f181ad9d640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c15f100 0x7f181c159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f181a59c640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c06c5c0 0x7f181c160fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f180c00e870 tx=0x7f180c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f180affd640 1 -- 172.21.15.55:0/2473213344 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c017070 con 0x7f181c06c5c0 2024-05-20T22:08:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f180affd640 1 -- 172.21.15.55:0/2473213344 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180c00c940 con 0x7f181c06c5c0 2024-05-20T22:08:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f182092f640 1 -- 172.21.15.55:0/2473213344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c1588c0 con 0x7f181c06c5c0 2024-05-20T22:08:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f180affd640 1 -- 172.21.15.55:0/2473213344 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c00cd10 con 0x7f181c06c5c0 2024-05-20T22:08:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.717+0000 7f182092f640 1 -- 172.21.15.55:0/2473213344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c159040 con 0x7f181c06c5c0 2024-05-20T22:08:17.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.718+0000 7f182092f640 1 -- 172.21.15.55:0/2473213344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181c069090 con 0x7f181c06c5c0 2024-05-20T22:08:17.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.719+0000 7f180affd640 1 -- 172.21.15.55:0/2473213344 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f180c01a070 con 0x7f181c06c5c0 2024-05-20T22:08:17.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.719+0000 7f180affd640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f17fc06be30 0x7f17fc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:17.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.720+0000 7f1819d9b640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f17fc06be30 0x7f17fc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:17.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.720+0000 7f180affd640 1 -- 172.21.15.55:0/2473213344 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f180c08af70 con 0x7f181c06c5c0 2024-05-20T22:08:17.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.720+0000 7f1819d9b640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f17fc06be30 0x7f17fc06e340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1810034000 tx=0x7f181002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:17.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.724+0000 7f180affd640 1 -- 172.21.15.55:0/2473213344 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f180c058f90 con 0x7f181c06c5c0 2024-05-20T22:08:17.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.905+0000 7f182092f640 1 -- 172.21.15.55:0/2473213344 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f181c0d2ee0 con 0x7f17fc06be30 2024-05-20T22:08:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.913+0000 7f180affd640 1 -- 172.21.15.55:0/2473213344 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332538 (secure 0 0 0) 0x7f181c0d2ee0 con 0x7f17fc06be30 2024-05-20T22:08:17.917 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:17.919 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.916+0000 7f182092f640 1 -- 172.21.15.55:0/2473213344 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f17fc06be30 msgr2=0x7f17fc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.916+0000 7f182092f640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f17fc06be30 0x7f17fc06e340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1810034000 tx=0x7f181002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.916+0000 7f182092f640 1 -- 172.21.15.55:0/2473213344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c06c5c0 msgr2=0x7f181c160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.916+0000 7f182092f640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c06c5c0 0x7f181c160fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f180c00e870 tx=0x7f180c00ed30 comp rx=0 tx=0).stop 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.916+0000 7f182092f640 1 -- 172.21.15.55:0/2473213344 shutdown_connections 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.916+0000 7f182092f640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f17fc06be30 0x7f17fc06e340 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.916+0000 7f182092f640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c15f100 0x7f181c159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.916+0000 7f182092f640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181c0b3b30 0x7f181c143cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.916+0000 7f182092f640 1 --2- 172.21.15.55:0/2473213344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f181c06c5c0 0x7f181c160fe0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.916+0000 7f182092f640 1 -- 172.21.15.55:0/2473213344 >> 172.21.15.55:0/2473213344 conn(0x7f181c0d51d0 msgr2=0x7f181c15cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.917+0000 7f182092f640 1 -- 172.21.15.55:0/2473213344 shutdown_connections 2024-05-20T22:08:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:17.917+0000 7f182092f640 1 -- 172.21.15.55:0/2473213344 wait complete. 2024-05-20T22:08:17.929 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-05-20T22:08:16.332123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236020,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738700,"statfs":{"total":773094113280,"available":772852428800,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002582"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581341+0000","last_change":"2024-05-20T22:07:52.664054+0000","last_active":"2024-05-20T22:07:59.581341+0000","last_peered":"2024-05-20T22:07:59.581341+0000","last_clean":"2024-05-20T22:07:59.581341+0000","last_became_active":"2024-05-20T22:07:52.663784+0000","last_became_peered":"2024-05-20T22:07:52.663784+0000","last_unstale":"2024-05-20T22:07:59.581341+0000","last_undegraded":"2024-05-20T22:07:59.581341+0000","last_fullsized":"2024-05-20T22:07:59.581341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:37.756571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092411+0000","last_change":"2024-05-20T22:07:53.678918+0000","last_active":"2024-05-20T22:08:00.092411+0000","last_peered":"2024-05-20T22:08:00.092411+0000","last_clean":"2024-05-20T22:08:00.092411+0000","last_became_active":"2024-05-20T22:07:53.678467+0000","last_became_peered":"2024-05-20T22:07:53.678467+0000","last_unstale":"2024-05-20T22:08:00.092411+0000","last_undegraded":"2024-05-20T22:08:00.092411+0000","last_fullsized":"2024-05-20T22:08:00.092411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092012+0000","last_change":"2024-05-20T22:07:57.732174+0000","last_active":"2024-05-20T22:08:00.092012+0000","last_peered":"2024-05-20T22:08:00.092012+0000","last_clean":"2024-05-20T22:08:00.092012+0000","last_became_active":"2024-05-20T22:07:57.731124+0000","last_became_peered":"2024-05-20T22:07:57.731124+0000","last_unstale":"2024-05-20T22:08:00.092012+0000","last_undegraded":"2024-05-20T22:08:00.092012+0000","last_fullsized":"2024-05-20T22:08:00.092012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:11.233995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092785+0000","last_change":"2024-05-20T22:07:52.664457+0000","last_active":"2024-05-20T22:08:00.092785+0000","last_peered":"2024-05-20T22:08:00.092785+0000","last_clean":"2024-05-20T22:08:00.092785+0000","last_became_active":"2024-05-20T22:07:52.664121+0000","last_became_peered":"2024-05-20T22:07:52.664121+0000","last_unstale":"2024-05-20T22:08:00.092785+0000","last_undegraded":"2024-05-20T22:08:00.092785+0000","last_fullsized":"2024-05-20T22:08:00.092785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581131+0000","last_change":"2024-05-20T22:07:53.669570+0000","last_active":"2024-05-20T22:07:59.581131+0000","last_peered":"2024-05-20T22:07:59.581131+0000","last_clean":"2024-05-20T22:07:59.581131+0000","last_became_active":"2024-05-20T22:07:53.668913+0000","last_became_peered":"2024-05-20T22:07:53.668913+0000","last_unstale":"2024-05-20T22:07:59.581131+0000","last_undegraded":"2024-05-20T22:07:59.581131+0000","last_fullsized":"2024-05-20T22:07:59.581131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092313+0000","last_change":"2024-05-20T22:07:57.728254+0000","last_active":"2024-05-20T22:08:00.092313+0000","last_peered":"2024-05-20T22:08:00.092313+0000","last_clean":"2024-05-20T22:08:00.092313+0000","last_became_active":"2024-05-20T22:07:57.727787+0000","last_became_peered":"2024-05-20T22:07:57.727787+0000","last_unstale":"2024-05-20T22:08:00.092313+0000","last_undegraded":"2024-05-20T22:08:00.092313+0000","last_fullsized":"2024-05-20T22:08:00.092313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:43.588870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706646+0000","last_change":"2024-05-20T22:07:52.657626+0000","last_active":"2024-05-20T22:07:58.706646+0000","last_peered":"2024-05-20T22:07:58.706646+0000","last_clean":"2024-05-20T22:07:58.706646+0000","last_became_active":"2024-05-20T22:07:52.657298+0000","last_became_peered":"2024-05-20T22:07:52.657298+0000","last_unstale":"2024-05-20T22:07:58.706646+0000","last_undegraded":"2024-05-20T22:07:58.706646+0000","last_fullsized":"2024-05-20T22:07:58.706646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:41.518373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092705+0000","last_change":"2024-05-20T22:07:53.669706+0000","last_active":"2024-05-20T22:08:00.092705+0000","last_peered":"2024-05-20T22:08:00.092705+0000","last_clean":"2024-05-20T22:08:00.092705+0000","last_became_active":"2024-05-20T22:07:53.669332+0000","last_became_peered":"2024-05-20T22:07:53.669332+0000","last_unstale":"2024-05-20T22:08:00.092705+0000","last_undegraded":"2024-05-20T22:08:00.092705+0000","last_fullsized":"2024-05-20T22:08:00.092705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092885+0000","last_change":"2024-05-20T22:07:57.734186+0000","last_active":"2024-05-20T22:08:00.092885+0000","last_peered":"2024-05-20T22:08:00.092885+0000","last_clean":"2024-05-20T22:08:00.092885+0000","last_became_active":"2024-05-20T22:07:57.733618+0000","last_became_peered":"2024-05-20T22:07:57.733618+0000","last_unstale":"2024-05-20T22:08:00.092885+0000","last_undegraded":"2024-05-20T22:08:00.092885+0000","last_fullsized":"2024-05-20T22:08:00.092885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:46.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706028+0000","last_change":"2024-05-20T22:07:52.660033+0000","last_active":"2024-05-20T22:07:58.706028+0000","last_peered":"2024-05-20T22:07:58.706028+0000","last_clean":"2024-05-20T22:07:58.706028+0000","last_became_active":"2024-05-20T22:07:52.659733+0000","last_became_peered":"2024-05-20T22:07:52.659733+0000","last_unstale":"2024-05-20T22:07:58.706028+0000","last_undegraded":"2024-05-20T22:07:58.706028+0000","last_fullsized":"2024-05-20T22:07:58.706028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:41.160812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093319+0000","last_change":"2024-05-20T22:07:53.669432+0000","last_active":"2024-05-20T22:08:00.093319+0000","last_peered":"2024-05-20T22:08:00.093319+0000","last_clean":"2024-05-20T22:08:00.093319+0000","last_became_active":"2024-05-20T22:07:53.668235+0000","last_became_peered":"2024-05-20T22:07:53.668235+0000","last_unstale":"2024-05-20T22:08:00.093319+0000","last_undegraded":"2024-05-20T22:08:00.093319+0000","last_fullsized":"2024-05-20T22:08:00.093319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:42.044571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092545+0000","last_change":"2024-05-20T22:07:57.734065+0000","last_active":"2024-05-20T22:08:00.092545+0000","last_peered":"2024-05-20T22:08:00.092545+0000","last_clean":"2024-05-20T22:08:00.092545+0000","last_became_active":"2024-05-20T22:07:57.733537+0000","last_became_peered":"2024-05-20T22:07:57.733537+0000","last_unstale":"2024-05-20T22:08:00.092545+0000","last_undegraded":"2024-05-20T22:08:00.092545+0000","last_fullsized":"2024-05-20T22:08:00.092545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:16.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705724+0000","last_change":"2024-05-20T22:07:53.675279+0000","last_active":"2024-05-20T22:07:58.705724+0000","last_peered":"2024-05-20T22:07:58.705724+0000","last_clean":"2024-05-20T22:07:58.705724+0000","last_became_active":"2024-05-20T22:07:53.674924+0000","last_became_peered":"2024-05-20T22:07:53.674924+0000","last_unstale":"2024-05-20T22:07:58.705724+0000","last_undegraded":"2024-05-20T22:07:58.705724+0000","last_fullsized":"2024-05-20T22:07:58.705724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705726+0000","last_change":"2024-05-20T22:07:52.655127+0000","last_active":"2024-05-20T22:07:58.705726+0000","last_peered":"2024-05-20T22:07:58.705726+0000","last_clean":"2024-05-20T22:07:58.705726+0000","last_became_active":"2024-05-20T22:07:52.654807+0000","last_became_peered":"2024-05-20T22:07:52.654807+0000","last_unstale":"2024-05-20T22:07:58.705726+0000","last_undegraded":"2024-05-20T22:07:58.705726+0000","last_fullsized":"2024-05-20T22:07:58.705726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093607+0000","last_change":"2024-05-20T22:07:57.732155+0000","last_active":"2024-05-20T22:08:00.093607+0000","last_peered":"2024-05-20T22:08:00.093607+0000","last_clean":"2024-05-20T22:08:00.093607+0000","last_became_active":"2024-05-20T22:07:57.731370+0000","last_became_peered":"2024-05-20T22:07:57.731370+0000","last_unstale":"2024-05-20T22:08:00.093607+0000","last_undegraded":"2024-05-20T22:08:00.093607+0000","last_fullsized":"2024-05-20T22:08:00.093607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:30.698775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092498+0000","last_change":"2024-05-20T22:07:53.677185+0000","last_active":"2024-05-20T22:08:00.092498+0000","last_peered":"2024-05-20T22:08:00.092498+0000","last_clean":"2024-05-20T22:08:00.092498+0000","last_became_active":"2024-05-20T22:07:53.676661+0000","last_became_peered":"2024-05-20T22:07:53.676661+0000","last_unstale":"2024-05-20T22:08:00.092498+0000","last_undegraded":"2024-05-20T22:08:00.092498+0000","last_fullsized":"2024-05-20T22:08:00.092498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093251+0000","last_change":"2024-05-20T22:07:52.668583+0000","last_active":"2024-05-20T22:08:00.093251+0000","last_peered":"2024-05-20T22:08:00.093251+0000","last_clean":"2024-05-20T22:08:00.093251+0000","last_became_active":"2024-05-20T22:07:52.667850+0000","last_became_peered":"2024-05-20T22:07:52.667850+0000","last_unstale":"2024-05-20T22:08:00.093251+0000","last_undegraded":"2024-05-20T22:08:00.093251+0000","last_fullsized":"2024-05-20T22:08:00.093251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092843+0000","last_change":"2024-05-20T22:07:57.730407+0000","last_active":"2024-05-20T22:08:00.092843+0000","last_peered":"2024-05-20T22:08:00.092843+0000","last_clean":"2024-05-20T22:08:00.092843+0000","last_became_active":"2024-05-20T22:07:57.729918+0000","last_became_peered":"2024-05-20T22:07:57.729918+0000","last_unstale":"2024-05-20T22:08:00.092843+0000","last_undegraded":"2024-05-20T22:08:00.092843+0000","last_fullsized":"2024-05-20T22:08:00.092843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:44.596866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092522+0000","last_change":"2024-05-20T22:07:53.675389+0000","last_active":"2024-05-20T22:08:00.092522+0000","last_peered":"2024-05-20T22:08:00.092522+0000","last_clean":"2024-05-20T22:08:00.092522+0000","last_became_active":"2024-05-20T22:07:53.675027+0000","last_became_peered":"2024-05-20T22:07:53.675027+0000","last_unstale":"2024-05-20T22:08:00.092522+0000","last_undegraded":"2024-05-20T22:08:00.092522+0000","last_fullsized":"2024-05-20T22:08:00.092522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092234+0000","last_change":"2024-05-20T22:07:52.662236+0000","last_active":"2024-05-20T22:08:00.092234+0000","last_peered":"2024-05-20T22:08:00.092234+0000","last_clean":"2024-05-20T22:08:00.092234+0000","last_became_active":"2024-05-20T22:07:52.660352+0000","last_became_peered":"2024-05-20T22:07:52.660352+0000","last_unstale":"2024-05-20T22:08:00.092234+0000","last_undegraded":"2024-05-20T22:08:00.092234+0000","last_fullsized":"2024-05-20T22:08:00.092234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:41.518373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093269+0000","last_change":"2024-05-20T22:07:57.736022+0000","last_active":"2024-05-20T22:08:00.093269+0000","last_peered":"2024-05-20T22:08:00.093269+0000","last_clean":"2024-05-20T22:08:00.093269+0000","last_became_active":"2024-05-20T22:07:57.735570+0000","last_became_peered":"2024-05-20T22:07:57.735570+0000","last_unstale":"2024-05-20T22:08:00.093269+0000","last_undegraded":"2024-05-20T22:08:00.093269+0000","last_fullsized":"2024-05-20T22:08:00.093269+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:53.866913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092952+0000","last_change":"2024-05-20T22:07:53.662746+0000","last_active":"2024-05-20T22:08:00.092952+0000","last_peered":"2024-05-20T22:08:00.092952+0000","last_clean":"2024-05-20T22:08:00.092952+0000","last_became_active":"2024-05-20T22:07:53.662458+0000","last_became_peered":"2024-05-20T22:07:53.662458+0000","last_unstale":"2024-05-20T22:08:00.092952+0000","last_undegraded":"2024-05-20T22:08:00.092952+0000","last_fullsized":"2024-05-20T22:08:00.092952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:18.996690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705082+0000","last_change":"2024-05-20T22:07:52.656158+0000","last_active":"2024-05-20T22:07:58.705082+0000","last_peered":"2024-05-20T22:07:58.705082+0000","last_clean":"2024-05-20T22:07:58.705082+0000","last_became_active":"2024-05-20T22:07:52.655448+0000","last_became_peered":"2024-05-20T22:07:52.655448+0000","last_unstale":"2024-05-20T22:07:58.705082+0000","last_undegraded":"2024-05-20T22:07:58.705082+0000","last_fullsized":"2024-05-20T22:07:58.705082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093755+0000","last_change":"2024-05-20T22:07:57.734374+0000","last_active":"2024-05-20T22:08:00.093755+0000","last_peered":"2024-05-20T22:08:00.093755+0000","last_clean":"2024-05-20T22:08:00.093755+0000","last_became_active":"2024-05-20T22:07:57.733737+0000","last_became_peered":"2024-05-20T22:07:57.733737+0000","last_unstale":"2024-05-20T22:08:00.093755+0000","last_undegraded":"2024-05-20T22:08:00.093755+0000","last_fullsized":"2024-05-20T22:08:00.093755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581291+0000","last_change":"2024-05-20T22:07:53.670700+0000","last_active":"2024-05-20T22:07:59.581291+0000","last_peered":"2024-05-20T22:07:59.581291+0000","last_clean":"2024-05-20T22:07:59.581291+0000","last_became_active":"2024-05-20T22:07:53.670361+0000","last_became_peered":"2024-05-20T22:07:53.670361+0000","last_unstale":"2024-05-20T22:07:59.581291+0000","last_undegraded":"2024-05-20T22:07:59.581291+0000","last_fullsized":"2024-05-20T22:07:59.581291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706857+0000","last_change":"2024-05-20T22:07:52.658539+0000","last_active":"2024-05-20T22:07:58.706857+0000","last_peered":"2024-05-20T22:07:58.706857+0000","last_clean":"2024-05-20T22:07:58.706857+0000","last_became_active":"2024-05-20T22:07:52.658232+0000","last_became_peered":"2024-05-20T22:07:52.658232+0000","last_unstale":"2024-05-20T22:07:58.706857+0000","last_undegraded":"2024-05-20T22:07:58.706857+0000","last_fullsized":"2024-05-20T22:07:58.706857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092305+0000","last_change":"2024-05-20T22:07:57.726798+0000","last_active":"2024-05-20T22:08:00.092305+0000","last_peered":"2024-05-20T22:08:00.092305+0000","last_clean":"2024-05-20T22:08:00.092305+0000","last_became_active":"2024-05-20T22:07:57.726321+0000","last_became_peered":"2024-05-20T22:07:57.726321+0000","last_unstale":"2024-05-20T22:08:00.092305+0000","last_undegraded":"2024-05-20T22:08:00.092305+0000","last_fullsized":"2024-05-20T22:08:00.092305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:00:00.091936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093912+0000","last_change":"2024-05-20T22:07:53.673830+0000","last_active":"2024-05-20T22:08:00.093912+0000","last_peered":"2024-05-20T22:08:00.093912+0000","last_clean":"2024-05-20T22:08:00.093912+0000","last_became_active":"2024-05-20T22:07:53.673509+0000","last_became_peered":"2024-05-20T22:07:53.673509+0000","last_unstale":"2024-05-20T22:08:00.093912+0000","last_undegraded":"2024-05-20T22:08:00.093912+0000","last_fullsized":"2024-05-20T22:08:00.093912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092565+0000","last_change":"2024-05-20T22:07:52.661954+0000","last_active":"2024-05-20T22:08:00.092565+0000","last_peered":"2024-05-20T22:08:00.092565+0000","last_clean":"2024-05-20T22:08:00.092565+0000","last_became_active":"2024-05-20T22:07:52.661304+0000","last_became_peered":"2024-05-20T22:07:52.661304+0000","last_unstale":"2024-05-20T22:08:00.092565+0000","last_undegraded":"2024-05-20T22:08:00.092565+0000","last_fullsized":"2024-05-20T22:08:00.092565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.091968+0000","last_change":"2024-05-20T22:07:57.731395+0000","last_active":"2024-05-20T22:08:00.091968+0000","last_peered":"2024-05-20T22:08:00.091968+0000","last_clean":"2024-05-20T22:08:00.091968+0000","last_became_active":"2024-05-20T22:07:57.730839+0000","last_became_peered":"2024-05-20T22:07:57.730839+0000","last_unstale":"2024-05-20T22:08:00.091968+0000","last_undegraded":"2024-05-20T22:08:00.091968+0000","last_fullsized":"2024-05-20T22:08:00.091968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:44:34.494247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092791+0000","last_change":"2024-05-20T22:07:53.673439+0000","last_active":"2024-05-20T22:08:00.092791+0000","last_peered":"2024-05-20T22:08:00.092791+0000","last_clean":"2024-05-20T22:08:00.092791+0000","last_became_active":"2024-05-20T22:07:53.672746+0000","last_became_peered":"2024-05-20T22:07:53.672746+0000","last_unstale":"2024-05-20T22:08:00.092791+0000","last_undegraded":"2024-05-20T22:08:00.092791+0000","last_fullsized":"2024-05-20T22:08:00.092791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:11:53.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706775+0000","last_change":"2024-05-20T22:07:52.662817+0000","last_active":"2024-05-20T22:07:58.706775+0000","last_peered":"2024-05-20T22:07:58.706775+0000","last_clean":"2024-05-20T22:07:58.706775+0000","last_became_active":"2024-05-20T22:07:52.662463+0000","last_became_peered":"2024-05-20T22:07:52.662463+0000","last_unstale":"2024-05-20T22:07:58.706775+0000","last_undegraded":"2024-05-20T22:07:58.706775+0000","last_fullsized":"2024-05-20T22:07:58.706775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:07.396536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580916+0000","last_change":"2024-05-20T22:07:57.730858+0000","last_active":"2024-05-20T22:07:59.580916+0000","last_peered":"2024-05-20T22:07:59.580916+0000","last_clean":"2024-05-20T22:07:59.580916+0000","last_became_active":"2024-05-20T22:07:57.730370+0000","last_became_peered":"2024-05-20T22:07:57.730370+0000","last_unstale":"2024-05-20T22:07:59.580916+0000","last_undegraded":"2024-05-20T22:07:59.580916+0000","last_fullsized":"2024-05-20T22:07:59.580916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093484+0000","last_change":"2024-05-20T22:07:53.673073+0000","last_active":"2024-05-20T22:08:00.093484+0000","last_peered":"2024-05-20T22:08:00.093484+0000","last_clean":"2024-05-20T22:08:00.093484+0000","last_became_active":"2024-05-20T22:07:53.672744+0000","last_became_peered":"2024-05-20T22:07:53.672744+0000","last_unstale":"2024-05-20T22:08:00.093484+0000","last_undegraded":"2024-05-20T22:08:00.093484+0000","last_fullsized":"2024-05-20T22:08:00.093484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:49.823373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093340+0000","last_change":"2024-05-20T22:07:52.668045+0000","last_active":"2024-05-20T22:08:00.093340+0000","last_peered":"2024-05-20T22:08:00.093340+0000","last_clean":"2024-05-20T22:08:00.093340+0000","last_became_active":"2024-05-20T22:07:52.666855+0000","last_became_peered":"2024-05-20T22:07:52.666855+0000","last_unstale":"2024-05-20T22:08:00.093340+0000","last_undegraded":"2024-05-20T22:08:00.093340+0000","last_fullsized":"2024-05-20T22:08:00.093340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092837+0000","last_change":"2024-05-20T22:07:57.740710+0000","last_active":"2024-05-20T22:08:00.092837+0000","last_peered":"2024-05-20T22:08:00.092837+0000","last_clean":"2024-05-20T22:08:00.092837+0000","last_became_active":"2024-05-20T22:07:57.737102+0000","last_became_peered":"2024-05-20T22:07:57.737102+0000","last_unstale":"2024-05-20T22:08:00.092837+0000","last_undegraded":"2024-05-20T22:08:00.092837+0000","last_fullsized":"2024-05-20T22:08:00.092837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092905+0000","last_change":"2024-05-20T22:07:53.667120+0000","last_active":"2024-05-20T22:08:00.092905+0000","last_peered":"2024-05-20T22:08:00.092905+0000","last_clean":"2024-05-20T22:08:00.092905+0000","last_became_active":"2024-05-20T22:07:53.666809+0000","last_became_peered":"2024-05-20T22:07:53.666809+0000","last_unstale":"2024-05-20T22:08:00.092905+0000","last_undegraded":"2024-05-20T22:08:00.092905+0000","last_fullsized":"2024-05-20T22:08:00.092905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:08.402858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093723+0000","last_change":"2024-05-20T22:07:52.664415+0000","last_active":"2024-05-20T22:08:00.093723+0000","last_peered":"2024-05-20T22:08:00.093723+0000","last_clean":"2024-05-20T22:08:00.093723+0000","last_became_active":"2024-05-20T22:07:52.664085+0000","last_became_peered":"2024-05-20T22:07:52.664085+0000","last_unstale":"2024-05-20T22:08:00.093723+0000","last_undegraded":"2024-05-20T22:08:00.093723+0000","last_fullsized":"2024-05-20T22:08:00.093723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092731+0000","last_change":"2024-05-20T22:07:57.736468+0000","last_active":"2024-05-20T22:08:00.092731+0000","last_peered":"2024-05-20T22:08:00.092731+0000","last_clean":"2024-05-20T22:08:00.092731+0000","last_became_active":"2024-05-20T22:07:57.736003+0000","last_became_peered":"2024-05-20T22:07:57.736003+0000","last_unstale":"2024-05-20T22:08:00.092731+0000","last_undegraded":"2024-05-20T22:08:00.092731+0000","last_fullsized":"2024-05-20T22:08:00.092731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.091885+0000","last_change":"2024-05-20T22:07:53.674097+0000","last_active":"2024-05-20T22:08:00.091885+0000","last_peered":"2024-05-20T22:08:00.091885+0000","last_clean":"2024-05-20T22:08:00.091885+0000","last_became_active":"2024-05-20T22:07:53.673766+0000","last_became_peered":"2024-05-20T22:07:53.673766+0000","last_unstale":"2024-05-20T22:08:00.091885+0000","last_undegraded":"2024-05-20T22:08:00.091885+0000","last_fullsized":"2024-05-20T22:08:00.091885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706155+0000","last_change":"2024-05-20T22:07:52.662684+0000","last_active":"2024-05-20T22:07:58.706155+0000","last_peered":"2024-05-20T22:07:58.706155+0000","last_clean":"2024-05-20T22:07:58.706155+0000","last_became_active":"2024-05-20T22:07:52.662220+0000","last_became_peered":"2024-05-20T22:07:52.662220+0000","last_unstale":"2024-05-20T22:07:58.706155+0000","last_undegraded":"2024-05-20T22:07:58.706155+0000","last_fullsized":"2024-05-20T22:07:58.706155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093363+0000","last_change":"2024-05-20T22:07:57.740613+0000","last_active":"2024-05-20T22:08:00.093363+0000","last_peered":"2024-05-20T22:08:00.093363+0000","last_clean":"2024-05-20T22:08:00.093363+0000","last_became_active":"2024-05-20T22:07:57.740165+0000","last_became_peered":"2024-05-20T22:07:57.740165+0000","last_unstale":"2024-05-20T22:08:00.093363+0000","last_undegraded":"2024-05-20T22:08:00.093363+0000","last_fullsized":"2024-05-20T22:08:00.093363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:03:25.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580372+0000","last_change":"2024-05-20T22:07:53.656427+0000","last_active":"2024-05-20T22:07:59.580372+0000","last_peered":"2024-05-20T22:07:59.580372+0000","last_clean":"2024-05-20T22:07:59.580372+0000","last_became_active":"2024-05-20T22:07:53.656081+0000","last_became_peered":"2024-05-20T22:07:53.656081+0000","last_unstale":"2024-05-20T22:07:59.580372+0000","last_undegraded":"2024-05-20T22:07:59.580372+0000","last_fullsized":"2024-05-20T22:07:59.580372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:58:02.316451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705143+0000","last_change":"2024-05-20T22:07:52.656503+0000","last_active":"2024-05-20T22:07:58.705143+0000","last_peered":"2024-05-20T22:07:58.705143+0000","last_clean":"2024-05-20T22:07:58.705143+0000","last_became_active":"2024-05-20T22:07:52.654665+0000","last_became_peered":"2024-05-20T22:07:52.654665+0000","last_unstale":"2024-05-20T22:07:58.705143+0000","last_undegraded":"2024-05-20T22:07:58.705143+0000","last_fullsized":"2024-05-20T22:07:58.705143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:33.670892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581267+0000","last_change":"2024-05-20T22:07:57.731186+0000","last_active":"2024-05-20T22:07:59.581267+0000","last_peered":"2024-05-20T22:07:59.581267+0000","last_clean":"2024-05-20T22:07:59.581267+0000","last_became_active":"2024-05-20T22:07:57.730239+0000","last_became_peered":"2024-05-20T22:07:57.730239+0000","last_unstale":"2024-05-20T22:07:59.581267+0000","last_undegraded":"2024-05-20T22:07:59.581267+0000","last_fullsized":"2024-05-20T22:07:59.581267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:29.190558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093069+0000","last_change":"2024-05-20T22:07:53.667981+0000","last_active":"2024-05-20T22:08:00.093069+0000","last_peered":"2024-05-20T22:08:00.093069+0000","last_clean":"2024-05-20T22:08:00.093069+0000","last_became_active":"2024-05-20T22:07:53.667320+0000","last_became_peered":"2024-05-20T22:07:53.667320+0000","last_unstale":"2024-05-20T22:08:00.093069+0000","last_undegraded":"2024-05-20T22:08:00.093069+0000","last_fullsized":"2024-05-20T22:08:00.093069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093094+0000","last_change":"2024-05-20T22:07:52.657838+0000","last_active":"2024-05-20T22:08:00.093094+0000","last_peered":"2024-05-20T22:08:00.093094+0000","last_clean":"2024-05-20T22:08:00.093094+0000","last_became_active":"2024-05-20T22:07:52.657530+0000","last_became_peered":"2024-05-20T22:07:52.657530+0000","last_unstale":"2024-05-20T22:08:00.093094+0000","last_undegraded":"2024-05-20T22:08:00.093094+0000","last_fullsized":"2024-05-20T22:08:00.093094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706234+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:07:58.706234+0000","last_peered":"2024-05-20T22:07:58.706234+0000","last_clean":"2024-05-20T22:07:58.706234+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:07:58.706234+0000","last_undegraded":"2024-05-20T22:07:58.706234+0000","last_fullsized":"2024-05-20T22:07:58.706234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706485+0000","last_change":"2024-05-20T22:07:57.738452+0000","last_active":"2024-05-20T22:07:58.706485+0000","last_peered":"2024-05-20T22:07:58.706485+0000","last_clean":"2024-05-20T22:07:58.706485+0000","last_became_active":"2024-05-20T22:07:57.737967+0000","last_became_peered":"2024-05-20T22:07:57.737967+0000","last_unstale":"2024-05-20T22:07:58.706485+0000","last_undegraded":"2024-05-20T22:07:58.706485+0000","last_fullsized":"2024-05-20T22:07:58.706485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:07.737798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706425+0000","last_change":"2024-05-20T22:07:53.677833+0000","last_active":"2024-05-20T22:07:58.706425+0000","last_peered":"2024-05-20T22:07:58.706425+0000","last_clean":"2024-05-20T22:07:58.706425+0000","last_became_active":"2024-05-20T22:07:53.677513+0000","last_became_peered":"2024-05-20T22:07:53.677513+0000","last_unstale":"2024-05-20T22:07:58.706425+0000","last_undegraded":"2024-05-20T22:07:58.706425+0000","last_fullsized":"2024-05-20T22:07:58.706425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706468+0000","last_change":"2024-05-20T22:07:52.659290+0000","last_active":"2024-05-20T22:07:58.706468+0000","last_peered":"2024-05-20T22:07:58.706468+0000","last_clean":"2024-05-20T22:07:58.706468+0000","last_became_active":"2024-05-20T22:07:52.658939+0000","last_became_peered":"2024-05-20T22:07:52.658939+0000","last_unstale":"2024-05-20T22:07:58.706468+0000","last_undegraded":"2024-05-20T22:07:58.706468+0000","last_fullsized":"2024-05-20T22:07:58.706468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:30.832431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705627+0000","last_change":"2024-05-20T22:07:57.732095+0000","last_active":"2024-05-20T22:07:58.705627+0000","last_peered":"2024-05-20T22:07:58.705627+0000","last_clean":"2024-05-20T22:07:58.705627+0000","last_became_active":"2024-05-20T22:07:57.731276+0000","last_became_peered":"2024-05-20T22:07:57.731276+0000","last_unstale":"2024-05-20T22:07:58.705627+0000","last_undegraded":"2024-05-20T22:07:58.705627+0000","last_fullsized":"2024-05-20T22:07:58.705627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:30.698775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092825+0000","last_change":"2024-05-20T22:07:53.669593+0000","last_active":"2024-05-20T22:08:00.092825+0000","last_peered":"2024-05-20T22:08:00.092825+0000","last_clean":"2024-05-20T22:08:00.092825+0000","last_became_active":"2024-05-20T22:07:53.668982+0000","last_became_peered":"2024-05-20T22:07:53.668982+0000","last_unstale":"2024-05-20T22:08:00.092825+0000","last_undegraded":"2024-05-20T22:08:00.092825+0000","last_fullsized":"2024-05-20T22:08:00.092825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:26.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092921+0000","last_change":"2024-05-20T22:07:52.666183+0000","last_active":"2024-05-20T22:08:00.092921+0000","last_peered":"2024-05-20T22:08:00.092921+0000","last_clean":"2024-05-20T22:08:00.092921+0000","last_became_active":"2024-05-20T22:07:52.662670+0000","last_became_peered":"2024-05-20T22:07:52.662670+0000","last_unstale":"2024-05-20T22:08:00.092921+0000","last_undegraded":"2024-05-20T22:08:00.092921+0000","last_fullsized":"2024-05-20T22:08:00.092921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581046+0000","last_change":"2024-05-20T22:07:57.734443+0000","last_active":"2024-05-20T22:07:59.581046+0000","last_peered":"2024-05-20T22:07:59.581046+0000","last_clean":"2024-05-20T22:07:59.581046+0000","last_became_active":"2024-05-20T22:07:57.734021+0000","last_became_peered":"2024-05-20T22:07:57.734021+0000","last_unstale":"2024-05-20T22:07:59.581046+0000","last_undegraded":"2024-05-20T22:07:59.581046+0000","last_fullsized":"2024-05-20T22:07:59.581046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:30.960467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707194+0000","last_change":"2024-05-20T22:07:53.666815+0000","last_active":"2024-05-20T22:07:58.707194+0000","last_peered":"2024-05-20T22:07:58.707194+0000","last_clean":"2024-05-20T22:07:58.707194+0000","last_became_active":"2024-05-20T22:07:53.666480+0000","last_became_peered":"2024-05-20T22:07:53.666480+0000","last_unstale":"2024-05-20T22:07:58.707194+0000","last_undegraded":"2024-05-20T22:07:58.707194+0000","last_fullsized":"2024-05-20T22:07:58.707194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:49.823373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706498+0000","last_change":"2024-05-20T22:07:52.661745+0000","last_active":"2024-05-20T22:07:58.706498+0000","last_peered":"2024-05-20T22:07:58.706498+0000","last_clean":"2024-05-20T22:07:58.706498+0000","last_became_active":"2024-05-20T22:07:52.661304+0000","last_became_peered":"2024-05-20T22:07:52.661304+0000","last_unstale":"2024-05-20T22:07:58.706498+0000","last_undegraded":"2024-05-20T22:07:58.706498+0000","last_fullsized":"2024-05-20T22:07:58.706498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092249+0000","last_change":"2024-05-20T22:07:57.722732+0000","last_active":"2024-05-20T22:08:00.092249+0000","last_peered":"2024-05-20T22:08:00.092249+0000","last_clean":"2024-05-20T22:08:00.092249+0000","last_became_active":"2024-05-20T22:07:57.722287+0000","last_became_peered":"2024-05-20T22:07:57.722287+0000","last_unstale":"2024-05-20T22:08:00.092249+0000","last_undegraded":"2024-05-20T22:08:00.092249+0000","last_fullsized":"2024-05-20T22:08:00.092249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:48.169892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706818+0000","last_change":"2024-05-20T22:07:53.668146+0000","last_active":"2024-05-20T22:07:58.706818+0000","last_peered":"2024-05-20T22:07:58.706818+0000","last_clean":"2024-05-20T22:07:58.706818+0000","last_became_active":"2024-05-20T22:07:53.666969+0000","last_became_peered":"2024-05-20T22:07:53.666969+0000","last_unstale":"2024-05-20T22:07:58.706818+0000","last_undegraded":"2024-05-20T22:07:58.706818+0000","last_fullsized":"2024-05-20T22:07:58.706818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093917+0000","last_change":"2024-05-20T22:07:52.651502+0000","last_active":"2024-05-20T22:08:00.093917+0000","last_peered":"2024-05-20T22:08:00.093917+0000","last_clean":"2024-05-20T22:08:00.093917+0000","last_became_active":"2024-05-20T22:07:52.651166+0000","last_became_peered":"2024-05-20T22:07:52.651166+0000","last_unstale":"2024-05-20T22:08:00.093917+0000","last_undegraded":"2024-05-20T22:08:00.093917+0000","last_fullsized":"2024-05-20T22:08:00.093917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:41:50.914741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092214+0000","last_change":"2024-05-20T22:07:57.736957+0000","last_active":"2024-05-20T22:08:00.092214+0000","last_peered":"2024-05-20T22:08:00.092214+0000","last_clean":"2024-05-20T22:08:00.092214+0000","last_became_active":"2024-05-20T22:07:57.736512+0000","last_became_peered":"2024-05-20T22:07:57.736512+0000","last_unstale":"2024-05-20T22:08:00.092214+0000","last_undegraded":"2024-05-20T22:08:00.092214+0000","last_fullsized":"2024-05-20T22:08:00.092214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:46.233608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705356+0000","last_change":"2024-05-20T22:07:53.655287+0000","last_active":"2024-05-20T22:07:58.705356+0000","last_peered":"2024-05-20T22:07:58.705356+0000","last_clean":"2024-05-20T22:07:58.705356+0000","last_became_active":"2024-05-20T22:07:53.653689+0000","last_became_peered":"2024-05-20T22:07:53.653689+0000","last_unstale":"2024-05-20T22:07:58.705356+0000","last_undegraded":"2024-05-20T22:07:58.705356+0000","last_fullsized":"2024-05-20T22:07:58.705356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:26.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705945+0000","last_change":"2024-05-20T22:07:52.653862+0000","last_active":"2024-05-20T22:07:58.705945+0000","last_peered":"2024-05-20T22:07:58.705945+0000","last_clean":"2024-05-20T22:07:58.705945+0000","last_became_active":"2024-05-20T22:07:52.653527+0000","last_became_peered":"2024-05-20T22:07:52.653527+0000","last_unstale":"2024-05-20T22:07:58.705945+0000","last_undegraded":"2024-05-20T22:07:58.705945+0000","last_fullsized":"2024-05-20T22:07:58.705945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581093+0000","last_change":"2024-05-20T22:07:57.737164+0000","last_active":"2024-05-20T22:07:59.581093+0000","last_peered":"2024-05-20T22:07:59.581093+0000","last_clean":"2024-05-20T22:07:59.581093+0000","last_became_active":"2024-05-20T22:07:57.736733+0000","last_became_peered":"2024-05-20T22:07:57.736733+0000","last_unstale":"2024-05-20T22:07:59.581093+0000","last_undegraded":"2024-05-20T22:07:59.581093+0000","last_fullsized":"2024-05-20T22:07:59.581093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:56:53.866913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705408+0000","last_change":"2024-05-20T22:07:53.680648+0000","last_active":"2024-05-20T22:07:58.705408+0000","last_peered":"2024-05-20T22:07:58.705408+0000","last_clean":"2024-05-20T22:07:58.705408+0000","last_became_active":"2024-05-20T22:07:53.680316+0000","last_became_peered":"2024-05-20T22:07:53.680316+0000","last_unstale":"2024-05-20T22:07:58.705408+0000","last_undegraded":"2024-05-20T22:07:58.705408+0000","last_fullsized":"2024-05-20T22:07:58.705408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706034+0000","last_change":"2024-05-20T22:07:52.660545+0000","last_active":"2024-05-20T22:07:58.706034+0000","last_peered":"2024-05-20T22:07:58.706034+0000","last_clean":"2024-05-20T22:07:58.706034+0000","last_became_active":"2024-05-20T22:07:52.660191+0000","last_became_peered":"2024-05-20T22:07:52.660191+0000","last_unstale":"2024-05-20T22:07:58.706034+0000","last_undegraded":"2024-05-20T22:07:58.706034+0000","last_fullsized":"2024-05-20T22:07:58.706034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093896+0000","last_change":"2024-05-20T22:07:57.730561+0000","last_active":"2024-05-20T22:08:00.093896+0000","last_peered":"2024-05-20T22:08:00.093896+0000","last_clean":"2024-05-20T22:08:00.093896+0000","last_became_active":"2024-05-20T22:07:57.728162+0000","last_became_peered":"2024-05-20T22:07:57.728162+0000","last_unstale":"2024-05-20T22:08:00.093896+0000","last_undegraded":"2024-05-20T22:08:00.093896+0000","last_fullsized":"2024-05-20T22:08:00.093896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:18.032764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705624+0000","last_change":"2024-05-20T22:07:53.654238+0000","last_active":"2024-05-20T22:07:58.705624+0000","last_peered":"2024-05-20T22:07:58.705624+0000","last_clean":"2024-05-20T22:07:58.705624+0000","last_became_active":"2024-05-20T22:07:53.653629+0000","last_became_peered":"2024-05-20T22:07:53.653629+0000","last_unstale":"2024-05-20T22:07:58.705624+0000","last_undegraded":"2024-05-20T22:07:58.705624+0000","last_fullsized":"2024-05-20T22:07:58.705624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093667+0000","last_change":"2024-05-20T22:07:52.667566+0000","last_active":"2024-05-20T22:08:00.093667+0000","last_peered":"2024-05-20T22:08:00.093667+0000","last_clean":"2024-05-20T22:08:00.093667+0000","last_became_active":"2024-05-20T22:07:52.667214+0000","last_became_peered":"2024-05-20T22:07:52.667214+0000","last_unstale":"2024-05-20T22:08:00.093667+0000","last_undegraded":"2024-05-20T22:08:00.093667+0000","last_fullsized":"2024-05-20T22:08:00.093667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093690+0000","last_change":"2024-05-20T22:07:57.728380+0000","last_active":"2024-05-20T22:08:00.093690+0000","last_peered":"2024-05-20T22:08:00.093690+0000","last_clean":"2024-05-20T22:08:00.093690+0000","last_became_active":"2024-05-20T22:07:57.726788+0000","last_became_peered":"2024-05-20T22:07:57.726788+0000","last_unstale":"2024-05-20T22:08:00.093690+0000","last_undegraded":"2024-05-20T22:08:00.093690+0000","last_fullsized":"2024-05-20T22:08:00.093690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:46:04.066141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580651+0000","last_change":"2024-05-20T22:07:53.659126+0000","last_active":"2024-05-20T22:07:59.580651+0000","last_peered":"2024-05-20T22:07:59.580651+0000","last_clean":"2024-05-20T22:07:59.580651+0000","last_became_active":"2024-05-20T22:07:53.658751+0000","last_became_peered":"2024-05-20T22:07:53.658751+0000","last_unstale":"2024-05-20T22:07:59.580651+0000","last_undegraded":"2024-05-20T22:07:59.580651+0000","last_fullsized":"2024-05-20T22:07:59.580651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581140+0000","last_change":"2024-05-20T22:07:52.659607+0000","last_active":"2024-05-20T22:07:59.581140+0000","last_peered":"2024-05-20T22:07:59.581140+0000","last_clean":"2024-05-20T22:07:59.581140+0000","last_became_active":"2024-05-20T22:07:52.658939+0000","last_became_peered":"2024-05-20T22:07:52.658939+0000","last_unstale":"2024-05-20T22:07:59.581140+0000","last_undegraded":"2024-05-20T22:07:59.581140+0000","last_fullsized":"2024-05-20T22:07:59.581140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:12:51.889148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093500+0000","last_change":"2024-05-20T22:07:57.726495+0000","last_active":"2024-05-20T22:08:00.093500+0000","last_peered":"2024-05-20T22:08:00.093500+0000","last_clean":"2024-05-20T22:08:00.093500+0000","last_became_active":"2024-05-20T22:07:57.726012+0000","last_became_peered":"2024-05-20T22:07:57.726012+0000","last_unstale":"2024-05-20T22:08:00.093500+0000","last_undegraded":"2024-05-20T22:08:00.093500+0000","last_fullsized":"2024-05-20T22:08:00.093500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:44:34.494247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705411+0000","last_change":"2024-05-20T22:07:53.666422+0000","last_active":"2024-05-20T22:07:58.705411+0000","last_peered":"2024-05-20T22:07:58.705411+0000","last_clean":"2024-05-20T22:07:58.705411+0000","last_became_active":"2024-05-20T22:07:53.666109+0000","last_became_peered":"2024-05-20T22:07:53.666109+0000","last_unstale":"2024-05-20T22:07:58.705411+0000","last_undegraded":"2024-05-20T22:07:58.705411+0000","last_fullsized":"2024-05-20T22:07:58.705411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:03.694258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:08.748697+0000","last_change":"2024-05-20T22:07:52.661686+0000","last_active":"2024-05-20T22:08:08.748697+0000","last_peered":"2024-05-20T22:08:08.748697+0000","last_clean":"2024-05-20T22:08:08.748697+0000","last_became_active":"2024-05-20T22:07:52.661192+0000","last_became_peered":"2024-05-20T22:07:52.661192+0000","last_unstale":"2024-05-20T22:08:08.748697+0000","last_undegraded":"2024-05-20T22:08:08.748697+0000","last_fullsized":"2024-05-20T22:08:08.748697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093188+0000","last_change":"2024-05-20T22:07:57.733250+0000","last_active":"2024-05-20T22:08:00.093188+0000","last_peered":"2024-05-20T22:08:00.093188+0000","last_clean":"2024-05-20T22:08:00.093188+0000","last_became_active":"2024-05-20T22:07:57.732096+0000","last_became_peered":"2024-05-20T22:07:57.732096+0000","last_unstale":"2024-05-20T22:08:00.093188+0000","last_undegraded":"2024-05-20T22:08:00.093188+0000","last_fullsized":"2024-05-20T22:08:00.093188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:30.960467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093127+0000","last_change":"2024-05-20T22:07:53.664378+0000","last_active":"2024-05-20T22:08:00.093127+0000","last_peered":"2024-05-20T22:08:00.093127+0000","last_clean":"2024-05-20T22:08:00.093127+0000","last_became_active":"2024-05-20T22:07:53.664061+0000","last_became_peered":"2024-05-20T22:07:53.664061+0000","last_unstale":"2024-05-20T22:08:00.093127+0000","last_undegraded":"2024-05-20T22:08:00.093127+0000","last_fullsized":"2024-05-20T22:08:00.093127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:42.167135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092791+0000","last_change":"2024-05-20T22:07:52.665754+0000","last_active":"2024-05-20T22:08:00.092791+0000","last_peered":"2024-05-20T22:08:00.092791+0000","last_clean":"2024-05-20T22:08:00.092791+0000","last_became_active":"2024-05-20T22:07:52.663248+0000","last_became_peered":"2024-05-20T22:07:52.663248+0000","last_unstale":"2024-05-20T22:08:00.092791+0000","last_undegraded":"2024-05-20T22:08:00.092791+0000","last_fullsized":"2024-05-20T22:08:00.092791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:49.857547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092574+0000","last_change":"2024-05-20T22:07:57.732444+0000","last_active":"2024-05-20T22:08:00.092574+0000","last_peered":"2024-05-20T22:08:00.092574+0000","last_clean":"2024-05-20T22:08:00.092574+0000","last_became_active":"2024-05-20T22:07:57.731063+0000","last_became_peered":"2024-05-20T22:07:57.731063+0000","last_unstale":"2024-05-20T22:08:00.092574+0000","last_undegraded":"2024-05-20T22:08:00.092574+0000","last_fullsized":"2024-05-20T22:08:00.092574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:17.240623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581288+0000","last_change":"2024-05-20T22:07:53.669268+0000","last_active":"2024-05-20T22:07:59.581288+0000","last_peered":"2024-05-20T22:07:59.581288+0000","last_clean":"2024-05-20T22:07:59.581288+0000","last_became_active":"2024-05-20T22:07:53.666127+0000","last_became_peered":"2024-05-20T22:07:53.666127+0000","last_unstale":"2024-05-20T22:07:59.581288+0000","last_undegraded":"2024-05-20T22:07:59.581288+0000","last_fullsized":"2024-05-20T22:07:59.581288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093187+0000","last_change":"2024-05-20T22:07:52.660573+0000","last_active":"2024-05-20T22:08:00.093187+0000","last_peered":"2024-05-20T22:08:00.093187+0000","last_clean":"2024-05-20T22:08:00.093187+0000","last_became_active":"2024-05-20T22:07:52.660227+0000","last_became_peered":"2024-05-20T22:07:52.660227+0000","last_unstale":"2024-05-20T22:08:00.093187+0000","last_undegraded":"2024-05-20T22:08:00.093187+0000","last_fullsized":"2024-05-20T22:08:00.093187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:21:17.990368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092346+0000","last_change":"2024-05-20T22:07:57.726865+0000","last_active":"2024-05-20T22:08:00.092346+0000","last_peered":"2024-05-20T22:08:00.092346+0000","last_clean":"2024-05-20T22:08:00.092346+0000","last_became_active":"2024-05-20T22:07:57.726411+0000","last_became_peered":"2024-05-20T22:07:57.726411+0000","last_unstale":"2024-05-20T22:08:00.092346+0000","last_undegraded":"2024-05-20T22:08:00.092346+0000","last_fullsized":"2024-05-20T22:08:00.092346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:12:14.532804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093027+0000","last_change":"2024-05-20T22:07:53.674069+0000","last_active":"2024-05-20T22:08:00.093027+0000","last_peered":"2024-05-20T22:08:00.093027+0000","last_clean":"2024-05-20T22:08:00.093027+0000","last_became_active":"2024-05-20T22:07:53.673668+0000","last_became_peered":"2024-05-20T22:07:53.673668+0000","last_unstale":"2024-05-20T22:08:00.093027+0000","last_undegraded":"2024-05-20T22:08:00.093027+0000","last_fullsized":"2024-05-20T22:08:00.093027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:03.694258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706745+0000","last_change":"2024-05-20T22:07:52.659232+0000","last_active":"2024-05-20T22:07:58.706745+0000","last_peered":"2024-05-20T22:07:58.706745+0000","last_clean":"2024-05-20T22:07:58.706745+0000","last_became_active":"2024-05-20T22:07:52.657718+0000","last_became_peered":"2024-05-20T22:07:52.657718+0000","last_unstale":"2024-05-20T22:07:58.706745+0000","last_undegraded":"2024-05-20T22:07:58.706745+0000","last_fullsized":"2024-05-20T22:07:58.706745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:00.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092928+0000","last_change":"2024-05-20T22:07:57.730328+0000","last_active":"2024-05-20T22:08:00.092928+0000","last_peered":"2024-05-20T22:08:00.092928+0000","last_clean":"2024-05-20T22:08:00.092928+0000","last_became_active":"2024-05-20T22:07:57.729093+0000","last_became_peered":"2024-05-20T22:07:57.729093+0000","last_unstale":"2024-05-20T22:08:00.092928+0000","last_undegraded":"2024-05-20T22:08:00.092928+0000","last_fullsized":"2024-05-20T22:08:00.092928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:42:24.385164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581132+0000","last_change":"2024-05-20T22:07:57.738586+0000","last_active":"2024-05-20T22:07:59.581132+0000","last_peered":"2024-05-20T22:07:59.581132+0000","last_clean":"2024-05-20T22:07:59.581132+0000","last_became_active":"2024-05-20T22:07:57.736969+0000","last_became_peered":"2024-05-20T22:07:57.736969+0000","last_unstale":"2024-05-20T22:07:59.581132+0000","last_undegraded":"2024-05-20T22:07:59.581132+0000","last_fullsized":"2024-05-20T22:07:59.581132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:03:25.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092006+0000","last_change":"2024-05-20T22:07:53.677156+0000","last_active":"2024-05-20T22:08:00.092006+0000","last_peered":"2024-05-20T22:08:00.092006+0000","last_clean":"2024-05-20T22:08:00.092006+0000","last_became_active":"2024-05-20T22:07:53.676805+0000","last_became_peered":"2024-05-20T22:07:53.676805+0000","last_unstale":"2024-05-20T22:08:00.092006+0000","last_undegraded":"2024-05-20T22:08:00.092006+0000","last_fullsized":"2024-05-20T22:08:00.092006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092929+0000","last_change":"2024-05-20T22:07:52.659254+0000","last_active":"2024-05-20T22:08:00.092929+0000","last_peered":"2024-05-20T22:08:00.092929+0000","last_clean":"2024-05-20T22:08:00.092929+0000","last_became_active":"2024-05-20T22:07:52.658938+0000","last_became_peered":"2024-05-20T22:07:52.658938+0000","last_unstale":"2024-05-20T22:08:00.092929+0000","last_undegraded":"2024-05-20T22:08:00.092929+0000","last_fullsized":"2024-05-20T22:08:00.092929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706615+0000","last_change":"2024-05-20T22:07:57.731289+0000","last_active":"2024-05-20T22:07:58.706615+0000","last_peered":"2024-05-20T22:07:58.706615+0000","last_clean":"2024-05-20T22:07:58.706615+0000","last_became_active":"2024-05-20T22:07:57.730844+0000","last_became_peered":"2024-05-20T22:07:57.730844+0000","last_unstale":"2024-05-20T22:07:58.706615+0000","last_undegraded":"2024-05-20T22:07:58.706615+0000","last_fullsized":"2024-05-20T22:07:58.706615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:46.088111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092411+0000","last_change":"2024-05-20T22:07:53.676088+0000","last_active":"2024-05-20T22:08:00.092411+0000","last_peered":"2024-05-20T22:08:00.092411+0000","last_clean":"2024-05-20T22:08:00.092411+0000","last_became_active":"2024-05-20T22:07:53.675756+0000","last_became_peered":"2024-05-20T22:07:53.675756+0000","last_unstale":"2024-05-20T22:08:00.092411+0000","last_undegraded":"2024-05-20T22:08:00.092411+0000","last_fullsized":"2024-05-20T22:08:00.092411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:58.952407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093073+0000","last_change":"2024-05-20T22:07:52.660691+0000","last_active":"2024-05-20T22:08:00.093073+0000","last_peered":"2024-05-20T22:08:00.093073+0000","last_clean":"2024-05-20T22:08:00.093073+0000","last_became_active":"2024-05-20T22:07:52.660324+0000","last_became_peered":"2024-05-20T22:07:52.660324+0000","last_unstale":"2024-05-20T22:08:00.093073+0000","last_undegraded":"2024-05-20T22:08:00.093073+0000","last_fullsized":"2024-05-20T22:08:00.093073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:37.756571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706140+0000","last_change":"2024-05-20T22:07:52.660131+0000","last_active":"2024-05-20T22:07:58.706140+0000","last_peered":"2024-05-20T22:07:58.706140+0000","last_clean":"2024-05-20T22:07:58.706140+0000","last_became_active":"2024-05-20T22:07:52.659784+0000","last_became_peered":"2024-05-20T22:07:52.659784+0000","last_unstale":"2024-05-20T22:07:58.706140+0000","last_undegraded":"2024-05-20T22:07:58.706140+0000","last_fullsized":"2024-05-20T22:07:58.706140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:45.930435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.581103+0000","last_change":"2024-05-20T22:07:53.659806+0000","last_active":"2024-05-20T22:07:59.581103+0000","last_peered":"2024-05-20T22:07:59.581103+0000","last_clean":"2024-05-20T22:07:59.581103+0000","last_became_active":"2024-05-20T22:07:53.659275+0000","last_became_peered":"2024-05-20T22:07:53.659275+0000","last_unstale":"2024-05-20T22:07:59.581103+0000","last_undegraded":"2024-05-20T22:07:59.581103+0000","last_fullsized":"2024-05-20T22:07:59.581103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706349+0000","last_change":"2024-05-20T22:07:57.732871+0000","last_active":"2024-05-20T22:07:58.706349+0000","last_peered":"2024-05-20T22:07:58.706349+0000","last_clean":"2024-05-20T22:07:58.706349+0000","last_became_active":"2024-05-20T22:07:57.732391+0000","last_became_peered":"2024-05-20T22:07:57.732391+0000","last_unstale":"2024-05-20T22:07:58.706349+0000","last_undegraded":"2024-05-20T22:07:58.706349+0000","last_fullsized":"2024-05-20T22:07:58.706349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:30.967143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092608+0000","last_change":"2024-05-20T22:07:52.665108+0000","last_active":"2024-05-20T22:08:00.092608+0000","last_peered":"2024-05-20T22:08:00.092608+0000","last_clean":"2024-05-20T22:08:00.092608+0000","last_became_active":"2024-05-20T22:07:52.664750+0000","last_became_peered":"2024-05-20T22:07:52.664750+0000","last_unstale":"2024-05-20T22:08:00.092608+0000","last_undegraded":"2024-05-20T22:08:00.092608+0000","last_fullsized":"2024-05-20T22:08:00.092608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:51.627456+0000","last_clean_scrub_stamp":"2024-05-20T22:07:51.627456+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:30.832431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092570+0000","last_change":"2024-05-20T22:07:53.671188+0000","last_active":"2024-05-20T22:08:00.092570+0000","last_peered":"2024-05-20T22:08:00.092570+0000","last_clean":"2024-05-20T22:08:00.092570+0000","last_became_active":"2024-05-20T22:07:53.670842+0000","last_became_peered":"2024-05-20T22:07:53.670842+0000","last_unstale":"2024-05-20T22:08:00.092570+0000","last_undegraded":"2024-05-20T22:08:00.092570+0000","last_fullsized":"2024-05-20T22:08:00.092570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:11:34.677214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093617+0000","last_change":"2024-05-20T22:07:57.725024+0000","last_active":"2024-05-20T22:08:00.093617+0000","last_peered":"2024-05-20T22:08:00.093617+0000","last_clean":"2024-05-20T22:08:00.093617+0000","last_became_active":"2024-05-20T22:07:57.724576+0000","last_became_peered":"2024-05-20T22:07:57.724576+0000","last_unstale":"2024-05-20T22:08:00.093617+0000","last_undegraded":"2024-05-20T22:08:00.093617+0000","last_fullsized":"2024-05-20T22:08:00.093617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:56.677318+0000","last_clean_scrub_stamp":"2024-05-20T22:07:56.677318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:46:27.003055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705082+0000","last_change":"2024-05-20T22:07:53.680796+0000","last_active":"2024-05-20T22:07:58.705082+0000","last_peered":"2024-05-20T22:07:58.705082+0000","last_clean":"2024-05-20T22:07:58.705082+0000","last_became_active":"2024-05-20T22:07:53.680460+0000","last_became_peered":"2024-05-20T22:07:53.680460+0000","last_unstale":"2024-05-20T22:07:58.705082+0000","last_undegraded":"2024-05-20T22:07:58.705082+0000","last_fullsized":"2024-05-20T22:07:58.705082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:33:57.252433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705955+0000","last_change":"2024-05-20T22:07:53.664643+0000","last_active":"2024-05-20T22:07:58.705955+0000","last_peered":"2024-05-20T22:07:58.705955+0000","last_clean":"2024-05-20T22:07:58.705955+0000","last_became_active":"2024-05-20T22:07:53.664307+0000","last_became_peered":"2024-05-20T22:07:53.664307+0000","last_unstale":"2024-05-20T22:07:58.705955+0000","last_undegraded":"2024-05-20T22:07:58.705955+0000","last_fullsized":"2024-05-20T22:07:58.705955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:37:08.402858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093600+0000","last_change":"2024-05-20T22:07:53.669161+0000","last_active":"2024-05-20T22:08:00.093600+0000","last_peered":"2024-05-20T22:08:00.093600+0000","last_clean":"2024-05-20T22:08:00.093600+0000","last_became_active":"2024-05-20T22:07:53.668720+0000","last_became_peered":"2024-05-20T22:07:53.668720+0000","last_unstale":"2024-05-20T22:08:00.093600+0000","last_undegraded":"2024-05-20T22:08:00.093600+0000","last_fullsized":"2024-05-20T22:08:00.093600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:42.757306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580972+0000","last_change":"2024-05-20T22:07:53.667728+0000","last_active":"2024-05-20T22:07:59.580972+0000","last_peered":"2024-05-20T22:07:59.580972+0000","last_clean":"2024-05-20T22:07:59.580972+0000","last_became_active":"2024-05-20T22:07:53.665722+0000","last_became_peered":"2024-05-20T22:07:53.665722+0000","last_unstale":"2024-05-20T22:07:59.580972+0000","last_undegraded":"2024-05-20T22:07:59.580972+0000","last_fullsized":"2024-05-20T22:07:59.580972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:48.605408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093771+0000","last_change":"2024-05-20T22:07:53.665673+0000","last_active":"2024-05-20T22:08:00.093771+0000","last_peered":"2024-05-20T22:08:00.093771+0000","last_clean":"2024-05-20T22:08:00.093771+0000","last_became_active":"2024-05-20T22:07:53.665322+0000","last_became_peered":"2024-05-20T22:07:53.665322+0000","last_unstale":"2024-05-20T22:08:00.093771+0000","last_undegraded":"2024-05-20T22:08:00.093771+0000","last_fullsized":"2024-05-20T22:08:00.093771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092400+0000","last_change":"2024-05-20T22:07:53.673539+0000","last_active":"2024-05-20T22:08:00.092400+0000","last_peered":"2024-05-20T22:08:00.092400+0000","last_clean":"2024-05-20T22:08:00.092400+0000","last_became_active":"2024-05-20T22:07:53.673213+0000","last_became_peered":"2024-05-20T22:07:53.673213+0000","last_unstale":"2024-05-20T22:08:00.092400+0000","last_undegraded":"2024-05-20T22:08:00.092400+0000","last_fullsized":"2024-05-20T22:08:00.092400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706633+0000","last_change":"2024-05-20T22:07:53.676094+0000","last_active":"2024-05-20T22:07:58.706633+0000","last_peered":"2024-05-20T22:07:58.706633+0000","last_clean":"2024-05-20T22:07:58.706633+0000","last_became_active":"2024-05-20T22:07:53.675752+0000","last_became_peered":"2024-05-20T22:07:53.675752+0000","last_unstale":"2024-05-20T22:07:58.706633+0000","last_undegraded":"2024-05-20T22:07:58.706633+0000","last_fullsized":"2024-05-20T22:07:58.706633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093799+0000","last_change":"2024-05-20T22:07:53.665953+0000","last_active":"2024-05-20T22:08:00.093799+0000","last_peered":"2024-05-20T22:08:00.093799+0000","last_clean":"2024-05-20T22:08:00.093799+0000","last_became_active":"2024-05-20T22:07:53.665631+0000","last_became_peered":"2024-05-20T22:07:53.665631+0000","last_unstale":"2024-05-20T22:08:00.093799+0000","last_undegraded":"2024-05-20T22:08:00.093799+0000","last_fullsized":"2024-05-20T22:08:00.093799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092802+0000","last_change":"2024-05-20T22:07:53.669986+0000","last_active":"2024-05-20T22:08:00.092802+0000","last_peered":"2024-05-20T22:08:00.092802+0000","last_clean":"2024-05-20T22:08:00.092802+0000","last_became_active":"2024-05-20T22:07:53.669565+0000","last_became_peered":"2024-05-20T22:07:53.669565+0000","last_unstale":"2024-05-20T22:08:00.092802+0000","last_undegraded":"2024-05-20T22:08:00.092802+0000","last_fullsized":"2024-05-20T22:08:00.092802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092993+0000","last_change":"2024-05-20T22:07:53.667290+0000","last_active":"2024-05-20T22:08:00.092993+0000","last_peered":"2024-05-20T22:08:00.092993+0000","last_clean":"2024-05-20T22:08:00.092993+0000","last_became_active":"2024-05-20T22:07:53.666953+0000","last_became_peered":"2024-05-20T22:07:53.666953+0000","last_unstale":"2024-05-20T22:08:00.092993+0000","last_undegraded":"2024-05-20T22:08:00.092993+0000","last_fullsized":"2024-05-20T22:08:00.092993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:48.605408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093536+0000","last_change":"2024-05-20T22:07:53.673713+0000","last_active":"2024-05-20T22:08:00.093536+0000","last_peered":"2024-05-20T22:08:00.093536+0000","last_clean":"2024-05-20T22:08:00.093536+0000","last_became_active":"2024-05-20T22:07:53.673419+0000","last_became_peered":"2024-05-20T22:07:53.673419+0000","last_unstale":"2024-05-20T22:08:00.093536+0000","last_undegraded":"2024-05-20T22:08:00.093536+0000","last_fullsized":"2024-05-20T22:08:00.093536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580805+0000","last_change":"2024-05-20T22:07:53.657177+0000","last_active":"2024-05-20T22:07:59.580805+0000","last_peered":"2024-05-20T22:07:59.580805+0000","last_clean":"2024-05-20T22:07:59.580805+0000","last_became_active":"2024-05-20T22:07:53.656847+0000","last_became_peered":"2024-05-20T22:07:53.656847+0000","last_unstale":"2024-05-20T22:07:59.580805+0000","last_undegraded":"2024-05-20T22:07:59.580805+0000","last_fullsized":"2024-05-20T22:07:59.580805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706361+0000","last_change":"2024-05-20T22:07:53.675562+0000","last_active":"2024-05-20T22:07:58.706361+0000","last_peered":"2024-05-20T22:07:58.706361+0000","last_clean":"2024-05-20T22:07:58.706361+0000","last_became_active":"2024-05-20T22:07:53.673333+0000","last_became_peered":"2024-05-20T22:07:53.673333+0000","last_unstale":"2024-05-20T22:07:58.706361+0000","last_undegraded":"2024-05-20T22:07:58.706361+0000","last_fullsized":"2024-05-20T22:07:58.706361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706689+0000","last_change":"2024-05-20T22:07:53.664701+0000","last_active":"2024-05-20T22:07:58.706689+0000","last_peered":"2024-05-20T22:07:58.706689+0000","last_clean":"2024-05-20T22:07:58.706689+0000","last_became_active":"2024-05-20T22:07:53.663423+0000","last_became_peered":"2024-05-20T22:07:53.663423+0000","last_unstale":"2024-05-20T22:07:58.706689+0000","last_undegraded":"2024-05-20T22:07:58.706689+0000","last_fullsized":"2024-05-20T22:07:58.706689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706441+0000","last_change":"2024-05-20T22:07:53.677915+0000","last_active":"2024-05-20T22:07:58.706441+0000","last_peered":"2024-05-20T22:07:58.706441+0000","last_clean":"2024-05-20T22:07:58.706441+0000","last_became_active":"2024-05-20T22:07:53.677479+0000","last_became_peered":"2024-05-20T22:07:53.677479+0000","last_unstale":"2024-05-20T22:07:58.706441+0000","last_undegraded":"2024-05-20T22:07:58.706441+0000","last_fullsized":"2024-05-20T22:07:58.706441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:41:42.167135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092947+0000","last_change":"2024-05-20T22:07:53.674336+0000","last_active":"2024-05-20T22:08:00.092947+0000","last_peered":"2024-05-20T22:08:00.092947+0000","last_clean":"2024-05-20T22:08:00.092947+0000","last_became_active":"2024-05-20T22:07:53.673643+0000","last_became_peered":"2024-05-20T22:07:53.673643+0000","last_unstale":"2024-05-20T22:08:00.092947+0000","last_undegraded":"2024-05-20T22:08:00.092947+0000","last_fullsized":"2024-05-20T22:08:00.092947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:26.916927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092025+0000","last_change":"2024-05-20T22:07:53.680694+0000","last_active":"2024-05-20T22:08:00.092025+0000","last_peered":"2024-05-20T22:08:00.092025+0000","last_clean":"2024-05-20T22:08:00.092025+0000","last_became_active":"2024-05-20T22:07:53.680338+0000","last_became_peered":"2024-05-20T22:07:53.680338+0000","last_unstale":"2024-05-20T22:08:00.092025+0000","last_undegraded":"2024-05-20T22:08:00.092025+0000","last_fullsized":"2024-05-20T22:08:00.092025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092124+0000","last_change":"2024-05-20T22:07:53.668909+0000","last_active":"2024-05-20T22:08:00.092124+0000","last_peered":"2024-05-20T22:08:00.092124+0000","last_clean":"2024-05-20T22:08:00.092124+0000","last_became_active":"2024-05-20T22:07:53.668540+0000","last_became_peered":"2024-05-20T22:07:53.668540+0000","last_unstale":"2024-05-20T22:08:00.092124+0000","last_undegraded":"2024-05-20T22:08:00.092124+0000","last_fullsized":"2024-05-20T22:08:00.092124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580610+0000","last_change":"2024-05-20T22:07:53.671760+0000","last_active":"2024-05-20T22:07:59.580610+0000","last_peered":"2024-05-20T22:07:59.580610+0000","last_clean":"2024-05-20T22:07:59.580610+0000","last_became_active":"2024-05-20T22:07:53.671454+0000","last_became_peered":"2024-05-20T22:07:53.671454+0000","last_unstale":"2024-05-20T22:07:59.580610+0000","last_undegraded":"2024-05-20T22:07:59.580610+0000","last_fullsized":"2024-05-20T22:07:59.580610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:56.576260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092384+0000","last_change":"2024-05-20T22:07:53.678862+0000","last_active":"2024-05-20T22:08:00.092384+0000","last_peered":"2024-05-20T22:08:00.092384+0000","last_clean":"2024-05-20T22:08:00.092384+0000","last_became_active":"2024-05-20T22:07:53.678323+0000","last_became_peered":"2024-05-20T22:07:53.678323+0000","last_unstale":"2024-05-20T22:08:00.092384+0000","last_undegraded":"2024-05-20T22:08:00.092384+0000","last_fullsized":"2024-05-20T22:08:00.092384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:39.545331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093008+0000","last_change":"2024-05-20T22:07:53.677127+0000","last_active":"2024-05-20T22:08:00.093008+0000","last_peered":"2024-05-20T22:08:00.093008+0000","last_clean":"2024-05-20T22:08:00.093008+0000","last_became_active":"2024-05-20T22:07:53.676804+0000","last_became_peered":"2024-05-20T22:07:53.676804+0000","last_unstale":"2024-05-20T22:08:00.093008+0000","last_undegraded":"2024-05-20T22:08:00.093008+0000","last_fullsized":"2024-05-20T22:08:00.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:42.524695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706507+0000","last_change":"2024-05-20T22:07:53.677487+0000","last_active":"2024-05-20T22:07:58.706507+0000","last_peered":"2024-05-20T22:07:58.706507+0000","last_clean":"2024-05-20T22:07:58.706507+0000","last_became_active":"2024-05-20T22:07:53.677160+0000","last_became_peered":"2024-05-20T22:07:53.677160+0000","last_unstale":"2024-05-20T22:07:58.706507+0000","last_undegraded":"2024-05-20T22:07:58.706507+0000","last_fullsized":"2024-05-20T22:07:58.706507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:53:42.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.706545+0000","last_change":"2024-05-20T22:07:53.675130+0000","last_active":"2024-05-20T22:07:58.706545+0000","last_peered":"2024-05-20T22:07:58.706545+0000","last_clean":"2024-05-20T22:07:58.706545+0000","last_became_active":"2024-05-20T22:07:53.674566+0000","last_became_peered":"2024-05-20T22:07:53.674566+0000","last_unstale":"2024-05-20T22:07:58.706545+0000","last_undegraded":"2024-05-20T22:07:58.706545+0000","last_fullsized":"2024-05-20T22:07:58.706545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707188+0000","last_change":"2024-05-20T22:07:53.668737+0000","last_active":"2024-05-20T22:07:58.707188+0000","last_peered":"2024-05-20T22:07:58.707188+0000","last_clean":"2024-05-20T22:07:58.707188+0000","last_became_active":"2024-05-20T22:07:53.668436+0000","last_became_peered":"2024-05-20T22:07:53.668436+0000","last_unstale":"2024-05-20T22:07:58.707188+0000","last_undegraded":"2024-05-20T22:07:58.707188+0000","last_fullsized":"2024-05-20T22:07:58.707188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:26.916927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092105+0000","last_change":"2024-05-20T22:07:53.667651+0000","last_active":"2024-05-20T22:08:00.092105+0000","last_peered":"2024-05-20T22:08:00.092105+0000","last_clean":"2024-05-20T22:08:00.092105+0000","last_became_active":"2024-05-20T22:07:53.667290+0000","last_became_peered":"2024-05-20T22:07:53.667290+0000","last_unstale":"2024-05-20T22:08:00.092105+0000","last_undegraded":"2024-05-20T22:08:00.092105+0000","last_fullsized":"2024-05-20T22:08:00.092105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:40.553326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705225+0000","last_change":"2024-05-20T22:07:53.655295+0000","last_active":"2024-05-20T22:07:58.705225+0000","last_peered":"2024-05-20T22:07:58.705225+0000","last_clean":"2024-05-20T22:07:58.705225+0000","last_became_active":"2024-05-20T22:07:53.654945+0000","last_became_peered":"2024-05-20T22:07:53.654945+0000","last_unstale":"2024-05-20T22:07:58.705225+0000","last_undegraded":"2024-05-20T22:07:58.705225+0000","last_fullsized":"2024-05-20T22:07:58.705225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:42.757306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.093460+0000","last_change":"2024-05-20T22:07:53.681737+0000","last_active":"2024-05-20T22:08:00.093460+0000","last_peered":"2024-05-20T22:08:00.093460+0000","last_clean":"2024-05-20T22:08:00.093460+0000","last_became_active":"2024-05-20T22:07:53.681436+0000","last_became_peered":"2024-05-20T22:07:53.681436+0000","last_unstale":"2024-05-20T22:08:00.093460+0000","last_undegraded":"2024-05-20T22:08:00.093460+0000","last_fullsized":"2024-05-20T22:08:00.093460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:31:13.197083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.705391+0000","last_change":"2024-05-20T22:07:53.662939+0000","last_active":"2024-05-20T22:07:58.705391+0000","last_peered":"2024-05-20T22:07:58.705391+0000","last_clean":"2024-05-20T22:07:58.705391+0000","last_became_active":"2024-05-20T22:07:53.662545+0000","last_became_peered":"2024-05-20T22:07:53.662545+0000","last_unstale":"2024-05-20T22:07:58.705391+0000","last_undegraded":"2024-05-20T22:07:58.705391+0000","last_fullsized":"2024-05-20T22:07:58.705391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:59:56.048396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092568+0000","last_change":"2024-05-20T22:07:53.677132+0000","last_active":"2024-05-20T22:08:00.092568+0000","last_peered":"2024-05-20T22:08:00.092568+0000","last_clean":"2024-05-20T22:08:00.092568+0000","last_became_active":"2024-05-20T22:07:53.676804+0000","last_became_peered":"2024-05-20T22:07:53.676804+0000","last_unstale":"2024-05-20T22:08:00.092568+0000","last_undegraded":"2024-05-20T22:08:00.092568+0000","last_fullsized":"2024-05-20T22:08:00.092568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:27:12.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:59.580901+0000","last_change":"2024-05-20T22:07:53.669468+0000","last_active":"2024-05-20T22:07:59.580901+0000","last_peered":"2024-05-20T22:07:59.580901+0000","last_clean":"2024-05-20T22:07:59.580901+0000","last_became_active":"2024-05-20T22:07:53.669074+0000","last_became_peered":"2024-05-20T22:07:53.669074+0000","last_unstale":"2024-05-20T22:07:59.580901+0000","last_undegraded":"2024-05-20T22:07:59.580901+0000","last_fullsized":"2024-05-20T22:07:59.580901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:26.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:08:00.092451+0000","last_change":"2024-05-20T22:07:53.680851+0000","last_active":"2024-05-20T22:08:00.092451+0000","last_peered":"2024-05-20T22:08:00.092451+0000","last_clean":"2024-05-20T22:08:00.092451+0000","last_became_active":"2024-05-20T22:07:53.680468+0000","last_became_peered":"2024-05-20T22:07:53.680468+0000","last_unstale":"2024-05-20T22:08:00.092451+0000","last_undegraded":"2024-05-20T22:08:00.092451+0000","last_fullsized":"2024-05-20T22:08:00.092451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:57.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-20T22:07:58.707096+0000","last_change":"2024-05-20T22:07:53.675836+0000","last_active":"2024-05-20T22:07:58.707096+0000","last_peered":"2024-05-20T22:07:58.707096+0000","last_clean":"2024-05-20T22:07:58.707096+0000","last_became_active":"2024-05-20T22:07:53.675544+0000","last_became_peered":"2024-05-20T22:07:53.675544+0000","last_unstale":"2024-05-20T22:07:58.707096+0000","last_undegraded":"2024-05-20T22:07:58.707096+0000","last_fullsized":"2024-05-20T22:07:58.707096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:52.633778+0000","last_clean_scrub_stamp":"2024-05-20T22:07:52.633778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:42.044571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:08:17.934 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-20T22:08:17.934 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-20T22:08:17.934 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-20T22:08:17.934 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-20T22:08:18.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.081+0000 7f7553232640 1 Processor -- start 2024-05-20T22:08:18.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.082+0000 7f7553232640 1 -- start start 2024-05-20T22:08:18.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f7553232640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754c0d3390 0x7f754c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f7553232640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f754c0b3c90 0x7f754c0b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f7553232640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f754c0b8550 0x7f754c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f7553232640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f754c06ce50 con 0x7f754c0b8550 2024-05-20T22:08:18.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f7553232640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f754c06b870 con 0x7f754c0b3c90 2024-05-20T22:08:18.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f7553232640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f754c06bb50 con 0x7f754c0d3390 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f75517cf640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f754c0b8550 0x7f754c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f75517cf640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f754c0b8550 0x7f754c15e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58340/0 (socket says 172.21.15.55:58340) 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f75517cf640 1 -- 172.21.15.55:0/2733880493 learned_addr learned my addr 172.21.15.55:0/2733880493 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f754bfff640 1 --2- 172.21.15.55:0/2733880493 >> v2:172.21.15.138:3300/0 conn(0x7f754c0b3c90 0x7f754c0b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f7550fce640 1 --2- 172.21.15.55:0/2733880493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754c0d3390 0x7f754c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f75517cf640 1 -- 172.21.15.55:0/2733880493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754c0d3390 msgr2=0x7f754c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f75517cf640 1 --2- 172.21.15.55:0/2733880493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754c0d3390 0x7f754c06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f75517cf640 1 -- 172.21.15.55:0/2733880493 >> v2:172.21.15.138:3300/0 conn(0x7f754c0b3c90 msgr2=0x7f754c0b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f75517cf640 1 --2- 172.21.15.55:0/2733880493 >> v2:172.21.15.138:3300/0 conn(0x7f754c0b3c90 0x7f754c0b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f75517cf640 1 -- 172.21.15.55:0/2733880493 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c06be30 con 0x7f754c0b8550 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.083+0000 7f754bfff640 1 --2- 172.21.15.55:0/2733880493 >> v2:172.21.15.138:3300/0 conn(0x7f754c0b3c90 0x7f754c0b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.084+0000 7f75517cf640 1 --2- 172.21.15.55:0/2733880493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0b8550 0x7f754c15e750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f754c08d3a0 tx=0x7f753402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e8549c4e77a38f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.084+0000 7f754b7fe640 1 -- 172.21.15.55:0/2733880493 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534037070 con 0x7f754c0b8550 2024-05-20T22:08:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.084+0000 7f754b7fe640 1 -- 172.21.15.55:0/2733880493 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7534032440 con 0x7f754c0b8550 2024-05-20T22:08:18.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.084+0000 7f754b7fe640 1 -- 172.21.15.55:0/2733880493 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534032810 con 0x7f754c0b8550 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.085+0000 7f7553232640 1 -- 172.21.15.55:0/2733880493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0b8550 msgr2=0x7f754c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.085+0000 7f7553232640 1 --2- 172.21.15.55:0/2733880493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0b8550 0x7f754c15e750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f754c08d3a0 tx=0x7f753402a140 comp rx=0 tx=0).stop 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.085+0000 7f7553232640 1 -- 172.21.15.55:0/2733880493 shutdown_connections 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.085+0000 7f7553232640 1 --2- 172.21.15.55:0/2733880493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0b8550 0x7f754c15e750 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.085+0000 7f7553232640 1 --2- 172.21.15.55:0/2733880493 >> v2:172.21.15.138:3300/0 conn(0x7f754c0b3c90 0x7f754c0b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.085+0000 7f7553232640 1 --2- 172.21.15.55:0/2733880493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754c0d3390 0x7f754c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.085+0000 7f7553232640 1 -- 172.21.15.55:0/2733880493 >> 172.21.15.55:0/2733880493 conn(0x7f754c0b65f0 msgr2=0x7f754c0b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.085+0000 7f7553232640 1 -- 172.21.15.55:0/2733880493 shutdown_connections 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.085+0000 7f7553232640 1 -- 172.21.15.55:0/2733880493 wait complete. 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.087+0000 7f7553232640 1 Processor -- start 2024-05-20T22:08:18.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.087+0000 7f7553232640 1 -- start start 2024-05-20T22:08:18.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.088+0000 7f7553232640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0b3c90 0x7f754c14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.088+0000 7f7553232640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754c0b8550 0x7f754c15a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.088+0000 7f7553232640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0d3390 0x7f754c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.088+0000 7f7553232640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f754c06b500 con 0x7f754c0d3390 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.088+0000 7f7553232640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f754c06c3f0 con 0x7f754c0b3c90 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.088+0000 7f7553232640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f754c0b40a0 con 0x7f754c0b8550 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75517cf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0d3390 0x7f754c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75517cf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0d3390 0x7f754c15a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58350/0 (socket says 172.21.15.55:58350) 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75517cf640 1 -- 172.21.15.55:0/3149808825 learned_addr learned my addr 172.21.15.55:0/3149808825 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f754bfff640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754c0b8550 0x7f754c15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f7550fce640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0b3c90 0x7f754c14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75517cf640 1 -- 172.21.15.55:0/3149808825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754c0b8550 msgr2=0x7f754c15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75517cf640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754c0b8550 0x7f754c15a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75517cf640 1 -- 172.21.15.55:0/3149808825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0b3c90 msgr2=0x7f754c14e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75517cf640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0b3c90 0x7f754c14e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75517cf640 1 -- 172.21.15.55:0/3149808825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c14eb90 con 0x7f754c0d3390 2024-05-20T22:08:18.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75517cf640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0d3390 0x7f754c15a870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f75340090b0 tx=0x7f75340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:18.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75497fa640 1 -- 172.21.15.55:0/3149808825 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534037040 con 0x7f754c0d3390 2024-05-20T22:08:18.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75497fa640 1 -- 172.21.15.55:0/3149808825 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7534042070 con 0x7f754c0d3390 2024-05-20T22:08:18.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f7553232640 1 -- 172.21.15.55:0/3149808825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c1545b0 con 0x7f754c0d3390 2024-05-20T22:08:18.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f75497fa640 1 -- 172.21.15.55:0/3149808825 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534030920 con 0x7f754c0d3390 2024-05-20T22:08:18.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.089+0000 7f7553232640 1 -- 172.21.15.55:0/3149808825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c154b70 con 0x7f754c0d3390 2024-05-20T22:08:18.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.090+0000 7f7553232640 1 -- 172.21.15.55:0/3149808825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c0b6a00 con 0x7f754c0d3390 2024-05-20T22:08:18.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.091+0000 7f75497fa640 1 -- 172.21.15.55:0/3149808825 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7534030ac0 con 0x7f754c0d3390 2024-05-20T22:08:18.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.091+0000 7f75497fa640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f752c06be30 0x7f752c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.092+0000 7f75497fa640 1 -- 172.21.15.55:0/3149808825 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f75340ab9c0 con 0x7f754c0d3390 2024-05-20T22:08:18.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.092+0000 7f7550fce640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f752c06be30 0x7f752c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.092+0000 7f7550fce640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f752c06be30 0x7f752c06e340 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f754c08d370 tx=0x7f7544008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:18.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.096+0000 7f75497fa640 1 -- 172.21.15.55:0/3149808825 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f754c0b6a00 con 0x7f754c0d3390 2024-05-20T22:08:18.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.318+0000 7f7553232640 1 -- 172.21.15.55:0/3149808825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f754c154340 con 0x7f754c0d3390 2024-05-20T22:08:18.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.319+0000 7f75497fa640 1 -- 172.21.15.55:0/3149808825 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f75340ade20 con 0x7f754c0d3390 2024-05-20T22:08:18.320 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:18.320 INFO:teuthology.orchestra.run.smithi055.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-20T22:08:18.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 -- 172.21.15.55:0/3149808825 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f752c06be30 msgr2=0x7f752c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f752c06be30 0x7f752c06e340 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f754c08d370 tx=0x7f7544008040 comp rx=0 tx=0).stop 2024-05-20T22:08:18.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 -- 172.21.15.55:0/3149808825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0d3390 msgr2=0x7f754c15a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0d3390 0x7f754c15a870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f75340090b0 tx=0x7f75340046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 -- 172.21.15.55:0/3149808825 shutdown_connections 2024-05-20T22:08:18.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f752c06be30 0x7f752c06e340 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f754c0d3390 0x7f754c15a870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754c0b8550 0x7f754c15a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 --2- 172.21.15.55:0/3149808825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c0b3c90 0x7f754c14e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 -- 172.21.15.55:0/3149808825 >> 172.21.15.55:0/3149808825 conn(0x7f754c0b65f0 msgr2=0x7f754c0b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:18.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 -- 172.21.15.55:0/3149808825 shutdown_connections 2024-05-20T22:08:18.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.321+0000 7f7553232640 1 -- 172.21.15.55:0/3149808825 wait complete. 2024-05-20T22:08:18.333 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-20T22:08:18.333 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:18.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.447+0000 7efef594c640 1 Processor -- start 2024-05-20T22:08:18.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.447+0000 7efef594c640 1 -- start start 2024-05-20T22:08:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.448+0000 7efef594c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef006d8f0 0x7efef006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.448+0000 7efef594c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efef0163510 0x7efef0163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.448+0000 7efef594c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef006e370 con 0x7efef0163e60 2024-05-20T22:08:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.448+0000 7efef594c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efef00d2000 con 0x7efef0163510 2024-05-20T22:08:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.448+0000 7efef594c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efef00b7e20 con 0x7efef006d8f0 2024-05-20T22:08:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.449+0000 7efeeeffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef006d8f0 0x7efef006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.449+0000 7efeee7fc640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efef0163510 0x7efef0163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.449+0000 7efeeeffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef006d8f0 0x7efef006dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47562/0 (socket says 172.21.15.55:47562) 2024-05-20T22:08:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.449+0000 7efeeeffd640 1 -- 172.21.15.55:0/310882214 learned_addr learned my addr 172.21.15.55:0/310882214 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.449+0000 7efeeeffd640 1 -- 172.21.15.55:0/310882214 >> v2:172.21.15.138:3300/0 conn(0x7efef0163510 msgr2=0x7efef0163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.449+0000 7efeeeffd640 1 --2- 172.21.15.55:0/310882214 >> v2:172.21.15.138:3300/0 conn(0x7efef0163510 0x7efef0163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.449+0000 7efeeeffd640 1 -- 172.21.15.55:0/310882214 >> v1:172.21.15.138:6789/0 conn(0x7efef0163e60 legacy=0x7efef00b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.449+0000 7efeeeffd640 1 -- 172.21.15.55:0/310882214 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef0164270 con 0x7efef006d8f0 2024-05-20T22:08:18.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.450+0000 7efeeeffd640 1 --2- 172.21.15.55:0/310882214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef006d8f0 0x7efef006dd00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efee0009850 tx=0x7efee002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e3b3cbf8d3736c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:18.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.450+0000 7efeedffb640 1 -- 172.21.15.55:0/310882214 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0004030 con 0x7efef006d8f0 2024-05-20T22:08:18.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.450+0000 7efeedffb640 1 -- 172.21.15.55:0/310882214 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee00041d0 con 0x7efef006d8f0 2024-05-20T22:08:18.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.450+0000 7efeedffb640 1 -- 172.21.15.55:0/310882214 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0003610 con 0x7efef006d8f0 2024-05-20T22:08:18.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.451+0000 7efef594c640 1 -- 172.21.15.55:0/310882214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef006d8f0 msgr2=0x7efef006dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.451+0000 7efef594c640 1 --2- 172.21.15.55:0/310882214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef006d8f0 0x7efef006dd00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efee0009850 tx=0x7efee002a1a0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.451+0000 7efef594c640 1 -- 172.21.15.55:0/310882214 shutdown_connections 2024-05-20T22:08:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.451+0000 7efef594c640 1 --2- 172.21.15.55:0/310882214 >> v2:172.21.15.138:3300/0 conn(0x7efef0163510 0x7efef0163920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.451+0000 7efef594c640 1 --2- 172.21.15.55:0/310882214 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef006d8f0 0x7efef006dd00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.451+0000 7efef594c640 1 -- 172.21.15.55:0/310882214 >> 172.21.15.55:0/310882214 conn(0x7efef00b4340 msgr2=0x7efef00b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.451+0000 7efef594c640 1 -- 172.21.15.55:0/310882214 shutdown_connections 2024-05-20T22:08:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.451+0000 7efef594c640 1 -- 172.21.15.55:0/310882214 wait complete. 2024-05-20T22:08:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.453+0000 7efef594c640 1 Processor -- start 2024-05-20T22:08:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.453+0000 7efef594c640 1 -- start start 2024-05-20T22:08:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efef594c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef006d8f0 0x7efef009f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efef594c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0163510 0x7efef00b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efef594c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef0163e60 0x7efef0157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efef594c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efef009c1e0 con 0x7efef006d8f0 2024-05-20T22:08:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efef594c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efef009c060 con 0x7efef0163510 2024-05-20T22:08:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efef594c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efef006d1c0 con 0x7efef0163e60 2024-05-20T22:08:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efeeeffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef006d8f0 0x7efef009f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efeeeffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef006d8f0 0x7efef009f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58360/0 (socket says 172.21.15.55:58360) 2024-05-20T22:08:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efeeeffd640 1 -- 172.21.15.55:0/2544635015 learned_addr learned my addr 172.21.15.55:0/2544635015 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efeee7fc640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0163510 0x7efef00b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.454+0000 7efeef7fe640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef0163e60 0x7efef0157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efeeeffd640 1 -- 172.21.15.55:0/2544635015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef0163e60 msgr2=0x7efef0157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efeeeffd640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef0163e60 0x7efef0157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efeeeffd640 1 -- 172.21.15.55:0/2544635015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0163510 msgr2=0x7efef00b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efeeeffd640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0163510 0x7efef00b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efeeeffd640 1 -- 172.21.15.55:0/2544635015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef009fb20 con 0x7efef006d8f0 2024-05-20T22:08:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efeef7fe640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef0163e60 0x7efef0157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efeeeffd640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef006d8f0 0x7efef009f5e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efee002a1e0 tx=0x7efee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efee7fff640 1 -- 172.21.15.55:0/2544635015 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0041020 con 0x7efef006d8f0 2024-05-20T22:08:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efee7fff640 1 -- 172.21.15.55:0/2544635015 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee0003c50 con 0x7efef006d8f0 2024-05-20T22:08:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efef594c640 1 -- 172.21.15.55:0/2544635015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef00a0030 con 0x7efef006d8f0 2024-05-20T22:08:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efee7fff640 1 -- 172.21.15.55:0/2544635015 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0004030 con 0x7efef006d8f0 2024-05-20T22:08:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.455+0000 7efef594c640 1 -- 172.21.15.55:0/2544635015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef00c9a30 con 0x7efef006d8f0 2024-05-20T22:08:18.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.456+0000 7efef594c640 1 -- 172.21.15.55:0/2544635015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef00cb390 con 0x7efef006d8f0 2024-05-20T22:08:18.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.457+0000 7efee7fff640 1 -- 172.21.15.55:0/2544635015 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efee0044020 con 0x7efef006d8f0 2024-05-20T22:08:18.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.457+0000 7efee7fff640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efed006be60 0x7efed006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.458+0000 7efee7fff640 1 -- 172.21.15.55:0/2544635015 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7efee00abcf0 con 0x7efef006d8f0 2024-05-20T22:08:18.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.458+0000 7efeee7fc640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efed006be60 0x7efed006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.458+0000 7efeee7fc640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efed006be60 0x7efed006e370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efef00b3530 tx=0x7efed8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.462+0000 7efee7fff640 1 -- 172.21.15.55:0/2544635015 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efef00cb390 con 0x7efef006d8f0 2024-05-20T22:08:18.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.648+0000 7efef594c640 1 -- 172.21.15.55:0/2544635015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efef0069090 con 0x7efef006d8f0 2024-05-20T22:08:18.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.649+0000 7efee7fff640 1 -- 172.21.15.55:0/2544635015 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7efee004b030 con 0x7efef006d8f0 2024-05-20T22:08:18.650 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:18.651 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.651+0000 7efef594c640 1 -- 172.21.15.55:0/2544635015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efed006be60 msgr2=0x7efed006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.651+0000 7efef594c640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efed006be60 0x7efed006e370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efef00b3530 tx=0x7efed8009190 comp rx=0 tx=0).stop 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.651+0000 7efef594c640 1 -- 172.21.15.55:0/2544635015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef006d8f0 msgr2=0x7efef009f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.651+0000 7efef594c640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef006d8f0 0x7efef009f5e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efee002a1e0 tx=0x7efee00046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.652+0000 7efef594c640 1 -- 172.21.15.55:0/2544635015 shutdown_connections 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.652+0000 7efef594c640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efed006be60 0x7efed006e370 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.652+0000 7efef594c640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efef0163e60 0x7efef0157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.652+0000 7efef594c640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0163510 0x7efef00b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.652+0000 7efef594c640 1 --2- 172.21.15.55:0/2544635015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef006d8f0 0x7efef009f5e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.652+0000 7efef594c640 1 -- 172.21.15.55:0/2544635015 >> 172.21.15.55:0/2544635015 conn(0x7efef00b4340 msgr2=0x7efef0070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.652+0000 7efef594c640 1 -- 172.21.15.55:0/2544635015 shutdown_connections 2024-05-20T22:08:18.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.652+0000 7efef594c640 1 -- 172.21.15.55:0/2544635015 wait complete. 2024-05-20T22:08:18.665 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:18.665 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:18.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.781+0000 7f100d365640 1 Processor -- start 2024-05-20T22:08:18.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.781+0000 7f100d365640 1 -- start start 2024-05-20T22:08:18.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.782+0000 7f100d365640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10080b4080 0x7f100815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f100d365640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080d3290 0x7f10080d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f100d365640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f100806b500 con 0x7f10080b35e0 2024-05-20T22:08:18.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f100d365640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1008163960 con 0x7f10080b4080 2024-05-20T22:08:18.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f100d365640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f10080b3e10 con 0x7f10080d3290 2024-05-20T22:08:18.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f10077fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080d3290 0x7f10080d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f1006ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10080b4080 0x7f100815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f1006ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10080b4080 0x7f100815f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:53100/0 (socket says 172.21.15.55:53100) 2024-05-20T22:08:18.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f10077fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080d3290 0x7f10080d36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:48436/0 (socket says 172.21.15.55:48436) 2024-05-20T22:08:18.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f1006ffd640 1 -- 172.21.15.55:0/1368943593 learned_addr learned my addr 172.21.15.55:0/1368943593 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:18.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f1006ffd640 1 -- 172.21.15.55:0/1368943593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080d3290 msgr2=0x7f10080d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:18.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f1006ffd640 1 --2- 172.21.15.55:0/1368943593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080d3290 0x7f10080d36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f1006ffd640 1 -- 172.21.15.55:0/1368943593 >> v1:172.21.15.138:6789/0 conn(0x7f10080b35e0 legacy=0x7f100815f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f1005ffb640 1 -- 172.21.15.55:0/1368943593 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3246966924 0 0) 0x7f0ffc001110 con 0x7f10080b35e0 2024-05-20T22:08:18.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f1006ffd640 1 -- 172.21.15.55:0/1368943593 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10080b3b30 con 0x7f10080b4080 2024-05-20T22:08:18.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.783+0000 7f10077fe640 1 --2- 172.21.15.55:0/1368943593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080d3290 0x7f10080d36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:18.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.784+0000 7f1006ffd640 1 --2- 172.21.15.55:0/1368943593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10080b4080 0x7f100815f2e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009870 tx=0x7f0ff802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e50ea2c2b38d71d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:18.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.784+0000 7f1005ffb640 1 -- 172.21.15.55:0/1368943593 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8037070 con 0x7f10080b4080 2024-05-20T22:08:18.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.784+0000 7f1005ffb640 1 -- 172.21.15.55:0/1368943593 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff8032440 con 0x7f10080b4080 2024-05-20T22:08:18.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.784+0000 7f1005ffb640 1 -- 172.21.15.55:0/1368943593 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8032810 con 0x7f10080b4080 2024-05-20T22:08:18.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.785+0000 7f100d365640 1 -- 172.21.15.55:0/1368943593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10080b4080 msgr2=0x7f100815f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.785+0000 7f100d365640 1 --2- 172.21.15.55:0/1368943593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10080b4080 0x7f100815f2e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009870 tx=0x7f0ff802a250 comp rx=0 tx=0).stop 2024-05-20T22:08:18.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.785+0000 7f100d365640 1 -- 172.21.15.55:0/1368943593 shutdown_connections 2024-05-20T22:08:18.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.785+0000 7f100d365640 1 --2- 172.21.15.55:0/1368943593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080d3290 0x7f10080d36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.785+0000 7f100d365640 1 --2- 172.21.15.55:0/1368943593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10080b4080 0x7f100815f2e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.785+0000 7f100d365640 1 -- 172.21.15.55:0/1368943593 >> 172.21.15.55:0/1368943593 conn(0x7f10080d51b0 msgr2=0x7f10080b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:18.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.785+0000 7f100d365640 1 -- 172.21.15.55:0/1368943593 shutdown_connections 2024-05-20T22:08:18.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.785+0000 7f100d365640 1 -- 172.21.15.55:0/1368943593 wait complete. 2024-05-20T22:08:18.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.787+0000 7f100d365640 1 Processor -- start 2024-05-20T22:08:18.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.787+0000 7f100d365640 1 -- start start 2024-05-20T22:08:18.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f100d365640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080b35e0 0x7f10080b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f100d365640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10080b4080 0x7f10081f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f100d365640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10080d3290 0x7f10080b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f100d365640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f10081635f0 con 0x7f10080b4080 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f100d365640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10081604f0 con 0x7f10080d3290 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f100d365640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f100815ff30 con 0x7f10080b35e0 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f10067fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10080b4080 0x7f10081f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f10067fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10080b4080 0x7f10081f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35208/0 (socket says 172.21.15.55:35208) 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f10067fc640 1 -- 172.21.15.55:0/2313090979 learned_addr learned my addr 172.21.15.55:0/2313090979 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f1006ffd640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080b35e0 0x7f10080b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f10077fe640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10080d3290 0x7f10080b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f10067fc640 1 -- 172.21.15.55:0/2313090979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080b35e0 msgr2=0x7f10080b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f10067fc640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080b35e0 0x7f10080b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f10067fc640 1 -- 172.21.15.55:0/2313090979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10080d3290 msgr2=0x7f10080b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f10067fc640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10080d3290 0x7f10080b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f10067fc640 1 -- 172.21.15.55:0/2313090979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10081f6440 con 0x7f10080b4080 2024-05-20T22:08:18.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.789+0000 7f10067fc640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10080b4080 0x7f10081f5a10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c410 tx=0x7f0ffc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:18.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.790+0000 7f0fef7fe640 1 -- 172.21.15.55:0/2313090979 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc017070 con 0x7f10080b4080 2024-05-20T22:08:18.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.790+0000 7f0fef7fe640 1 -- 172.21.15.55:0/2313090979 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ffc0035f0 con 0x7f10080b4080 2024-05-20T22:08:18.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.790+0000 7f100d365640 1 -- 172.21.15.55:0/2313090979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10081f6720 con 0x7f10080b4080 2024-05-20T22:08:18.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.790+0000 7f0fef7fe640 1 -- 172.21.15.55:0/2313090979 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc003a20 con 0x7f10080b4080 2024-05-20T22:08:18.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.791+0000 7f100d365640 1 -- 172.21.15.55:0/2313090979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10081f5f50 con 0x7f10080b4080 2024-05-20T22:08:18.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.792+0000 7f100d365640 1 -- 172.21.15.55:0/2313090979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10081f61b0 con 0x7f10080b4080 2024-05-20T22:08:18.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.792+0000 7f0fef7fe640 1 -- 172.21.15.55:0/2313090979 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ffc01a070 con 0x7f10080b4080 2024-05-20T22:08:18.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.793+0000 7f0fef7fe640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0ff406be30 0x7f0ff406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:18.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.793+0000 7f1006ffd640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0ff406be30 0x7f0ff406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:18.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.793+0000 7f0fef7fe640 1 -- 172.21.15.55:0/2313090979 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0ffc08a500 con 0x7f10080b4080 2024-05-20T22:08:18.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.794+0000 7f1006ffd640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0ff406be30 0x7f0ff406e340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8034000 tx=0x7f0ff802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:18.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.800+0000 7f0fef7fe640 1 -- 172.21.15.55:0/2313090979 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f100806b500 con 0x7f10080b4080 2024-05-20T22:08:18.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.983+0000 7f100d365640 1 -- 172.21.15.55:0/2313090979 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1008069090 con 0x7f10080b4080 2024-05-20T22:08:18.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.984+0000 7f0fef7fe640 1 -- 172.21.15.55:0/2313090979 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f0ffc058480 con 0x7f10080b4080 2024-05-20T22:08:18.985 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 -- 172.21.15.55:0/2313090979 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0ff406be30 msgr2=0x7f0ff406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0ff406be30 0x7f0ff406e340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8034000 tx=0x7f0ff802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 -- 172.21.15.55:0/2313090979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10080b4080 msgr2=0x7f10081f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10080b4080 0x7f10081f5a10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c410 tx=0x7f0ffc00c810 comp rx=0 tx=0).stop 2024-05-20T22:08:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 -- 172.21.15.55:0/2313090979 shutdown_connections 2024-05-20T22:08:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0ff406be30 0x7f0ff406e340 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10080d3290 0x7f10080b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10080b4080 0x7f10081f5a10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 --2- 172.21.15.55:0/2313090979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10080b35e0 0x7f10080b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 -- 172.21.15.55:0/2313090979 >> 172.21.15.55:0/2313090979 conn(0x7f10080d51b0 msgr2=0x7f10081616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 -- 172.21.15.55:0/2313090979 shutdown_connections 2024-05-20T22:08:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:18.986+0000 7f100d365640 1 -- 172.21.15.55:0/2313090979 wait complete. 2024-05-20T22:08:18.998 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:19.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.114+0000 7fd5432fe640 1 Processor -- start 2024-05-20T22:08:19.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.114+0000 7fd5432fe640 1 -- start start 2024-05-20T22:08:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd5432fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd53c0cd020 0x7fd53c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd5432fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd53c0d08a0 0x7fd53c0d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd5432fe640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd53c0cc3c0 con 0x7fd53c0cdac0 2024-05-20T22:08:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd5432fe640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fd53c0cc170 con 0x7fd53c0cd020 2024-05-20T22:08:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd5432fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd53c0ccc60 con 0x7fd53c0d08a0 2024-05-20T22:08:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd54109a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd53c0cd020 0x7fd53c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd540899640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd53c0cdac0 0x7fd53c0cded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46788/0 (socket says 172.21.15.55:46788) 2024-05-20T22:08:19.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd540899640 1 -- 172.21.15.55:0/469076057 learned_addr learned my addr 172.21.15.55:0/469076057 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:19.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd54189b640 1 --2- 172.21.15.55:0/469076057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd53c0d08a0 0x7fd53c0d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd54109a640 1 -- 172.21.15.55:0/469076057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd53c0d08a0 msgr2=0x7fd53c0d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:19.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd54109a640 1 --2- 172.21.15.55:0/469076057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd53c0d08a0 0x7fd53c0d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd54109a640 1 -- 172.21.15.55:0/469076057 >> v1:172.21.15.55:6789/0 conn(0x7fd53c0cdac0 legacy=0x7fd53c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd533fff640 1 -- 172.21.15.55:0/469076057 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3111690152 0 0) 0x7fd53c0cc3c0 con 0x7fd53c0cdac0 2024-05-20T22:08:19.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.116+0000 7fd54109a640 1 -- 172.21.15.55:0/469076057 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c0cc6a0 con 0x7fd53c0cd020 2024-05-20T22:08:19.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.117+0000 7fd54109a640 1 --2- 172.21.15.55:0/469076057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cd020 0x7fd53c0cd430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd5380097a0 tx=0x7fd53802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31044d315eea2db server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:19.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.117+0000 7fd533fff640 1 -- 172.21.15.55:0/469076057 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538037070 con 0x7fd53c0cd020 2024-05-20T22:08:19.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.117+0000 7fd533fff640 1 -- 172.21.15.55:0/469076057 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd538032440 con 0x7fd53c0cd020 2024-05-20T22:08:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.117+0000 7fd533fff640 1 -- 172.21.15.55:0/469076057 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538032810 con 0x7fd53c0cd020 2024-05-20T22:08:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.118+0000 7fd5432fe640 1 -- 172.21.15.55:0/469076057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cd020 msgr2=0x7fd53c0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.118+0000 7fd5432fe640 1 --2- 172.21.15.55:0/469076057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cd020 0x7fd53c0cd430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd5380097a0 tx=0x7fd53802a180 comp rx=0 tx=0).stop 2024-05-20T22:08:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.118+0000 7fd5432fe640 1 -- 172.21.15.55:0/469076057 shutdown_connections 2024-05-20T22:08:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.118+0000 7fd5432fe640 1 --2- 172.21.15.55:0/469076057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd53c0d08a0 0x7fd53c0d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.118+0000 7fd5432fe640 1 --2- 172.21.15.55:0/469076057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cd020 0x7fd53c0cd430 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.118+0000 7fd5432fe640 1 -- 172.21.15.55:0/469076057 >> 172.21.15.55:0/469076057 conn(0x7fd53c0b62c0 msgr2=0x7fd53c09a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.118+0000 7fd5432fe640 1 -- 172.21.15.55:0/469076057 shutdown_connections 2024-05-20T22:08:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.118+0000 7fd5432fe640 1 -- 172.21.15.55:0/469076057 wait complete. 2024-05-20T22:08:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.120+0000 7fd5432fe640 1 Processor -- start 2024-05-20T22:08:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.120+0000 7fd5432fe640 1 -- start start 2024-05-20T22:08:19.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd5432fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd53c0cd020 0x7fd53c0c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd5432fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cdac0 0x7fd53c153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd5432fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd53c0d08a0 0x7fd53c156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd5432fe640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c15f4d0 con 0x7fd53c0cdac0 2024-05-20T22:08:19.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd5432fe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c0d0380 con 0x7fd53c0d08a0 2024-05-20T22:08:19.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd5432fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd53c15f650 con 0x7fd53c0cd020 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd540899640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cdac0 0x7fd53c153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd540899640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cdac0 0x7fd53c153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35226/0 (socket says 172.21.15.55:35226) 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd540899640 1 -- 172.21.15.55:0/2921848177 learned_addr learned my addr 172.21.15.55:0/2921848177 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd54109a640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd53c0cd020 0x7fd53c0c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd54189b640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd53c0d08a0 0x7fd53c156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd540899640 1 -- 172.21.15.55:0/2921848177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd53c0cd020 msgr2=0x7fd53c0c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd540899640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd53c0cd020 0x7fd53c0c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd540899640 1 -- 172.21.15.55:0/2921848177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd53c0d08a0 msgr2=0x7fd53c156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd540899640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd53c0d08a0 0x7fd53c156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.121+0000 7fd540899640 1 -- 172.21.15.55:0/2921848177 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c1558d0 con 0x7fd53c0cdac0 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.122+0000 7fd540899640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cdac0 0x7fd53c153640 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00c410 tx=0x7fd52c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.122+0000 7fd531ffb640 1 -- 172.21.15.55:0/2921848177 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0074e0 con 0x7fd53c0cdac0 2024-05-20T22:08:19.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.122+0000 7fd531ffb640 1 -- 172.21.15.55:0/2921848177 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd52c0047e0 con 0x7fd53c0cdac0 2024-05-20T22:08:19.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.122+0000 7fd5432fe640 1 -- 172.21.15.55:0/2921848177 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53c155b70 con 0x7fd53c0cdac0 2024-05-20T22:08:19.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.122+0000 7fd531ffb640 1 -- 172.21.15.55:0/2921848177 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c004bd0 con 0x7fd53c0cdac0 2024-05-20T22:08:19.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.122+0000 7fd5432fe640 1 -- 172.21.15.55:0/2921848177 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53c156130 con 0x7fd53c0cdac0 2024-05-20T22:08:19.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.123+0000 7fd5432fe640 1 -- 172.21.15.55:0/2921848177 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd53c0c91f0 con 0x7fd53c0cdac0 2024-05-20T22:08:19.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.124+0000 7fd531ffb640 1 -- 172.21.15.55:0/2921848177 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd52c011070 con 0x7fd53c0cdac0 2024-05-20T22:08:19.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.124+0000 7fd531ffb640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd52006be60 0x7fd52006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.124+0000 7fd531ffb640 1 -- 172.21.15.55:0/2921848177 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd52c08adb0 con 0x7fd53c0cdac0 2024-05-20T22:08:19.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.124+0000 7fd54109a640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd52006be60 0x7fd52006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.125+0000 7fd54109a640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd52006be60 0x7fd52006e370 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd538034000 tx=0x7fd53802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:19.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.129+0000 7fd531ffb640 1 -- 172.21.15.55:0/2921848177 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd52c00e280 con 0x7fd53c0cdac0 2024-05-20T22:08:19.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.315+0000 7fd5432fe640 1 -- 172.21.15.55:0/2921848177 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd53c069090 con 0x7fd53c0cdac0 2024-05-20T22:08:19.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.315+0000 7fd531ffb640 1 -- 172.21.15.55:0/2921848177 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fd52c059390 con 0x7fd53c0cdac0 2024-05-20T22:08:19.316 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:19.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 -- 172.21.15.55:0/2921848177 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd52006be60 msgr2=0x7fd52006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd52006be60 0x7fd52006e370 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd538034000 tx=0x7fd53802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:19.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 -- 172.21.15.55:0/2921848177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cdac0 msgr2=0x7fd53c153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cdac0 0x7fd53c153640 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00c410 tx=0x7fd52c00c8d0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 -- 172.21.15.55:0/2921848177 shutdown_connections 2024-05-20T22:08:19.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd52006be60 0x7fd52006e370 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd53c0d08a0 0x7fd53c156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd53c0cdac0 0x7fd53c153640 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 --2- 172.21.15.55:0/2921848177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd53c0cd020 0x7fd53c0c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 -- 172.21.15.55:0/2921848177 >> 172.21.15.55:0/2921848177 conn(0x7fd53c0b62c0 msgr2=0x7fd53c0ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:19.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 -- 172.21.15.55:0/2921848177 shutdown_connections 2024-05-20T22:08:19.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.318+0000 7fd5432fe640 1 -- 172.21.15.55:0/2921848177 wait complete. 2024-05-20T22:08:19.330 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.447+0000 7f3fc6baa640 1 Processor -- start 2024-05-20T22:08:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.447+0000 7f3fc6baa640 1 -- start start 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.448+0000 7f3fc6baa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3fc00d32a0 0x7f3fc015f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.448+0000 7f3fc6baa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00cf690 0x7f3fc00cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.448+0000 7f3fc6baa640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3fc00cffe0 0x7f3fc0162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.448+0000 7f3fc6baa640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f3fc00d3780 con 0x7f3fc00cffe0 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc6baa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3fc00d3a60 con 0x7f3fc00d32a0 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc6baa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc00d3d40 con 0x7f3fc00cf690 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc5147640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3fc00cffe0 0x7f3fc0162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc5147640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3fc00cffe0 0x7f3fc0162cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35236/0 (socket says 172.21.15.55:35236) 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc5147640 1 -- 172.21.15.55:0/2032776916 learned_addr learned my addr 172.21.15.55:0/2032776916 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fbffff640 1 --2- 172.21.15.55:0/2032776916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00cf690 0x7f3fc00cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc4946640 1 --2- 172.21.15.55:0/2032776916 >> v2:172.21.15.138:3300/0 conn(0x7f3fc00d32a0 0x7f3fc015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc5147640 1 -- 172.21.15.55:0/2032776916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00cf690 msgr2=0x7f3fc00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc5147640 1 --2- 172.21.15.55:0/2032776916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00cf690 0x7f3fc00cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc5147640 1 -- 172.21.15.55:0/2032776916 >> v2:172.21.15.138:3300/0 conn(0x7f3fc00d32a0 msgr2=0x7f3fc015f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc5147640 1 --2- 172.21.15.55:0/2032776916 >> v2:172.21.15.138:3300/0 conn(0x7f3fc00d32a0 0x7f3fc015f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc5147640 1 -- 172.21.15.55:0/2032776916 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc00d4020 con 0x7f3fc00cffe0 2024-05-20T22:08:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fc4946640 1 --2- 172.21.15.55:0/2032776916 >> v2:172.21.15.138:3300/0 conn(0x7f3fc00d32a0 0x7f3fc015f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.449+0000 7f3fbffff640 1 --2- 172.21.15.55:0/2032776916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00cf690 0x7f3fc00cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.450+0000 7f3fc5147640 1 --2- 172.21.15.55:0/2032776916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cffe0 0x7f3fc0162cd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00d36b0 tx=0x7f3fb802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f2c709dc0b6900 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.450+0000 7f3fbf7fe640 1 -- 172.21.15.55:0/2032776916 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8036070 con 0x7f3fc00cffe0 2024-05-20T22:08:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.450+0000 7f3fbf7fe640 1 -- 172.21.15.55:0/2032776916 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb8031600 con 0x7f3fc00cffe0 2024-05-20T22:08:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.450+0000 7f3fbf7fe640 1 -- 172.21.15.55:0/2032776916 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80319d0 con 0x7f3fc00cffe0 2024-05-20T22:08:19.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.451+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2032776916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cffe0 msgr2=0x7f3fc0162cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.451+0000 7f3fc6baa640 1 --2- 172.21.15.55:0/2032776916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cffe0 0x7f3fc0162cd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00d36b0 tx=0x7f3fb802a6a0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.451+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2032776916 shutdown_connections 2024-05-20T22:08:19.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.451+0000 7f3fc6baa640 1 --2- 172.21.15.55:0/2032776916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cffe0 0x7f3fc0162cd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.451+0000 7f3fc6baa640 1 --2- 172.21.15.55:0/2032776916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00cf690 0x7f3fc00cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.451+0000 7f3fc6baa640 1 --2- 172.21.15.55:0/2032776916 >> v2:172.21.15.138:3300/0 conn(0x7f3fc00d32a0 0x7f3fc015f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.451+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2032776916 >> 172.21.15.55:0/2032776916 conn(0x7f3fc00b6de0 msgr2=0x7f3fc00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:19.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.451+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2032776916 shutdown_connections 2024-05-20T22:08:19.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.451+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2032776916 wait complete. 2024-05-20T22:08:19.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.452+0000 7f3fc6baa640 1 Processor -- start 2024-05-20T22:08:19.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.453+0000 7f3fc6baa640 1 -- start start 2024-05-20T22:08:19.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc6baa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cf690 0x7f3fc00b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc6baa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc00cffe0 0x7f3fc00b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc6baa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00d32a0 0x7f3fc014def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc6baa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc00d4820 con 0x7f3fc00cf690 2024-05-20T22:08:19.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc6baa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc00d03f0 con 0x7f3fc00cffe0 2024-05-20T22:08:19.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc6baa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc014d370 con 0x7f3fc00d32a0 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc4946640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cf690 0x7f3fc00b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc4946640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cf690 0x7f3fc00b93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35252/0 (socket says 172.21.15.55:35252) 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc4946640 1 -- 172.21.15.55:0/2262578798 learned_addr learned my addr 172.21.15.55:0/2262578798 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc5147640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00d32a0 0x7f3fc014def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fbffff640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc00cffe0 0x7f3fc00b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc4946640 1 -- 172.21.15.55:0/2262578798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00d32a0 msgr2=0x7f3fc014def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc4946640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00d32a0 0x7f3fc014def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc4946640 1 -- 172.21.15.55:0/2262578798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc00cffe0 msgr2=0x7f3fc00b98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc4946640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc00cffe0 0x7f3fc00b98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc4946640 1 -- 172.21.15.55:0/2262578798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc0161750 con 0x7f3fc00cf690 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fc5147640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00d32a0 0x7f3fc014def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.454+0000 7f3fbffff640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc00cffe0 0x7f3fc00b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.455+0000 7f3fc4946640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cf690 0x7f3fc00b93a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000d7d0 tx=0x7f3fb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.455+0000 7f3fbd7fa640 1 -- 172.21.15.55:0/2262578798 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0004030 con 0x7f3fc00cf690 2024-05-20T22:08:19.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.455+0000 7f3fbd7fa640 1 -- 172.21.15.55:0/2262578798 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb00041d0 con 0x7f3fc00cf690 2024-05-20T22:08:19.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.455+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2262578798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc01619c0 con 0x7f3fc00cf690 2024-05-20T22:08:19.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.455+0000 7f3fbd7fa640 1 -- 172.21.15.55:0/2262578798 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0009640 con 0x7f3fc00cf690 2024-05-20T22:08:19.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.455+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2262578798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc014bb10 con 0x7f3fc00cf690 2024-05-20T22:08:19.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.456+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2262578798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc0069090 con 0x7f3fc00cf690 2024-05-20T22:08:19.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.456+0000 7f3fbd7fa640 1 -- 172.21.15.55:0/2262578798 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3fb00097e0 con 0x7f3fc00cf690 2024-05-20T22:08:19.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.457+0000 7f3fbd7fa640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3fac06be30 0x7f3fac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.457+0000 7f3fbffff640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3fac06be30 0x7f3fac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.457+0000 7f3fbd7fa640 1 -- 172.21.15.55:0/2262578798 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3fb008b500 con 0x7f3fc00cf690 2024-05-20T22:08:19.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.458+0000 7f3fbffff640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3fac06be30 0x7f3fac06e340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3fc015cc40 tx=0x7f3fb4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:19.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.462+0000 7f3fbd7fa640 1 -- 172.21.15.55:0/2262578798 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3fb0059a40 con 0x7f3fc00cf690 2024-05-20T22:08:19.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.645+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2262578798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3fc00d2e50 con 0x7f3fc00cf690 2024-05-20T22:08:19.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.645+0000 7f3fbd7fa640 1 -- 172.21.15.55:0/2262578798 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f3fb0059340 con 0x7f3fc00cf690 2024-05-20T22:08:19.646 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.647+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2262578798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3fac06be30 msgr2=0x7f3fac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.647+0000 7f3fc6baa640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3fac06be30 0x7f3fac06e340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3fc015cc40 tx=0x7f3fb4008040 comp rx=0 tx=0).stop 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.647+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2262578798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cf690 msgr2=0x7f3fc00b93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.647+0000 7f3fc6baa640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cf690 0x7f3fc00b93a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000d7d0 tx=0x7f3fb000dc90 comp rx=0 tx=0).stop 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.648+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2262578798 shutdown_connections 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.648+0000 7f3fc6baa640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3fac06be30 0x7f3fac06e340 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.648+0000 7f3fc6baa640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fc00d32a0 0x7f3fc014def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.648+0000 7f3fc6baa640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc00cffe0 0x7f3fc00b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.648+0000 7f3fc6baa640 1 --2- 172.21.15.55:0/2262578798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00cf690 0x7f3fc00b93a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.648+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2262578798 >> 172.21.15.55:0/2262578798 conn(0x7f3fc00b6de0 msgr2=0x7f3fc00ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.648+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2262578798 shutdown_connections 2024-05-20T22:08:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.648+0000 7f3fc6baa640 1 -- 172.21.15.55:0/2262578798 wait complete. 2024-05-20T22:08:19.660 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:19.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.777+0000 7f9011a13640 1 Processor -- start 2024-05-20T22:08:19.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.777+0000 7f9011a13640 1 -- start start 2024-05-20T22:08:19.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.778+0000 7f9011a13640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f900c0ce230 0x7f900c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.778+0000 7f9011a13640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f900c09add0 0x7f900c15fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.778+0000 7f9011a13640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900c0b3180 con 0x7f900c0cf3f0 2024-05-20T22:08:19.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.778+0000 7f9011a13640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f900c0ce730 con 0x7f900c09add0 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.778+0000 7f9011a13640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f900c0cea10 con 0x7f900c0ce230 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.778+0000 7f900b7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f900c09add0 0x7f900c15fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.778+0000 7f900a7fc640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f900c0cf3f0 0x7f900c09a6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46802/0 (socket says 172.21.15.55:46802) 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.778+0000 7f900a7fc640 1 -- 172.21.15.55:0/2420692401 learned_addr learned my addr 172.21.15.55:0/2420692401 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.778+0000 7f900affd640 1 --2- 172.21.15.55:0/2420692401 >> v2:172.21.15.138:3300/0 conn(0x7f900c0ce230 0x7f900c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.779+0000 7f900b7fe640 1 -- 172.21.15.55:0/2420692401 >> v2:172.21.15.138:3300/0 conn(0x7f900c0ce230 msgr2=0x7f900c0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.779+0000 7f900b7fe640 1 --2- 172.21.15.55:0/2420692401 >> v2:172.21.15.138:3300/0 conn(0x7f900c0ce230 0x7f900c0b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.779+0000 7f900b7fe640 1 -- 172.21.15.55:0/2420692401 >> v1:172.21.15.55:6789/0 conn(0x7f900c0cf3f0 legacy=0x7f900c09a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.779+0000 7f900b7fe640 1 -- 172.21.15.55:0/2420692401 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c0cecf0 con 0x7f900c09add0 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.779+0000 7f9009ffb640 1 -- 172.21.15.55:0/2420692401 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1831387802 0 0) 0x7f900c0b3180 con 0x7f900c0cf3f0 2024-05-20T22:08:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.779+0000 7f900b7fe640 1 --2- 172.21.15.55:0/2420692401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c09add0 0x7f900c15fe70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f900c072a80 tx=0x7f900402c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81b027c2a178997 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9009ffb640 1 -- 172.21.15.55:0/2420692401 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004032770 con 0x7f900c09add0 2024-05-20T22:08:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9009ffb640 1 -- 172.21.15.55:0/2420692401 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9004038070 con 0x7f900c09add0 2024-05-20T22:08:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9009ffb640 1 -- 172.21.15.55:0/2420692401 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004032b30 con 0x7f900c09add0 2024-05-20T22:08:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9011a13640 1 -- 172.21.15.55:0/2420692401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c09add0 msgr2=0x7f900c15fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9011a13640 1 --2- 172.21.15.55:0/2420692401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c09add0 0x7f900c15fe70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f900c072a80 tx=0x7f900402c870 comp rx=0 tx=0).stop 2024-05-20T22:08:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9011a13640 1 -- 172.21.15.55:0/2420692401 shutdown_connections 2024-05-20T22:08:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9011a13640 1 --2- 172.21.15.55:0/2420692401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c09add0 0x7f900c15fe70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9011a13640 1 --2- 172.21.15.55:0/2420692401 >> v2:172.21.15.138:3300/0 conn(0x7f900c0ce230 0x7f900c0b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9011a13640 1 -- 172.21.15.55:0/2420692401 >> 172.21.15.55:0/2420692401 conn(0x7f900c0737c0 msgr2=0x7f900c0b6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9011a13640 1 -- 172.21.15.55:0/2420692401 shutdown_connections 2024-05-20T22:08:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.780+0000 7f9011a13640 1 -- 172.21.15.55:0/2420692401 wait complete. 2024-05-20T22:08:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.782+0000 7f9011a13640 1 Processor -- start 2024-05-20T22:08:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.782+0000 7f9011a13640 1 -- start start 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f9011a13640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f900c0ce230 0x7f900c1f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f9011a13640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f900c0cf3f0 0x7f900c0c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f9011a13640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c0c7290 0x7f900c15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f9011a13640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f900c15c200 con 0x7f900c0c7290 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f9011a13640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f900c15f950 con 0x7f900c0cf3f0 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f9011a13640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f900c06e7f0 con 0x7f900c0ce230 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c0c7290 0x7f900c15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c0c7290 0x7f900c15ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35274/0 (socket says 172.21.15.55:35274) 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900b7fe640 1 -- 172.21.15.55:0/699773887 learned_addr learned my addr 172.21.15.55:0/699773887 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900a7fc640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f900c0cf3f0 0x7f900c0c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900affd640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f900c0ce230 0x7f900c1f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900b7fe640 1 -- 172.21.15.55:0/699773887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f900c0ce230 msgr2=0x7f900c1f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900b7fe640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f900c0ce230 0x7f900c1f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900b7fe640 1 -- 172.21.15.55:0/699773887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f900c0cf3f0 msgr2=0x7f900c0c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900b7fe640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f900c0cf3f0 0x7f900c0c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900b7fe640 1 -- 172.21.15.55:0/699773887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c0c7a60 con 0x7f900c0c7290 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f900b7fe640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c0c7290 0x7f900c15ae40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f900400bb30 tx=0x7f9004004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.784+0000 7f9003fff640 1 -- 172.21.15.55:0/699773887 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004004230 con 0x7f900c0c7290 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.785+0000 7f9003fff640 1 -- 172.21.15.55:0/699773887 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90040043d0 con 0x7f900c0c7290 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.785+0000 7f9011a13640 1 -- 172.21.15.55:0/699773887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c0c7d40 con 0x7f900c0c7290 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.785+0000 7f9003fff640 1 -- 172.21.15.55:0/699773887 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900403d8d0 con 0x7f900c0c7290 2024-05-20T22:08:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.785+0000 7f9011a13640 1 -- 172.21.15.55:0/699773887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c1f5280 con 0x7f900c0c7290 2024-05-20T22:08:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.786+0000 7f9011a13640 1 -- 172.21.15.55:0/699773887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c0ae690 con 0x7f900c0c7290 2024-05-20T22:08:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.786+0000 7f9003fff640 1 -- 172.21.15.55:0/699773887 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9004032070 con 0x7f900c0c7290 2024-05-20T22:08:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.787+0000 7f9003fff640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8fec06be60 0x7f8fec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.787+0000 7f900affd640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8fec06be60 0x7f8fec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.787+0000 7f9003fff640 1 -- 172.21.15.55:0/699773887 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f90040abe60 con 0x7f900c0c7290 2024-05-20T22:08:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.787+0000 7f900affd640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8fec06be60 0x7f8fec06e370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f900c0b7240 tx=0x7f8ffc00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:19.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.792+0000 7f9003fff640 1 -- 172.21.15.55:0/699773887 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f900c0ae690 con 0x7f900c0c7290 2024-05-20T22:08:19.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.978+0000 7f9011a13640 1 -- 172.21.15.55:0/699773887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f900c0b4590 con 0x7f900c0c7290 2024-05-20T22:08:19.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.978+0000 7f9003fff640 1 -- 172.21.15.55:0/699773887 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f9004079f00 con 0x7f900c0c7290 2024-05-20T22:08:19.979 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:19.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 -- 172.21.15.55:0/699773887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8fec06be60 msgr2=0x7f8fec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8fec06be60 0x7f8fec06e370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f900c0b7240 tx=0x7f8ffc00b040 comp rx=0 tx=0).stop 2024-05-20T22:08:19.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 -- 172.21.15.55:0/699773887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c0c7290 msgr2=0x7f900c15ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:19.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c0c7290 0x7f900c15ae40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f900400bb30 tx=0x7f9004004730 comp rx=0 tx=0).stop 2024-05-20T22:08:19.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 -- 172.21.15.55:0/699773887 shutdown_connections 2024-05-20T22:08:19.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8fec06be60 0x7f8fec06e370 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f900c0c7290 0x7f900c15ae40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f900c0cf3f0 0x7f900c0c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 --2- 172.21.15.55:0/699773887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f900c0ce230 0x7f900c1f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:19.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 -- 172.21.15.55:0/699773887 >> 172.21.15.55:0/699773887 conn(0x7f900c0737c0 msgr2=0x7f900c06fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:19.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 -- 172.21.15.55:0/699773887 shutdown_connections 2024-05-20T22:08:19.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:19.980+0000 7f9011a13640 1 -- 172.21.15.55:0/699773887 wait complete. 2024-05-20T22:08:19.993 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:08:20.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.106+0000 7feedaa61640 1 Processor -- start 2024-05-20T22:08:20.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.106+0000 7feedaa61640 1 -- start start 2024-05-20T22:08:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feedaa61640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406d8f0 0x7feed406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feedaa61640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feed4163510 0x7feed4163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feedaa61640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feed4163e60 0x7feed40b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feedaa61640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7feed406e370 con 0x7feed4163e60 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feedaa61640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feed40d2000 con 0x7feed4163510 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feedaa61640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feed40b7e20 con 0x7feed406d8f0 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feed8ffe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feed4163e60 0x7feed40b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feed8ffe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feed4163e60 0x7feed40b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35288/0 (socket says 172.21.15.55:35288) 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feed8ffe640 1 -- 172.21.15.55:0/175791777 learned_addr learned my addr 172.21.15.55:0/175791777 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feed3fff640 1 --2- 172.21.15.55:0/175791777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406d8f0 0x7feed406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feecbfff640 1 --2- 172.21.15.55:0/175791777 >> v2:172.21.15.138:3300/0 conn(0x7feed4163510 0x7feed4163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feed8ffe640 1 -- 172.21.15.55:0/175791777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406d8f0 msgr2=0x7feed406dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feed8ffe640 1 --2- 172.21.15.55:0/175791777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406d8f0 0x7feed406dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feed8ffe640 1 -- 172.21.15.55:0/175791777 >> v2:172.21.15.138:3300/0 conn(0x7feed4163510 msgr2=0x7feed4163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:20.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feed8ffe640 1 --2- 172.21.15.55:0/175791777 >> v2:172.21.15.138:3300/0 conn(0x7feed4163510 0x7feed4163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.108+0000 7feed8ffe640 1 -- 172.21.15.55:0/175791777 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed4164270 con 0x7feed4163e60 2024-05-20T22:08:20.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.109+0000 7feed8ffe640 1 --2- 172.21.15.55:0/175791777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163e60 0x7feed40b2bc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feed4050040 tx=0x7feecc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c0ff35b465d776 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:20.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.109+0000 7feed37fe640 1 -- 172.21.15.55:0/175791777 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc036070 con 0x7feed4163e60 2024-05-20T22:08:20.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.109+0000 7feed37fe640 1 -- 172.21.15.55:0/175791777 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feecc031600 con 0x7feed4163e60 2024-05-20T22:08:20.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.109+0000 7feed37fe640 1 -- 172.21.15.55:0/175791777 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc0319d0 con 0x7feed4163e60 2024-05-20T22:08:20.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.110+0000 7feedaa61640 1 -- 172.21.15.55:0/175791777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163e60 msgr2=0x7feed40b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.110+0000 7feedaa61640 1 --2- 172.21.15.55:0/175791777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163e60 0x7feed40b2bc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feed4050040 tx=0x7feecc02a610 comp rx=0 tx=0).stop 2024-05-20T22:08:20.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.110+0000 7feedaa61640 1 -- 172.21.15.55:0/175791777 shutdown_connections 2024-05-20T22:08:20.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.110+0000 7feedaa61640 1 --2- 172.21.15.55:0/175791777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163e60 0x7feed40b2bc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.110+0000 7feedaa61640 1 --2- 172.21.15.55:0/175791777 >> v2:172.21.15.138:3300/0 conn(0x7feed4163510 0x7feed4163920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.110+0000 7feedaa61640 1 --2- 172.21.15.55:0/175791777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed406d8f0 0x7feed406dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.110+0000 7feedaa61640 1 -- 172.21.15.55:0/175791777 >> 172.21.15.55:0/175791777 conn(0x7feed40b4340 msgr2=0x7feed40b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:20.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.110+0000 7feedaa61640 1 -- 172.21.15.55:0/175791777 shutdown_connections 2024-05-20T22:08:20.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.110+0000 7feedaa61640 1 -- 172.21.15.55:0/175791777 wait complete. 2024-05-20T22:08:20.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.112+0000 7feedaa61640 1 Processor -- start 2024-05-20T22:08:20.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.112+0000 7feedaa61640 1 -- start start 2024-05-20T22:08:20.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.113+0000 7feedaa61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed406d8f0 0x7feed4162170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.113+0000 7feedaa61640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163510 0x7feed40c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.113+0000 7feedaa61640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed4163e60 0x7feed40c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.113+0000 7feedaa61640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7feed406d340 con 0x7feed4163510 2024-05-20T22:08:20.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.113+0000 7feedaa61640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feed406d1c0 con 0x7feed406d8f0 2024-05-20T22:08:20.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.113+0000 7feedaa61640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feed40d2480 con 0x7feed4163e60 2024-05-20T22:08:20.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feecbfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163510 0x7feed40c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feecbfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163510 0x7feed40c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35298/0 (socket says 172.21.15.55:35298) 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feecbfff640 1 -- 172.21.15.55:0/3218808557 learned_addr learned my addr 172.21.15.55:0/3218808557 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feed3fff640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed406d8f0 0x7feed4162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feed8ffe640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed4163e60 0x7feed40c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feecbfff640 1 -- 172.21.15.55:0/3218808557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed4163e60 msgr2=0x7feed40c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feecbfff640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed4163e60 0x7feed40c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feecbfff640 1 -- 172.21.15.55:0/3218808557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed406d8f0 msgr2=0x7feed4162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feecbfff640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed406d8f0 0x7feed4162170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feecbfff640 1 -- 172.21.15.55:0/3218808557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed41f5510 con 0x7feed4163510 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feecbfff640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163510 0x7feed40c8a10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feebc00d5f0 tx=0x7feebc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feed17fa640 1 -- 172.21.15.55:0/3218808557 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc0125f0 con 0x7feed4163510 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feed17fa640 1 -- 172.21.15.55:0/3218808557 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc019070 con 0x7feed4163510 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feedaa61640 1 -- 172.21.15.55:0/3218808557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed41f57f0 con 0x7feed4163510 2024-05-20T22:08:20.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feed17fa640 1 -- 172.21.15.55:0/3218808557 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc0129d0 con 0x7feed4163510 2024-05-20T22:08:20.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.114+0000 7feedaa61640 1 -- 172.21.15.55:0/3218808557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41f5db0 con 0x7feed4163510 2024-05-20T22:08:20.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.115+0000 7feedaa61640 1 -- 172.21.15.55:0/3218808557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed4069090 con 0x7feed4163510 2024-05-20T22:08:20.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.116+0000 7feed17fa640 1 -- 172.21.15.55:0/3218808557 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feebc012b70 con 0x7feed4163510 2024-05-20T22:08:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.116+0000 7feed17fa640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be30 0x7feeb406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.116+0000 7feed17fa640 1 -- 172.21.15.55:0/3218808557 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feebc08b3b0 con 0x7feed4163510 2024-05-20T22:08:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.117+0000 7feed3fff640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be30 0x7feeb406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.117+0000 7feed3fff640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be30 0x7feeb406e340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feed40d0570 tx=0x7feec4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:20.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.121+0000 7feed17fa640 1 -- 172.21.15.55:0/3218808557 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7feebc059460 con 0x7feed4163510 2024-05-20T22:08:20.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.344+0000 7feedaa61640 1 -- 172.21.15.55:0/3218808557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feed40c9520 con 0x7feed4163510 2024-05-20T22:08:20.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.345+0000 7feed17fa640 1 -- 172.21.15.55:0/3218808557 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7feebc05d760 con 0x7feed4163510 2024-05-20T22:08:20.346 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:20.346 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:07:53.028952+0000","modified":"2024-05-20T22:08:11.135754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-05-20T22:08:20.346 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-20T22:08:20.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 -- 172.21.15.55:0/3218808557 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be30 msgr2=0x7feeb406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be30 0x7feeb406e340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feed40d0570 tx=0x7feec4008040 comp rx=0 tx=0).stop 2024-05-20T22:08:20.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 -- 172.21.15.55:0/3218808557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163510 msgr2=0x7feed40c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163510 0x7feed40c8a10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feebc00d5f0 tx=0x7feebc00dab0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 -- 172.21.15.55:0/3218808557 shutdown_connections 2024-05-20T22:08:20.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be30 0x7feeb406e340 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feed4163e60 0x7feed40c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feed4163510 0x7feed40c8a10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 --2- 172.21.15.55:0/3218808557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feed406d8f0 0x7feed4162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 -- 172.21.15.55:0/3218808557 >> 172.21.15.55:0/3218808557 conn(0x7feed40b4340 msgr2=0x7feed40cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:20.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 -- 172.21.15.55:0/3218808557 shutdown_connections 2024-05-20T22:08:20.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.347+0000 7feedaa61640 1 -- 172.21.15.55:0/3218808557 wait complete. 2024-05-20T22:08:20.360 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.474+0000 7f4122b85640 1 Processor -- start 2024-05-20T22:08:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.474+0000 7f4122b85640 1 -- start start 2024-05-20T22:08:20.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.475+0000 7f4122b85640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f411c0b3b30 0x7f411c0b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.475+0000 7f4122b85640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f411c15f100 0x7f411c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.475+0000 7f4122b85640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411c0d3dd0 con 0x7f411c06c5c0 2024-05-20T22:08:20.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.475+0000 7f4122b85640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f411c0d40b0 con 0x7f411c0b3b30 2024-05-20T22:08:20.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.475+0000 7f4122b85640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f411c0d4390 con 0x7f411c15f100 2024-05-20T22:08:20.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.475+0000 7f4120921640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f411c0b3b30 0x7f411c0b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.475+0000 7f4120921640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f411c0b3b30 0x7f411c0b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35308/0 (socket says 172.21.15.55:35308) 2024-05-20T22:08:20.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.475+0000 7f4120921640 1 -- 172.21.15.55:0/994151017 learned_addr learned my addr 172.21.15.55:0/994151017 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:20.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.475+0000 7f411bfff640 1 --2- 172.21.15.55:0/994151017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f411c15f100 0x7f411c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.476+0000 7f4120921640 1 -- 172.21.15.55:0/994151017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f411c15f100 msgr2=0x7f411c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:20.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.476+0000 7f4120921640 1 --2- 172.21.15.55:0/994151017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f411c15f100 0x7f411c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.476+0000 7f4120921640 1 -- 172.21.15.55:0/994151017 >> v1:172.21.15.138:6789/0 conn(0x7f411c06c5c0 legacy=0x7f411c162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.476+0000 7f4120921640 1 -- 172.21.15.55:0/994151017 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c06b5a0 con 0x7f411c0b3b30 2024-05-20T22:08:20.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.476+0000 7f4120921640 1 --2- 172.21.15.55:0/994151017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 0x7f411c0b7fe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4110009870 tx=0x7f411002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77511c8cff47d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:20.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.476+0000 7f411b7fe640 1 -- 172.21.15.55:0/994151017 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110037070 con 0x7f411c0b3b30 2024-05-20T22:08:20.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.476+0000 7f411b7fe640 1 -- 172.21.15.55:0/994151017 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4110032440 con 0x7f411c0b3b30 2024-05-20T22:08:20.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.477+0000 7f411b7fe640 1 -- 172.21.15.55:0/994151017 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110032810 con 0x7f411c0b3b30 2024-05-20T22:08:20.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.477+0000 7f4122b85640 1 -- 172.21.15.55:0/994151017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 msgr2=0x7f411c0b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.477+0000 7f4122b85640 1 --2- 172.21.15.55:0/994151017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 0x7f411c0b7fe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4110009870 tx=0x7f411002a270 comp rx=0 tx=0).stop 2024-05-20T22:08:20.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.477+0000 7f4122b85640 1 -- 172.21.15.55:0/994151017 shutdown_connections 2024-05-20T22:08:20.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.477+0000 7f4122b85640 1 --2- 172.21.15.55:0/994151017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f411c15f100 0x7f411c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.477+0000 7f4122b85640 1 --2- 172.21.15.55:0/994151017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 0x7f411c0b7fe0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.477+0000 7f4122b85640 1 -- 172.21.15.55:0/994151017 >> 172.21.15.55:0/994151017 conn(0x7f411c0d51d0 msgr2=0x7f411c0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:20.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.477+0000 7f4122b85640 1 -- 172.21.15.55:0/994151017 shutdown_connections 2024-05-20T22:08:20.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.477+0000 7f4122b85640 1 -- 172.21.15.55:0/994151017 wait complete. 2024-05-20T22:08:20.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.479+0000 7f4122b85640 1 Processor -- start 2024-05-20T22:08:20.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.479+0000 7f4122b85640 1 -- start start 2024-05-20T22:08:20.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.480+0000 7f4122b85640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c06c5c0 0x7f411c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.480+0000 7f4122b85640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 0x7f411c152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f4122b85640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f411c15f100 0x7f411c15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f4122b85640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f411c06c9d0 con 0x7f411c0b3b30 2024-05-20T22:08:20.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f4122b85640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f411c06bb60 con 0x7f411c06c5c0 2024-05-20T22:08:20.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f4122b85640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f411c06cb50 con 0x7f411c15f100 2024-05-20T22:08:20.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f411bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 0x7f411c152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f411bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 0x7f411c152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35322/0 (socket says 172.21.15.55:35322) 2024-05-20T22:08:20.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f411bfff640 1 -- 172.21.15.55:0/2098463100 learned_addr learned my addr 172.21.15.55:0/2098463100 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:20.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f4120921640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c06c5c0 0x7f411c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f4121122640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f411c15f100 0x7f411c15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f411bfff640 1 -- 172.21.15.55:0/2098463100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f411c15f100 msgr2=0x7f411c15cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f411bfff640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f411c15f100 0x7f411c15cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f411bfff640 1 -- 172.21.15.55:0/2098463100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c06c5c0 msgr2=0x7f411c15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f411bfff640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c06c5c0 0x7f411c15a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f411bfff640 1 -- 172.21.15.55:0/2098463100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c15aae0 con 0x7f411c0b3b30 2024-05-20T22:08:20.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f411bfff640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 0x7f411c152f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f410800d620 tx=0x7f410800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:20.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f41197fa640 1 -- 172.21.15.55:0/2098463100 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410800b620 con 0x7f411c0b3b30 2024-05-20T22:08:20.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f41197fa640 1 -- 172.21.15.55:0/2098463100 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4108019070 con 0x7f411c0b3b30 2024-05-20T22:08:20.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f4122b85640 1 -- 172.21.15.55:0/2098463100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c1534c0 con 0x7f411c0b3b30 2024-05-20T22:08:20.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f41197fa640 1 -- 172.21.15.55:0/2098463100 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410800ba00 con 0x7f411c0b3b30 2024-05-20T22:08:20.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.481+0000 7f4122b85640 1 -- 172.21.15.55:0/2098463100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c1f90d0 con 0x7f411c0b3b30 2024-05-20T22:08:20.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.482+0000 7f4122b85640 1 -- 172.21.15.55:0/2098463100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411c069090 con 0x7f411c0b3b30 2024-05-20T22:08:20.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.483+0000 7f41197fa640 1 -- 172.21.15.55:0/2098463100 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f410800bba0 con 0x7f411c0b3b30 2024-05-20T22:08:20.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.483+0000 7f41197fa640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f410006be30 0x7f410006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.483+0000 7f41197fa640 1 -- 172.21.15.55:0/2098463100 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f410808b5b0 con 0x7f411c0b3b30 2024-05-20T22:08:20.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.484+0000 7f4120921640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f410006be30 0x7f410006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.484+0000 7f4120921640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f410006be30 0x7f410006e340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4110034000 tx=0x7f411002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:20.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.488+0000 7f41197fa640 1 -- 172.21.15.55:0/2098463100 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f411c1534c0 con 0x7f411c0b3b30 2024-05-20T22:08:20.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.674+0000 7f4122b85640 1 -- 172.21.15.55:0/2098463100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f411c1638e0 con 0x7f411c0b3b30 2024-05-20T22:08:20.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.675+0000 7f41197fa640 1 -- 172.21.15.55:0/2098463100 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f4108059c20 con 0x7f411c0b3b30 2024-05-20T22:08:20.676 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:20.676 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:20.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 -- 172.21.15.55:0/2098463100 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f410006be30 msgr2=0x7f410006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f410006be30 0x7f410006e340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4110034000 tx=0x7f411002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:20.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 -- 172.21.15.55:0/2098463100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 msgr2=0x7f411c152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 0x7f411c152f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f410800d620 tx=0x7f410800dae0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 -- 172.21.15.55:0/2098463100 shutdown_connections 2024-05-20T22:08:20.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f410006be30 0x7f410006e340 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f411c15f100 0x7f411c15cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f411c0b3b30 0x7f411c152f80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 --2- 172.21.15.55:0/2098463100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c06c5c0 0x7f411c15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 -- 172.21.15.55:0/2098463100 >> 172.21.15.55:0/2098463100 conn(0x7f411c0d51d0 msgr2=0x7f411c1537c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:20.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 -- 172.21.15.55:0/2098463100 shutdown_connections 2024-05-20T22:08:20.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.677+0000 7f4122b85640 1 -- 172.21.15.55:0/2098463100 wait complete. 2024-05-20T22:08:20.690 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:20.690 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:20.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.807+0000 7f41e7c1a640 1 Processor -- start 2024-05-20T22:08:20.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.807+0000 7f41e7c1a640 1 -- start start 2024-05-20T22:08:20.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e7c1a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00d32a0 0x7f41e015f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e7c1a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41e00d01b0 0x7f41e006b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e7c1a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e00d3780 con 0x7f41e00cf690 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e7c1a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f41e00d3a60 con 0x7f41e00d01b0 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e7c1a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41e00d3d40 con 0x7f41e00d32a0 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e61b7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41e00d01b0 0x7f41e006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e59b6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00d32a0 0x7f41e015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e59b6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00d32a0 0x7f41e015f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:48522/0 (socket says 172.21.15.55:48522) 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e59b6640 1 -- 172.21.15.55:0/1546488598 learned_addr learned my addr 172.21.15.55:0/1546488598 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e51b5640 1 --1- 172.21.15.55:0/1546488598 >> v1:172.21.15.138:6789/0 conn(0x7f41e00cf690 0x7f41e00cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:51580/0 (socket says 172.21.15.55:51580) 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e59b6640 1 -- 172.21.15.55:0/1546488598 >> v2:172.21.15.138:3300/0 conn(0x7f41e00d01b0 msgr2=0x7f41e006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e59b6640 1 --2- 172.21.15.55:0/1546488598 >> v2:172.21.15.138:3300/0 conn(0x7f41e00d01b0 0x7f41e006b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e59b6640 1 -- 172.21.15.55:0/1546488598 >> v1:172.21.15.138:6789/0 conn(0x7f41e00cf690 legacy=0x7f41e00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e59b6640 1 -- 172.21.15.55:0/1546488598 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e00d4020 con 0x7f41e00d32a0 2024-05-20T22:08:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.809+0000 7f41e61b7640 1 --2- 172.21.15.55:0/1546488598 >> v2:172.21.15.138:3300/0 conn(0x7f41e00d01b0 0x7f41e006b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.810+0000 7f41e59b6640 1 --2- 172.21.15.55:0/1546488598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00d32a0 0x7f41e015f150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009870 tx=0x7f41dc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a66bf66b5a56582 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.810+0000 7f41e49b4640 1 -- 172.21.15.55:0/1546488598 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc037070 con 0x7f41e00d32a0 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.810+0000 7f41e49b4640 1 -- 172.21.15.55:0/1546488598 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41dc032440 con 0x7f41e00d32a0 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.810+0000 7f41e49b4640 1 -- 172.21.15.55:0/1546488598 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc032810 con 0x7f41e00d32a0 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.811+0000 7f41e7c1a640 1 -- 172.21.15.55:0/1546488598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00d32a0 msgr2=0x7f41e015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.811+0000 7f41e7c1a640 1 --2- 172.21.15.55:0/1546488598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00d32a0 0x7f41e015f150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009870 tx=0x7f41dc02a270 comp rx=0 tx=0).stop 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.811+0000 7f41e7c1a640 1 -- 172.21.15.55:0/1546488598 shutdown_connections 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.811+0000 7f41e7c1a640 1 --2- 172.21.15.55:0/1546488598 >> v2:172.21.15.138:3300/0 conn(0x7f41e00d01b0 0x7f41e006b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.811+0000 7f41e7c1a640 1 --2- 172.21.15.55:0/1546488598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00d32a0 0x7f41e015f150 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.811+0000 7f41e7c1a640 1 -- 172.21.15.55:0/1546488598 >> 172.21.15.55:0/1546488598 conn(0x7f41e00b6de0 msgr2=0x7f41e00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.811+0000 7f41e7c1a640 1 -- 172.21.15.55:0/1546488598 shutdown_connections 2024-05-20T22:08:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.811+0000 7f41e7c1a640 1 -- 172.21.15.55:0/1546488598 wait complete. 2024-05-20T22:08:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.813+0000 7f41e7c1a640 1 Processor -- start 2024-05-20T22:08:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.813+0000 7f41e7c1a640 1 -- start start 2024-05-20T22:08:20.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e7c1a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00cf690 0x7f41e01e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e7c1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41e00d01b0 0x7f41e015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e7c1a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00d32a0 0x7f41e015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e7c1a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f41e00d4820 con 0x7f41e00d32a0 2024-05-20T22:08:20.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e7c1a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41e00b4350 con 0x7f41e00d01b0 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e7c1a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41e01f6b00 con 0x7f41e00cf690 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e61b7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00d32a0 0x7f41e015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e61b7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00d32a0 0x7f41e015c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35332/0 (socket says 172.21.15.55:35332) 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e51b5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41e00d01b0 0x7f41e015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e61b7640 1 -- 172.21.15.55:0/3879820053 learned_addr learned my addr 172.21.15.55:0/3879820053 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.814+0000 7f41e59b6640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00cf690 0x7f41e01e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.815+0000 7f41e61b7640 1 -- 172.21.15.55:0/3879820053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00cf690 msgr2=0x7f41e01e9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.815+0000 7f41e61b7640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00cf690 0x7f41e01e9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.815+0000 7f41e61b7640 1 -- 172.21.15.55:0/3879820053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41e00d01b0 msgr2=0x7f41e015c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.815+0000 7f41e61b7640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41e00d01b0 0x7f41e015c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.815+0000 7f41e61b7640 1 -- 172.21.15.55:0/3879820053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e00ccb90 con 0x7f41e00d32a0 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.815+0000 7f41e59b6640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00cf690 0x7f41e01e9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.815+0000 7f41e61b7640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00d32a0 0x7f41e015c9d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41d800ef90 tx=0x7f41d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.817+0000 7f41d67fc640 1 -- 172.21.15.55:0/3879820053 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8018020 con 0x7f41e00d32a0 2024-05-20T22:08:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.817+0000 7f41d67fc640 1 -- 172.21.15.55:0/3879820053 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d8014480 con 0x7f41e00d32a0 2024-05-20T22:08:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.817+0000 7f41e7c1a640 1 -- 172.21.15.55:0/3879820053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e01ea810 con 0x7f41e00d32a0 2024-05-20T22:08:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.817+0000 7f41d67fc640 1 -- 172.21.15.55:0/3879820053 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8014870 con 0x7f41e00d32a0 2024-05-20T22:08:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.817+0000 7f41e7c1a640 1 -- 172.21.15.55:0/3879820053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e00cd0f0 con 0x7f41e00d32a0 2024-05-20T22:08:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.818+0000 7f41e7c1a640 1 -- 172.21.15.55:0/3879820053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e0069090 con 0x7f41e00d32a0 2024-05-20T22:08:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.819+0000 7f41d67fc640 1 -- 172.21.15.55:0/3879820053 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41d8005d10 con 0x7f41e00d32a0 2024-05-20T22:08:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.819+0000 7f41d67fc640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41c406be30 0x7f41c406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.819+0000 7f41d67fc640 1 -- 172.21.15.55:0/3879820053 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f41d808bcc0 con 0x7f41e00d32a0 2024-05-20T22:08:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.819+0000 7f41e59b6640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41c406be30 0x7f41c406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.820+0000 7f41e59b6640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41c406be30 0x7f41c406e340 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41dc034000 tx=0x7f41dc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:20.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:20.824+0000 7f41d67fc640 1 -- 172.21.15.55:0/3879820053 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f41d8022030 con 0x7f41e00d32a0 2024-05-20T22:08:21.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.010+0000 7f41e7c1a640 1 -- 172.21.15.55:0/3879820053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41e00c83f0 con 0x7f41e00d32a0 2024-05-20T22:08:21.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.010+0000 7f41d67fc640 1 -- 172.21.15.55:0/3879820053 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f41e00ccb90 con 0x7f41e00d32a0 2024-05-20T22:08:21.011 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:21.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 -- 172.21.15.55:0/3879820053 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41c406be30 msgr2=0x7f41c406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41c406be30 0x7f41c406e340 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41dc034000 tx=0x7f41dc02e000 comp rx=0 tx=0).stop 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 -- 172.21.15.55:0/3879820053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00d32a0 msgr2=0x7f41e015c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00d32a0 0x7f41e015c9d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41d800ef90 tx=0x7f41d800c490 comp rx=0 tx=0).stop 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 -- 172.21.15.55:0/3879820053 shutdown_connections 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41c406be30 0x7f41c406e340 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00d32a0 0x7f41e015c9d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41e00d01b0 0x7f41e015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 --2- 172.21.15.55:0/3879820053 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41e00cf690 0x7f41e01e9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 -- 172.21.15.55:0/3879820053 >> 172.21.15.55:0/3879820053 conn(0x7f41e00b6de0 msgr2=0x7f41e006c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 -- 172.21.15.55:0/3879820053 shutdown_connections 2024-05-20T22:08:21.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.012+0000 7f41e7c1a640 1 -- 172.21.15.55:0/3879820053 wait complete. 2024-05-20T22:08:21.025 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.141+0000 7f6d684c3640 1 Processor -- start 2024-05-20T22:08:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.141+0000 7f6d684c3640 1 -- start start 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.142+0000 7f6d684c3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6d6009ab70 0x7f6d6006df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.142+0000 7f6d684c3640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6d600d0930 0x7f6d600d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.142+0000 7f6d684c3640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d600ccdc0 con 0x7f6d600ce150 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.142+0000 7f6d684c3640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f6d600b2a60 con 0x7f6d600d0930 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.142+0000 7f6d684c3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6d600cda00 con 0x7f6d6009ab70 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.143+0000 7f6d65a5e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6d600d0930 0x7f6d600d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.143+0000 7f6d66a60640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6d600ce150 0x7f6d600ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46804/0 (socket says 172.21.15.55:46804) 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.143+0000 7f6d66a60640 1 -- 172.21.15.55:0/829323428 learned_addr learned my addr 172.21.15.55:0/829323428 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.143+0000 7f6d6625f640 1 --2- 172.21.15.55:0/829323428 >> v2:172.21.15.138:3300/0 conn(0x7f6d6009ab70 0x7f6d6006df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.143+0000 7f6d65a5e640 1 -- 172.21.15.55:0/829323428 >> v2:172.21.15.138:3300/0 conn(0x7f6d6009ab70 msgr2=0x7f6d6006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.143+0000 7f6d65a5e640 1 --2- 172.21.15.55:0/829323428 >> v2:172.21.15.138:3300/0 conn(0x7f6d6009ab70 0x7f6d6006df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.143+0000 7f6d65a5e640 1 -- 172.21.15.55:0/829323428 >> v1:172.21.15.55:6789/0 conn(0x7f6d600ce150 legacy=0x7f6d600ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.143+0000 7f6d6525d640 1 -- 172.21.15.55:0/829323428 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 564498090 0 0) 0x7f6d600ccdc0 con 0x7f6d600ce150 2024-05-20T22:08:21.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.143+0000 7f6d65a5e640 1 -- 172.21.15.55:0/829323428 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d600cdce0 con 0x7f6d600d0930 2024-05-20T22:08:21.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.144+0000 7f6d65a5e640 1 --2- 172.21.15.55:0/829323428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d600d0930 0x7f6d600d0d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6d600cf280 tx=0x7f6d5002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c25b338ad9fce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:21.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.144+0000 7f6d6525d640 1 -- 172.21.15.55:0/829323428 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50037070 con 0x7f6d600d0930 2024-05-20T22:08:21.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.144+0000 7f6d6525d640 1 -- 172.21.15.55:0/829323428 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d50032440 con 0x7f6d600d0930 2024-05-20T22:08:21.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.144+0000 7f6d6525d640 1 -- 172.21.15.55:0/829323428 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50032810 con 0x7f6d600d0930 2024-05-20T22:08:21.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.144+0000 7f6d684c3640 1 -- 172.21.15.55:0/829323428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d600d0930 msgr2=0x7f6d600d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.144+0000 7f6d684c3640 1 --2- 172.21.15.55:0/829323428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d600d0930 0x7f6d600d0d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6d600cf280 tx=0x7f6d5002a1a0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.145+0000 7f6d684c3640 1 -- 172.21.15.55:0/829323428 shutdown_connections 2024-05-20T22:08:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.145+0000 7f6d684c3640 1 --2- 172.21.15.55:0/829323428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d600d0930 0x7f6d600d0d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.145+0000 7f6d684c3640 1 --2- 172.21.15.55:0/829323428 >> v2:172.21.15.138:3300/0 conn(0x7f6d6009ab70 0x7f6d6006df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.145+0000 7f6d684c3640 1 -- 172.21.15.55:0/829323428 >> 172.21.15.55:0/829323428 conn(0x7f6d600736f0 msgr2=0x7f6d600b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.145+0000 7f6d684c3640 1 -- 172.21.15.55:0/829323428 shutdown_connections 2024-05-20T22:08:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.145+0000 7f6d684c3640 1 -- 172.21.15.55:0/829323428 wait complete. 2024-05-20T22:08:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.146+0000 7f6d684c3640 1 Processor -- start 2024-05-20T22:08:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.146+0000 7f6d684c3640 1 -- start start 2024-05-20T22:08:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d684c3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d6009ab70 0x7f6d6015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d684c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d600ce150 0x7f6d600c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d684c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d600d0930 0x7f6d60161430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d684c3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6d600b2ef0 con 0x7f6d6009ab70 2024-05-20T22:08:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d684c3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6006e6d0 con 0x7f6d600d0930 2024-05-20T22:08:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d684c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6d600b3070 con 0x7f6d600ce150 2024-05-20T22:08:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d6625f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d6009ab70 0x7f6d6015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d6625f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d6009ab70 0x7f6d6015da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35348/0 (socket says 172.21.15.55:35348) 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d6625f640 1 -- 172.21.15.55:0/3713680828 learned_addr learned my addr 172.21.15.55:0/3713680828 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d65a5e640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d600ce150 0x7f6d600c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d66a60640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d600d0930 0x7f6d60161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d6625f640 1 -- 172.21.15.55:0/3713680828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d600ce150 msgr2=0x7f6d600c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d6625f640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d600ce150 0x7f6d600c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d6625f640 1 -- 172.21.15.55:0/3713680828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d600d0930 msgr2=0x7f6d60161430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d6625f640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d600d0930 0x7f6d60161430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d6625f640 1 -- 172.21.15.55:0/3713680828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6015df40 con 0x7f6d6009ab70 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d66a60640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d600d0930 0x7f6d60161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d6625f640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d6009ab70 0x7f6d6015da00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00d5a0 tx=0x7f6d5c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.148+0000 7f6d56ffd640 1 -- 172.21.15.55:0/3713680828 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c019070 con 0x7f6d6009ab70 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.149+0000 7f6d56ffd640 1 -- 172.21.15.55:0/3713680828 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5c004240 con 0x7f6d6009ab70 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.149+0000 7f6d684c3640 1 -- 172.21.15.55:0/3713680828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d600c5650 con 0x7f6d6009ab70 2024-05-20T22:08:21.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.149+0000 7f6d56ffd640 1 -- 172.21.15.55:0/3713680828 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c003610 con 0x7f6d6009ab70 2024-05-20T22:08:21.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.149+0000 7f6d684c3640 1 -- 172.21.15.55:0/3713680828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d600c5c10 con 0x7f6d6009ab70 2024-05-20T22:08:21.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.150+0000 7f6d684c3640 1 -- 172.21.15.55:0/3713680828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d60069090 con 0x7f6d6009ab70 2024-05-20T22:08:21.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.150+0000 7f6d56ffd640 1 -- 172.21.15.55:0/3713680828 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d5c01c070 con 0x7f6d6009ab70 2024-05-20T22:08:21.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.151+0000 7f6d56ffd640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d4406be30 0x7f6d4406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.151+0000 7f6d65a5e640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d4406be30 0x7f6d4406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.151+0000 7f6d56ffd640 1 -- 172.21.15.55:0/3713680828 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6d5c08aaa0 con 0x7f6d6009ab70 2024-05-20T22:08:21.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.151+0000 7f6d65a5e640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d4406be30 0x7f6d4406e340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6d50034000 tx=0x7f6d5002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:21.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.156+0000 7f6d56ffd640 1 -- 172.21.15.55:0/3713680828 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6d5c058a30 con 0x7f6d6009ab70 2024-05-20T22:08:21.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.338+0000 7f6d684c3640 1 -- 172.21.15.55:0/3713680828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6d6006cab0 con 0x7f6d6009ab70 2024-05-20T22:08:21.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.339+0000 7f6d56ffd640 1 -- 172.21.15.55:0/3713680828 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f6d5c05cd30 con 0x7f6d6009ab70 2024-05-20T22:08:21.340 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:21.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 -- 172.21.15.55:0/3713680828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d4406be30 msgr2=0x7f6d4406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d4406be30 0x7f6d4406e340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6d50034000 tx=0x7f6d5002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:21.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 -- 172.21.15.55:0/3713680828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d6009ab70 msgr2=0x7f6d6015da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d6009ab70 0x7f6d6015da00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00d5a0 tx=0x7f6d5c00da60 comp rx=0 tx=0).stop 2024-05-20T22:08:21.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 -- 172.21.15.55:0/3713680828 shutdown_connections 2024-05-20T22:08:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d4406be30 0x7f6d4406e340 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d600d0930 0x7f6d60161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d600ce150 0x7f6d600c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 --2- 172.21.15.55:0/3713680828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d6009ab70 0x7f6d6015da00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 -- 172.21.15.55:0/3713680828 >> 172.21.15.55:0/3713680828 conn(0x7f6d600736f0 msgr2=0x7f6d600d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 -- 172.21.15.55:0/3713680828 shutdown_connections 2024-05-20T22:08:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.341+0000 7f6d684c3640 1 -- 172.21.15.55:0/3713680828 wait complete. 2024-05-20T22:08:21.353 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.469+0000 7fdf2fa8d640 1 Processor -- start 2024-05-20T22:08:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.469+0000 7fdf2fa8d640 1 -- start start 2024-05-20T22:08:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2fa8d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdf280d32a0 0x7fdf2815f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2fa8d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf280d3780 con 0x7fdf280cf690 2024-05-20T22:08:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2fa8d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf280d3a60 con 0x7fdf280d01b0 2024-05-20T22:08:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2fa8d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fdf280d3d40 con 0x7fdf280d32a0 2024-05-20T22:08:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2d829640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdf280d32a0 0x7fdf2815f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2d028640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdf280cf690 0x7fdf280cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46816/0 (socket says 172.21.15.55:46816) 2024-05-20T22:08:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2d028640 1 -- 172.21.15.55:0/160040303 learned_addr learned my addr 172.21.15.55:0/160040303 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2d829640 1 -- 172.21.15.55:0/160040303 >> v1:172.21.15.138:6789/0 conn(0x7fdf280d01b0 legacy=0x7fdf2806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2d829640 1 -- 172.21.15.55:0/160040303 >> v1:172.21.15.55:6789/0 conn(0x7fdf280cf690 legacy=0x7fdf280cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2d829640 1 -- 172.21.15.55:0/160040303 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf280d4020 con 0x7fdf280d32a0 2024-05-20T22:08:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.471+0000 7fdf2c827640 1 -- 172.21.15.55:0/160040303 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603276816 0 0) 0x7fdf280d3780 con 0x7fdf280cf690 2024-05-20T22:08:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.472+0000 7fdf2d829640 1 --2- 172.21.15.55:0/160040303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d32a0 0x7fdf2815f150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009d90 tx=0x7fdf1802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa7d25dc72ab45c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.472+0000 7fdf2c827640 1 -- 172.21.15.55:0/160040303 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18007740 con 0x7fdf280d32a0 2024-05-20T22:08:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.472+0000 7fdf2c827640 1 -- 172.21.15.55:0/160040303 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf180078e0 con 0x7fdf280d32a0 2024-05-20T22:08:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.472+0000 7fdf2c827640 1 -- 172.21.15.55:0/160040303 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18007cb0 con 0x7fdf280d32a0 2024-05-20T22:08:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.473+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/160040303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d32a0 msgr2=0x7fdf2815f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.473+0000 7fdf2fa8d640 1 --2- 172.21.15.55:0/160040303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d32a0 0x7fdf2815f150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009d90 tx=0x7fdf1802a6f0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.473+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/160040303 shutdown_connections 2024-05-20T22:08:21.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.473+0000 7fdf2fa8d640 1 --2- 172.21.15.55:0/160040303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d32a0 0x7fdf2815f150 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.473+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/160040303 >> 172.21.15.55:0/160040303 conn(0x7fdf280b6de0 msgr2=0x7fdf280d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:21.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.473+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/160040303 shutdown_connections 2024-05-20T22:08:21.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.473+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/160040303 wait complete. 2024-05-20T22:08:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.474+0000 7fdf2fa8d640 1 Processor -- start 2024-05-20T22:08:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.475+0000 7fdf2fa8d640 1 -- start start 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2fa8d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf280cf690 0x7fdf2815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2fa8d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d01b0 0x7fdf28160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2fa8d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf280d32a0 0x7fdf280cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2fa8d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdf280d4820 con 0x7fdf280d01b0 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2fa8d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf280b4350 con 0x7fdf280cf690 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2fa8d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf281f55e0 con 0x7fdf280d32a0 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2d028640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d01b0 0x7fdf28160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2d028640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d01b0 0x7fdf28160660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35362/0 (socket says 172.21.15.55:35362) 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2d028640 1 -- 172.21.15.55:0/2435313541 learned_addr learned my addr 172.21.15.55:0/2435313541 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2d829640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf280cf690 0x7fdf2815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2e02a640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf280d32a0 0x7fdf280cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2d028640 1 -- 172.21.15.55:0/2435313541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf280d32a0 msgr2=0x7fdf280cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2d028640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf280d32a0 0x7fdf280cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2d028640 1 -- 172.21.15.55:0/2435313541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf280cf690 msgr2=0x7fdf2815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2d028640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf280cf690 0x7fdf2815ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2d028640 1 -- 172.21.15.55:0/2435313541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf280ca610 con 0x7fdf280d01b0 2024-05-20T22:08:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.476+0000 7fdf2e02a640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf280d32a0 0x7fdf280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.477+0000 7fdf2d028640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d01b0 0x7fdf28160660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00c420 tx=0x7fdf1c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.477+0000 7fdf267fc640 1 -- 172.21.15.55:0/2435313541 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0074e0 con 0x7fdf280d01b0 2024-05-20T22:08:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.477+0000 7fdf267fc640 1 -- 172.21.15.55:0/2435313541 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0047e0 con 0x7fdf280d01b0 2024-05-20T22:08:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.477+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/2435313541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf280ca8f0 con 0x7fdf280d01b0 2024-05-20T22:08:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.477+0000 7fdf267fc640 1 -- 172.21.15.55:0/2435313541 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c004c10 con 0x7fdf280d01b0 2024-05-20T22:08:21.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.477+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/2435313541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf280caeb0 con 0x7fdf280d01b0 2024-05-20T22:08:21.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.478+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/2435313541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf28069090 con 0x7fdf280d01b0 2024-05-20T22:08:21.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.479+0000 7fdf267fc640 1 -- 172.21.15.55:0/2435313541 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf1c011070 con 0x7fdf280d01b0 2024-05-20T22:08:21.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.479+0000 7fdf267fc640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdf1406be30 0x7fdf1406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.479+0000 7fdf267fc640 1 -- 172.21.15.55:0/2435313541 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdf1c08ac90 con 0x7fdf280d01b0 2024-05-20T22:08:21.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.479+0000 7fdf2d829640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdf1406be30 0x7fdf1406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.480+0000 7fdf2d829640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdf1406be30 0x7fdf1406e340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf18034000 tx=0x7fdf1802f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:21.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.484+0000 7fdf267fc640 1 -- 172.21.15.55:0/2435313541 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdf1c059300 con 0x7fdf280d01b0 2024-05-20T22:08:21.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.666+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/2435313541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf280d2e50 con 0x7fdf280d01b0 2024-05-20T22:08:21.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.667+0000 7fdf267fc640 1 -- 172.21.15.55:0/2435313541 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fdf1c058c00 con 0x7fdf280d01b0 2024-05-20T22:08:21.668 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:21.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.668+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/2435313541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdf1406be30 msgr2=0x7fdf1406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.668+0000 7fdf2fa8d640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdf1406be30 0x7fdf1406e340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf18034000 tx=0x7fdf1802f000 comp rx=0 tx=0).stop 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.668+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/2435313541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d01b0 msgr2=0x7fdf28160660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.668+0000 7fdf2fa8d640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d01b0 0x7fdf28160660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00c420 tx=0x7fdf1c00c8e0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.669+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/2435313541 shutdown_connections 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.669+0000 7fdf2fa8d640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdf1406be30 0x7fdf1406e340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.669+0000 7fdf2fa8d640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf280d32a0 0x7fdf280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.669+0000 7fdf2fa8d640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf280d01b0 0x7fdf28160660 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.669+0000 7fdf2fa8d640 1 --2- 172.21.15.55:0/2435313541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf280cf690 0x7fdf2815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.669+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/2435313541 >> 172.21.15.55:0/2435313541 conn(0x7fdf280b6de0 msgr2=0x7fdf280b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.669+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/2435313541 shutdown_connections 2024-05-20T22:08:21.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.669+0000 7fdf2fa8d640 1 -- 172.21.15.55:0/2435313541 wait complete. 2024-05-20T22:08:21.681 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:21.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.800+0000 7f8b85444640 1 Processor -- start 2024-05-20T22:08:21.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.800+0000 7f8b85444640 1 -- start start 2024-05-20T22:08:21.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b85444640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 0x7f8b800cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b85444640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b800d08a0 0x7f8b800d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b85444640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b800cc3c0 con 0x7f8b800cd020 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b85444640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f8b800cc170 con 0x7f8b800d08a0 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b85444640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8b800ccc60 con 0x7f8b800ce1d0 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b7f7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b800d08a0 0x7f8b800d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b7f7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b800d08a0 0x7f8b800d0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35378/0 (socket says 172.21.15.55:35378) 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b7f7fe640 1 -- 172.21.15.55:0/1654904604 learned_addr learned my addr 172.21.15.55:0/1654904604 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b7e7fc640 1 --2- 172.21.15.55:0/1654904604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 0x7f8b800cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b7f7fe640 1 -- 172.21.15.55:0/1654904604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 msgr2=0x7f8b800cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b7f7fe640 1 --2- 172.21.15.55:0/1654904604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 0x7f8b800cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b7f7fe640 1 -- 172.21.15.55:0/1654904604 >> v1:172.21.15.138:6789/0 conn(0x7f8b800cd020 legacy=0x7f8b800cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b7f7fe640 1 -- 172.21.15.55:0/1654904604 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b800cc6a0 con 0x7f8b800d08a0 2024-05-20T22:08:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.802+0000 7f8b7e7fc640 1 --2- 172.21.15.55:0/1654904604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 0x7f8b800cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.803+0000 7f8b7f7fe640 1 --2- 172.21.15.55:0/1654904604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 0x7f8b800d0cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009870 tx=0x7f8b7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2a6173aa2102a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.803+0000 7f8b7dffb640 1 -- 172.21.15.55:0/1654904604 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78037070 con 0x7f8b800d08a0 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.803+0000 7f8b7dffb640 1 -- 172.21.15.55:0/1654904604 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b78032440 con 0x7f8b800d08a0 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.803+0000 7f8b7dffb640 1 -- 172.21.15.55:0/1654904604 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78032810 con 0x7f8b800d08a0 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.804+0000 7f8b85444640 1 -- 172.21.15.55:0/1654904604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 msgr2=0x7f8b800d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.804+0000 7f8b85444640 1 --2- 172.21.15.55:0/1654904604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 0x7f8b800d0cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009870 tx=0x7f8b7802a270 comp rx=0 tx=0).stop 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.804+0000 7f8b85444640 1 -- 172.21.15.55:0/1654904604 shutdown_connections 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.804+0000 7f8b85444640 1 --2- 172.21.15.55:0/1654904604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 0x7f8b800d0cb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.804+0000 7f8b85444640 1 --2- 172.21.15.55:0/1654904604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 0x7f8b800cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.804+0000 7f8b85444640 1 -- 172.21.15.55:0/1654904604 >> 172.21.15.55:0/1654904604 conn(0x7f8b800b62c0 msgr2=0x7f8b8009a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.804+0000 7f8b85444640 1 -- 172.21.15.55:0/1654904604 shutdown_connections 2024-05-20T22:08:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.804+0000 7f8b85444640 1 -- 172.21.15.55:0/1654904604 wait complete. 2024-05-20T22:08:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.806+0000 7f8b85444640 1 Processor -- start 2024-05-20T22:08:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.806+0000 7f8b85444640 1 -- start start 2024-05-20T22:08:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.807+0000 7f8b85444640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b800cd020 0x7f8b801f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.807+0000 7f8b85444640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 0x7f8b801f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.807+0000 7f8b85444640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 0x7f8b800af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.807+0000 7f8b85444640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8015f4d0 con 0x7f8b800d08a0 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.807+0000 7f8b85444640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b800d0380 con 0x7f8b800cd020 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.807+0000 7f8b85444640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8015f650 con 0x7f8b800ce1d0 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 0x7f8b800af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 0x7f8b800af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35390/0 (socket says 172.21.15.55:35390) 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7f7fe640 1 -- 172.21.15.55:0/3617639549 learned_addr learned my addr 172.21.15.55:0/3617639549 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7e7fc640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 0x7f8b801f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7effd640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b800cd020 0x7f8b801f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7f7fe640 1 -- 172.21.15.55:0/3617639549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 msgr2=0x7f8b801f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7f7fe640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 0x7f8b801f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7f7fe640 1 -- 172.21.15.55:0/3617639549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b800cd020 msgr2=0x7f8b801f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:21.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7f7fe640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b800cd020 0x7f8b801f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:21.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7f7fe640 1 -- 172.21.15.55:0/3617639549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b800ca530 con 0x7f8b800d08a0 2024-05-20T22:08:21.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7e7fc640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 0x7f8b801f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:21.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b7f7fe640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 0x7f8b800af390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009870 tx=0x7f8b780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:21.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b777fe640 1 -- 172.21.15.55:0/3617639549 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78037040 con 0x7f8b800d08a0 2024-05-20T22:08:21.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b777fe640 1 -- 172.21.15.55:0/3617639549 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b78042070 con 0x7f8b800d08a0 2024-05-20T22:08:21.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b777fe640 1 -- 172.21.15.55:0/3617639549 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78030920 con 0x7f8b800d08a0 2024-05-20T22:08:21.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b85444640 1 -- 172.21.15.55:0/3617639549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b800ca810 con 0x7f8b800d08a0 2024-05-20T22:08:21.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.808+0000 7f8b85444640 1 -- 172.21.15.55:0/3617639549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b800cadd0 con 0x7f8b800d08a0 2024-05-20T22:08:21.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.809+0000 7f8b85444640 1 -- 172.21.15.55:0/3617639549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b800ceef0 con 0x7f8b800d08a0 2024-05-20T22:08:21.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.810+0000 7f8b777fe640 1 -- 172.21.15.55:0/3617639549 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b78030ac0 con 0x7f8b800d08a0 2024-05-20T22:08:21.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.810+0000 7f8b777fe640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8b6006be60 0x7f8b6006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:21.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.810+0000 7f8b777fe640 1 -- 172.21.15.55:0/3617639549 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8b780abb10 con 0x7f8b800d08a0 2024-05-20T22:08:21.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.810+0000 7f8b7effd640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8b6006be60 0x7f8b6006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:21.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.811+0000 7f8b7effd640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8b6006be60 0x7f8b6006e370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8b70001ca0 tx=0x7f8b70009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:21.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.815+0000 7f8b777fe640 1 -- 172.21.15.55:0/3617639549 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8b800ceef0 con 0x7f8b800d08a0 2024-05-20T22:08:21.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.999+0000 7f8b85444640 1 -- 172.21.15.55:0/3617639549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8b800ca2c0 con 0x7f8b800d08a0 2024-05-20T22:08:22.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:21.999+0000 7f8b777fe640 1 -- 172.21.15.55:0/3617639549 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f8b78079aa0 con 0x7f8b800d08a0 2024-05-20T22:08:22.000 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 -- 172.21.15.55:0/3617639549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8b6006be60 msgr2=0x7f8b6006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8b6006be60 0x7f8b6006e370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8b70001ca0 tx=0x7f8b70009040 comp rx=0 tx=0).stop 2024-05-20T22:08:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 -- 172.21.15.55:0/3617639549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 msgr2=0x7f8b800af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 0x7f8b800af390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009870 tx=0x7f8b780046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 -- 172.21.15.55:0/3617639549 shutdown_connections 2024-05-20T22:08:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8b6006be60 0x7f8b6006e370 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b800d08a0 0x7f8b800af390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b800ce1d0 0x7f8b801f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 --2- 172.21.15.55:0/3617639549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b800cd020 0x7f8b801f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 -- 172.21.15.55:0/3617639549 >> 172.21.15.55:0/3617639549 conn(0x7f8b800b62c0 msgr2=0x7f8b80067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 -- 172.21.15.55:0/3617639549 shutdown_connections 2024-05-20T22:08:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.001+0000 7f8b85444640 1 -- 172.21.15.55:0/3617639549 wait complete. 2024-05-20T22:08:22.014 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:08:22.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.128+0000 7fbb101aa640 1 Processor -- start 2024-05-20T22:08:22.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.128+0000 7fbb101aa640 1 -- start start 2024-05-20T22:08:22.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.129+0000 7fbb101aa640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb080b3b30 0x7fbb080b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.129+0000 7fbb101aa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbb0815f100 0x7fbb0806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.129+0000 7fbb101aa640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb080d3dd0 con 0x7fbb0806c5c0 2024-05-20T22:08:22.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.129+0000 7fbb101aa640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fbb080d40b0 con 0x7fbb080b3b30 2024-05-20T22:08:22.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.129+0000 7fbb101aa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbb080d4390 con 0x7fbb0815f100 2024-05-20T22:08:22.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.130+0000 7fbb0df46640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb080b3b30 0x7fbb080b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.130+0000 7fbb0e747640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbb0806c5c0 0x7fbb08162b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46830/0 (socket says 172.21.15.55:46830) 2024-05-20T22:08:22.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.130+0000 7fbb0df46640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb080b3b30 0x7fbb080b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35394/0 (socket says 172.21.15.55:35394) 2024-05-20T22:08:22.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.130+0000 7fbb0e747640 1 -- 172.21.15.55:0/30280540 learned_addr learned my addr 172.21.15.55:0/30280540 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:22.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.130+0000 7fbb0d745640 1 --2- 172.21.15.55:0/30280540 >> v2:172.21.15.138:3300/0 conn(0x7fbb0815f100 0x7fbb0806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.130+0000 7fbb0df46640 1 -- 172.21.15.55:0/30280540 >> v2:172.21.15.138:3300/0 conn(0x7fbb0815f100 msgr2=0x7fbb0806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:22.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.130+0000 7fbb0df46640 1 --2- 172.21.15.55:0/30280540 >> v2:172.21.15.138:3300/0 conn(0x7fbb0815f100 0x7fbb0806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.130+0000 7fbb0df46640 1 -- 172.21.15.55:0/30280540 >> v1:172.21.15.55:6789/0 conn(0x7fbb0806c5c0 legacy=0x7fbb08162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.130+0000 7fbb0df46640 1 -- 172.21.15.55:0/30280540 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0806b5a0 con 0x7fbb080b3b30 2024-05-20T22:08:22.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.130+0000 7fbb0d745640 1 --2- 172.21.15.55:0/30280540 >> v2:172.21.15.138:3300/0 conn(0x7fbb0815f100 0x7fbb0806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:22.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.131+0000 7fbb0df46640 1 --2- 172.21.15.55:0/30280540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb080b3b30 0x7fbb080b7fe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbb080d2eb0 tx=0x7fbb0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415e22f8111c4d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:22.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.131+0000 7fbb0cf44640 1 -- 172.21.15.55:0/30280540 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04037070 con 0x7fbb080b3b30 2024-05-20T22:08:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.131+0000 7fbb0cf44640 1 -- 172.21.15.55:0/30280540 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb04032440 con 0x7fbb080b3b30 2024-05-20T22:08:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.131+0000 7fbb0cf44640 1 -- 172.21.15.55:0/30280540 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04032810 con 0x7fbb080b3b30 2024-05-20T22:08:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.131+0000 7fbb101aa640 1 -- 172.21.15.55:0/30280540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb080b3b30 msgr2=0x7fbb080b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.131+0000 7fbb101aa640 1 --2- 172.21.15.55:0/30280540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb080b3b30 0x7fbb080b7fe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbb080d2eb0 tx=0x7fbb0402a1a0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.132+0000 7fbb101aa640 1 -- 172.21.15.55:0/30280540 shutdown_connections 2024-05-20T22:08:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.132+0000 7fbb101aa640 1 --2- 172.21.15.55:0/30280540 >> v2:172.21.15.138:3300/0 conn(0x7fbb0815f100 0x7fbb0806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.132+0000 7fbb101aa640 1 --2- 172.21.15.55:0/30280540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb080b3b30 0x7fbb080b7fe0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.132+0000 7fbb101aa640 1 -- 172.21.15.55:0/30280540 >> 172.21.15.55:0/30280540 conn(0x7fbb080d51d0 msgr2=0x7fbb080b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.132+0000 7fbb101aa640 1 -- 172.21.15.55:0/30280540 shutdown_connections 2024-05-20T22:08:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.132+0000 7fbb101aa640 1 -- 172.21.15.55:0/30280540 wait complete. 2024-05-20T22:08:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.133+0000 7fbb101aa640 1 Processor -- start 2024-05-20T22:08:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.133+0000 7fbb101aa640 1 -- start start 2024-05-20T22:08:22.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb101aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb0806c5c0 0x7fbb0815af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb101aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb080b3b30 0x7fbb080c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb101aa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb0815f100 0x7fbb080c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb101aa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0806c9d0 con 0x7fbb0815f100 2024-05-20T22:08:22.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb101aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0806bb60 con 0x7fbb0806c5c0 2024-05-20T22:08:22.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb101aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0806cb50 con 0x7fbb080b3b30 2024-05-20T22:08:22.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0e747640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb0815f100 0x7fbb080c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0e747640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb0815f100 0x7fbb080c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35410/0 (socket says 172.21.15.55:35410) 2024-05-20T22:08:22.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0e747640 1 -- 172.21.15.55:0/396011810 learned_addr learned my addr 172.21.15.55:0/396011810 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:22.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0df46640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb0806c5c0 0x7fbb0815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0d745640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb080b3b30 0x7fbb080c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0e747640 1 -- 172.21.15.55:0/396011810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb080b3b30 msgr2=0x7fbb080c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0e747640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb080b3b30 0x7fbb080c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0e747640 1 -- 172.21.15.55:0/396011810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb0806c5c0 msgr2=0x7fbb0815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0e747640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb0806c5c0 0x7fbb0815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0e747640 1 -- 172.21.15.55:0/396011810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0815b750 con 0x7fbb0815f100 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0d745640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb080b3b30 0x7fbb080c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.135+0000 7fbb0e747640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb0815f100 0x7fbb080c8930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000cab0 tx=0x7fbb0000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.136+0000 7fbafe7fc640 1 -- 172.21.15.55:0/396011810 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00017070 con 0x7fbb0815f100 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.136+0000 7fbafe7fc640 1 -- 172.21.15.55:0/396011810 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0000ac30 con 0x7fbb0815f100 2024-05-20T22:08:22.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.136+0000 7fbb101aa640 1 -- 172.21.15.55:0/396011810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0815ba30 con 0x7fbb0815f100 2024-05-20T22:08:22.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.136+0000 7fbafe7fc640 1 -- 172.21.15.55:0/396011810 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00012400 con 0x7fbb0815f100 2024-05-20T22:08:22.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.136+0000 7fbb101aa640 1 -- 172.21.15.55:0/396011810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0815bff0 con 0x7fbb0815f100 2024-05-20T22:08:22.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.137+0000 7fbb101aa640 1 -- 172.21.15.55:0/396011810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb08069090 con 0x7fbb0815f100 2024-05-20T22:08:22.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.137+0000 7fbafe7fc640 1 -- 172.21.15.55:0/396011810 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb0001a070 con 0x7fbb0815f100 2024-05-20T22:08:22.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.138+0000 7fbafe7fc640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbaec06be30 0x7fbaec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.138+0000 7fbb0df46640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbaec06be30 0x7fbaec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.138+0000 7fbafe7fc640 1 -- 172.21.15.55:0/396011810 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbb0008adf0 con 0x7fbb0815f100 2024-05-20T22:08:22.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.138+0000 7fbb0df46640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbaec06be30 0x7fbaec06e340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb04034000 tx=0x7fbb0402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:22.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.142+0000 7fbafe7fc640 1 -- 172.21.15.55:0/396011810 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbb00058d80 con 0x7fbb0815f100 2024-05-20T22:08:22.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.371+0000 7fbb101aa640 1 -- 172.21.15.55:0/396011810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb080d2ee0 con 0x7fbb0815f100 2024-05-20T22:08:22.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.372+0000 7fbafe7fc640 1 -- 172.21.15.55:0/396011810 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7fbb0005d080 con 0x7fbb0815f100 2024-05-20T22:08:22.377 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:22.377 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:07:53.028952+0000","modified":"2024-05-20T22:08:11.135754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-05-20T22:08:22.378 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-20T22:08:22.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.373+0000 7fbb101aa640 1 -- 172.21.15.55:0/396011810 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbaec06be30 msgr2=0x7fbaec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.373+0000 7fbb101aa640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbaec06be30 0x7fbaec06e340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb04034000 tx=0x7fbb0402e000 comp rx=0 tx=0).stop 2024-05-20T22:08:22.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.373+0000 7fbb101aa640 1 -- 172.21.15.55:0/396011810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb0815f100 msgr2=0x7fbb080c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.373+0000 7fbb101aa640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb0815f100 0x7fbb080c8930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000cab0 tx=0x7fbb0000cf70 comp rx=0 tx=0).stop 2024-05-20T22:08:22.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.374+0000 7fbb101aa640 1 -- 172.21.15.55:0/396011810 shutdown_connections 2024-05-20T22:08:22.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.374+0000 7fbb101aa640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbaec06be30 0x7fbaec06e340 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.374+0000 7fbb101aa640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb0815f100 0x7fbb080c8930 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.374+0000 7fbb101aa640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb080b3b30 0x7fbb080c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.374+0000 7fbb101aa640 1 --2- 172.21.15.55:0/396011810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb0806c5c0 0x7fbb0815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.374+0000 7fbb101aa640 1 -- 172.21.15.55:0/396011810 >> 172.21.15.55:0/396011810 conn(0x7fbb080d51d0 msgr2=0x7fbb080b8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:22.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.374+0000 7fbb101aa640 1 -- 172.21.15.55:0/396011810 shutdown_connections 2024-05-20T22:08:22.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.374+0000 7fbb101aa640 1 -- 172.21.15.55:0/396011810 wait complete. 2024-05-20T22:08:22.386 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.500+0000 7fa6e6b38640 1 Processor -- start 2024-05-20T22:08:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.500+0000 7fa6e6b38640 1 -- start start 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.501+0000 7fa6e6b38640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6e00cf870 0x7fa6e00cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.501+0000 7fa6e6b38640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6e00d01c0 con 0x7fa6e00d2d20 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.501+0000 7fa6e6b38640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6e006b5a0 con 0x7fa6e00ba560 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.501+0000 7fa6e6b38640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa6e006c1b0 con 0x7fa6e00cf870 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.502+0000 7fa6e50d5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6e00cf870 0x7fa6e00cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.502+0000 7fa6e48d4640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa6e00d2d20 0x7fa6e00d3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46834/0 (socket says 172.21.15.55:46834) 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.502+0000 7fa6e50d5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6e00cf870 0x7fa6e00cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35422/0 (socket says 172.21.15.55:35422) 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.502+0000 7fa6e48d4640 1 -- 172.21.15.55:0/235933447 learned_addr learned my addr 172.21.15.55:0/235933447 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.502+0000 7fa6e50d5640 1 -- 172.21.15.55:0/235933447 >> v1:172.21.15.138:6789/0 conn(0x7fa6e00ba560 legacy=0x7fa6e006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.502+0000 7fa6e50d5640 1 -- 172.21.15.55:0/235933447 >> v1:172.21.15.55:6789/0 conn(0x7fa6e00d2d20 legacy=0x7fa6e00d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.502+0000 7fa6e50d5640 1 -- 172.21.15.55:0/235933447 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e006b880 con 0x7fa6e00cf870 2024-05-20T22:08:22.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.503+0000 7fa6e50d5640 1 --2- 172.21.15.55:0/235933447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 0x7fa6e00cfc80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009d90 tx=0x7fa6d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf69833652c1060 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.503+0000 7fa6d77fe640 1 -- 172.21.15.55:0/235933447 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8007740 con 0x7fa6e00cf870 2024-05-20T22:08:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.503+0000 7fa6d77fe640 1 -- 172.21.15.55:0/235933447 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d80078e0 con 0x7fa6e00cf870 2024-05-20T22:08:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.503+0000 7fa6d77fe640 1 -- 172.21.15.55:0/235933447 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8007cb0 con 0x7fa6e00cf870 2024-05-20T22:08:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.503+0000 7fa6e6b38640 1 -- 172.21.15.55:0/235933447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 msgr2=0x7fa6e00cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.503+0000 7fa6e6b38640 1 --2- 172.21.15.55:0/235933447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 0x7fa6e00cfc80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009d90 tx=0x7fa6d802a6a0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.504+0000 7fa6e6b38640 1 -- 172.21.15.55:0/235933447 shutdown_connections 2024-05-20T22:08:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.504+0000 7fa6e6b38640 1 --2- 172.21.15.55:0/235933447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 0x7fa6e00cfc80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.504+0000 7fa6e6b38640 1 -- 172.21.15.55:0/235933447 >> 172.21.15.55:0/235933447 conn(0x7fa6e00d1cb0 msgr2=0x7fa6e00d4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.504+0000 7fa6e6b38640 1 -- 172.21.15.55:0/235933447 shutdown_connections 2024-05-20T22:08:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.504+0000 7fa6e6b38640 1 -- 172.21.15.55:0/235933447 wait complete. 2024-05-20T22:08:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.505+0000 7fa6e6b38640 1 Processor -- start 2024-05-20T22:08:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.505+0000 7fa6e6b38640 1 -- start start 2024-05-20T22:08:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6e6b38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e00ba560 0x7fa6e0150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6e6b38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 0x7fa6e00caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6e6b38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d2d20 0x7fa6e00cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6e6b38640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e006cd50 con 0x7fa6e00cf870 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6e6b38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e006be40 con 0x7fa6e00d2d20 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6e6b38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e00ba970 con 0x7fa6e00ba560 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 0x7fa6e00caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 0x7fa6e00caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35426/0 (socket says 172.21.15.55:35426) 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6d7fff640 1 -- 172.21.15.55:0/3181313359 learned_addr learned my addr 172.21.15.55:0/3181313359 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6e48d4640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e00ba560 0x7fa6e0150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6e50d5640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d2d20 0x7fa6e00cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6d7fff640 1 -- 172.21.15.55:0/3181313359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e00ba560 msgr2=0x7fa6e0150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6d7fff640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e00ba560 0x7fa6e0150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6d7fff640 1 -- 172.21.15.55:0/3181313359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d2d20 msgr2=0x7fa6e00cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6d7fff640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d2d20 0x7fa6e00cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6d7fff640 1 -- 172.21.15.55:0/3181313359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e00cbd20 con 0x7fa6e00cf870 2024-05-20T22:08:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.507+0000 7fa6e50d5640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d2d20 0x7fa6e00cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.508+0000 7fa6d7fff640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 0x7fa6e00caf70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000b3e0 tx=0x7fa6d000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.508+0000 7fa6d57fa640 1 -- 172.21.15.55:0/3181313359 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0015020 con 0x7fa6e00cf870 2024-05-20T22:08:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.508+0000 7fa6d57fa640 1 -- 172.21.15.55:0/3181313359 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d0011830 con 0x7fa6e00cf870 2024-05-20T22:08:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.508+0000 7fa6e6b38640 1 -- 172.21.15.55:0/3181313359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e00cc000 con 0x7fa6e00cf870 2024-05-20T22:08:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.508+0000 7fa6d57fa640 1 -- 172.21.15.55:0/3181313359 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0011c20 con 0x7fa6e00cf870 2024-05-20T22:08:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.508+0000 7fa6e6b38640 1 -- 172.21.15.55:0/3181313359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e00cc5c0 con 0x7fa6e00cf870 2024-05-20T22:08:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.509+0000 7fa6e6b38640 1 -- 172.21.15.55:0/3181313359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e00cba90 con 0x7fa6e00cf870 2024-05-20T22:08:22.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.509+0000 7fa6d57fa640 1 -- 172.21.15.55:0/3181313359 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6d0018020 con 0x7fa6e00cf870 2024-05-20T22:08:22.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.510+0000 7fa6d57fa640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6cc06be30 0x7fa6cc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.510+0000 7fa6d57fa640 1 -- 172.21.15.55:0/3181313359 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa6d008b110 con 0x7fa6e00cf870 2024-05-20T22:08:22.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.510+0000 7fa6e48d4640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6cc06be30 0x7fa6cc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.510+0000 7fa6e48d4640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6cc06be30 0x7fa6cc06e340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc000f60 tx=0x7fa6dc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:22.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.514+0000 7fa6d57fa640 1 -- 172.21.15.55:0/3181313359 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa6d001f020 con 0x7fa6e00cf870 2024-05-20T22:08:22.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.699+0000 7fa6e6b38640 1 -- 172.21.15.55:0/3181313359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6e00b7420 con 0x7fa6e00cf870 2024-05-20T22:08:22.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.700+0000 7fa6d57fa640 1 -- 172.21.15.55:0/3181313359 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fa6e00b7420 con 0x7fa6e00cf870 2024-05-20T22:08:22.701 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:22.701 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:22.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 -- 172.21.15.55:0/3181313359 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6cc06be30 msgr2=0x7fa6cc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6cc06be30 0x7fa6cc06e340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc000f60 tx=0x7fa6dc009040 comp rx=0 tx=0).stop 2024-05-20T22:08:22.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 -- 172.21.15.55:0/3181313359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 msgr2=0x7fa6e00caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 0x7fa6e00caf70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000b3e0 tx=0x7fa6d000b8a0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 -- 172.21.15.55:0/3181313359 shutdown_connections 2024-05-20T22:08:22.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6cc06be30 0x7fa6cc06e340 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d2d20 0x7fa6e00cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00cf870 0x7fa6e00caf70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 --2- 172.21.15.55:0/3181313359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e00ba560 0x7fa6e0150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 -- 172.21.15.55:0/3181313359 >> 172.21.15.55:0/3181313359 conn(0x7fa6e00d1cb0 msgr2=0x7fa6e00b9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:22.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 -- 172.21.15.55:0/3181313359 shutdown_connections 2024-05-20T22:08:22.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.702+0000 7fa6e6b38640 1 -- 172.21.15.55:0/3181313359 wait complete. 2024-05-20T22:08:22.715 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:08:22.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.829+0000 7f156d538640 1 Processor -- start 2024-05-20T22:08:22.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.829+0000 7f156d538640 1 -- start start 2024-05-20T22:08:22.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.830+0000 7f156d538640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680d3390 0x7f156806c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f156d538640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f15680b3c90 0x7f15680b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f156d538640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f156806ce50 con 0x7f15680b8550 2024-05-20T22:08:22.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f156d538640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f156806b870 con 0x7f15680b3c90 2024-05-20T22:08:22.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f156d538640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f156806bb50 con 0x7f15680d3390 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f15677fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f15680b8550 0x7f156815e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46848/0 (socket says 172.21.15.55:46848) 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f15677fe640 1 -- 172.21.15.55:0/1031595528 learned_addr learned my addr 172.21.15.55:0/1031595528 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f15667fc640 1 --2- 172.21.15.55:0/1031595528 >> v2:172.21.15.138:3300/0 conn(0x7f15680b3c90 0x7f15680b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f1566ffd640 1 --2- 172.21.15.55:0/1031595528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680d3390 0x7f156806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f1565ffb640 1 -- 172.21.15.55:0/1031595528 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3001948940 0 0) 0x7f156806ce50 con 0x7f15680b8550 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f1565ffb640 1 -- 172.21.15.55:0/1031595528 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1554003520 con 0x7f15680b8550 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f1566ffd640 1 -- 172.21.15.55:0/1031595528 >> v2:172.21.15.138:3300/0 conn(0x7f15680b3c90 msgr2=0x7f15680b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f1566ffd640 1 --2- 172.21.15.55:0/1031595528 >> v2:172.21.15.138:3300/0 conn(0x7f15680b3c90 0x7f15680b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f1565ffb640 1 -- 172.21.15.55:0/1031595528 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1815054234 0 0) 0x7f1554003520 con 0x7f15680b8550 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f1566ffd640 1 -- 172.21.15.55:0/1031595528 >> v1:172.21.15.55:6789/0 conn(0x7f15680b8550 legacy=0x7f156815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f1566ffd640 1 -- 172.21.15.55:0/1031595528 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156806be30 con 0x7f15680d3390 2024-05-20T22:08:22.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.831+0000 7f15667fc640 1 --2- 172.21.15.55:0/1031595528 >> v2:172.21.15.138:3300/0 conn(0x7f15680b3c90 0x7f15680b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:22.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.832+0000 7f1566ffd640 1 --2- 172.21.15.55:0/1031595528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680d3390 0x7f156806c910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1558009d90 tx=0x7f155802a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86bd9318f514a5fb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:22.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.832+0000 7f1565ffb640 1 -- 172.21.15.55:0/1031595528 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560001670 con 0x7f15680d3390 2024-05-20T22:08:22.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.832+0000 7f1565ffb640 1 -- 172.21.15.55:0/1031595528 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1558007540 con 0x7f15680d3390 2024-05-20T22:08:22.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.832+0000 7f1565ffb640 1 -- 172.21.15.55:0/1031595528 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558007910 con 0x7f15680d3390 2024-05-20T22:08:22.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.833+0000 7f156d538640 1 -- 172.21.15.55:0/1031595528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680d3390 msgr2=0x7f156806c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.833+0000 7f156d538640 1 --2- 172.21.15.55:0/1031595528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680d3390 0x7f156806c910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1558009d90 tx=0x7f155802a560 comp rx=0 tx=0).stop 2024-05-20T22:08:22.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.833+0000 7f156d538640 1 -- 172.21.15.55:0/1031595528 shutdown_connections 2024-05-20T22:08:22.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.833+0000 7f156d538640 1 --2- 172.21.15.55:0/1031595528 >> v2:172.21.15.138:3300/0 conn(0x7f15680b3c90 0x7f15680b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.833+0000 7f156d538640 1 --2- 172.21.15.55:0/1031595528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680d3390 0x7f156806c910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.833+0000 7f156d538640 1 -- 172.21.15.55:0/1031595528 >> 172.21.15.55:0/1031595528 conn(0x7f15680b65f0 msgr2=0x7f15680b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:22.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.833+0000 7f156d538640 1 -- 172.21.15.55:0/1031595528 shutdown_connections 2024-05-20T22:08:22.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.833+0000 7f156d538640 1 -- 172.21.15.55:0/1031595528 wait complete. 2024-05-20T22:08:22.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.835+0000 7f156d538640 1 Processor -- start 2024-05-20T22:08:22.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.835+0000 7f156d538640 1 -- start start 2024-05-20T22:08:22.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.836+0000 7f156d538640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15680b3c90 0x7f15680cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.836+0000 7f156d538640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680b8550 0x7f15681e7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f156d538640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15680d3390 0x7f15681e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f156d538640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f156806b500 con 0x7f15680d3390 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f156d538640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f156806c3f0 con 0x7f15680b3c90 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f156d538640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f15680b40a0 con 0x7f15680b8550 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f15677fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15680d3390 0x7f15681e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f15677fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15680d3390 0x7f15681e7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35438/0 (socket says 172.21.15.55:35438) 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f15677fe640 1 -- 172.21.15.55:0/3717124357 learned_addr learned my addr 172.21.15.55:0/3717124357 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f15667fc640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680b8550 0x7f15681e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f1566ffd640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15680b3c90 0x7f15680cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f15677fe640 1 -- 172.21.15.55:0/3717124357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680b8550 msgr2=0x7f15681e7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f15677fe640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680b8550 0x7f15681e7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f15677fe640 1 -- 172.21.15.55:0/3717124357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15680b3c90 msgr2=0x7f15680cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:22.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f15677fe640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15680b3c90 0x7f15680cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:22.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f15677fe640 1 -- 172.21.15.55:0/3717124357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15681524c0 con 0x7f15680d3390 2024-05-20T22:08:22.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f15677fe640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15680d3390 0x7f15681e7d80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f156000c520 tx=0x7f156000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:22.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f155f7fe640 1 -- 172.21.15.55:0/3717124357 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560017070 con 0x7f15680d3390 2024-05-20T22:08:22.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f155f7fe640 1 -- 172.21.15.55:0/3717124357 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1560005ce0 con 0x7f15680d3390 2024-05-20T22:08:22.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f156d538640 1 -- 172.21.15.55:0/3717124357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15680ced60 con 0x7f15680d3390 2024-05-20T22:08:22.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f155f7fe640 1 -- 172.21.15.55:0/3717124357 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15600072b0 con 0x7f15680d3390 2024-05-20T22:08:22.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.837+0000 7f156d538640 1 -- 172.21.15.55:0/3717124357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15680cf320 con 0x7f15680d3390 2024-05-20T22:08:22.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.838+0000 7f156d538640 1 -- 172.21.15.55:0/3717124357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1568069090 con 0x7f15680d3390 2024-05-20T22:08:22.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.839+0000 7f155f7fe640 1 -- 172.21.15.55:0/3717124357 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f156001a070 con 0x7f15680d3390 2024-05-20T22:08:22.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.840+0000 7f155f7fe640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f154806be60 0x7f154806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:22.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.840+0000 7f155f7fe640 1 -- 172.21.15.55:0/3717124357 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f156008a610 con 0x7f15680d3390 2024-05-20T22:08:22.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.840+0000 7f1566ffd640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f154806be60 0x7f154806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:22.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.840+0000 7f1566ffd640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f154806be60 0x7f154806e370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1558034000 tx=0x7f1558033040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:22.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:22.844+0000 7f155f7fe640 1 -- 172.21.15.55:0/3717124357 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1560058bf0 con 0x7f15680d3390 2024-05-20T22:08:23.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.067+0000 7f156d538640 1 -- 172.21.15.55:0/3717124357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15680cc270 con 0x7f15680d3390 2024-05-20T22:08:23.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.068+0000 7f155f7fe640 1 -- 172.21.15.55:0/3717124357 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f15600584f0 con 0x7f15680d3390 2024-05-20T22:08:23.069 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:23.069 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:07:53.028952+0000","modified":"2024-05-20T22:08:11.135754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-05-20T22:08:23.070 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-20T22:08:23.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 -- 172.21.15.55:0/3717124357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f154806be60 msgr2=0x7f154806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f154806be60 0x7f154806e370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1558034000 tx=0x7f1558033040 comp rx=0 tx=0).stop 2024-05-20T22:08:23.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 -- 172.21.15.55:0/3717124357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15680d3390 msgr2=0x7f15681e7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15680d3390 0x7f15681e7d80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f156000c520 tx=0x7f156000c9e0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 -- 172.21.15.55:0/3717124357 shutdown_connections 2024-05-20T22:08:23.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f154806be60 0x7f154806e370 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15680d3390 0x7f15681e7d80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15680b8550 0x7f15681e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 --2- 172.21.15.55:0/3717124357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15680b3c90 0x7f15680cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 -- 172.21.15.55:0/3717124357 >> 172.21.15.55:0/3717124357 conn(0x7f15680b65f0 msgr2=0x7f15680b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 -- 172.21.15.55:0/3717124357 shutdown_connections 2024-05-20T22:08:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.070+0000 7f156d538640 1 -- 172.21.15.55:0/3717124357 wait complete. 2024-05-20T22:08:23.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:07:53.028952+0000', 'modified': '2024-05-20T22:08:11.135754+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6835/2341014737', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6833', 'nonce': 2341014737}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 2341014737}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4288, 'qdb_cluster': [4288]} 2024-05-20T22:08:23.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T22:08:23.486 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-20T22:08:23.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.607+0000 7efc278ca640 1 Processor -- start 2024-05-20T22:08:23.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.607+0000 7efc278ca640 1 -- start start 2024-05-20T22:08:23.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.608+0000 7efc278ca640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc2006d3b0 0x7efc200d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.608+0000 7efc278ca640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc200d2040 con 0x7efc200d1c30 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.608+0000 7efc278ca640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc200cf000 con 0x7efc2015d170 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.608+0000 7efc278ca640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efc200ce760 con 0x7efc2006d3b0 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.608+0000 7efc25e67640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efc200d1c30 0x7efc20166b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46864/0 (socket says 172.21.15.55:46864) 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.608+0000 7efc25e67640 1 -- 172.21.15.55:0/2893554377 learned_addr learned my addr 172.21.15.55:0/2893554377 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.609+0000 7efc24e65640 1 --2- 172.21.15.55:0/2893554377 >> v2:172.21.15.138:3300/0 conn(0x7efc2006d3b0 0x7efc200d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.609+0000 7efc1ffff640 1 -- 172.21.15.55:0/2893554377 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881155974 0 0) 0x7efc18001110 con 0x7efc200d1c30 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.609+0000 7efc1ffff640 1 -- 172.21.15.55:0/2893554377 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc08003520 con 0x7efc200d1c30 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.609+0000 7efc24e65640 1 -- 172.21.15.55:0/2893554377 >> v1:172.21.15.138:6789/0 conn(0x7efc2015d170 legacy=0x7efc200cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.609+0000 7efc24e65640 1 -- 172.21.15.55:0/2893554377 >> v1:172.21.15.55:6789/0 conn(0x7efc200d1c30 legacy=0x7efc20166b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.609+0000 7efc24e65640 1 -- 172.21.15.55:0/2893554377 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc200cea40 con 0x7efc2006d3b0 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.610+0000 7efc24e65640 1 --2- 172.21.15.55:0/2893554377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc2006d3b0 0x7efc200d16f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc14009d90 tx=0x7efc1402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0e4b9eb4d445547 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.610+0000 7efc1ffff640 1 -- 172.21.15.55:0/2893554377 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14007740 con 0x7efc2006d3b0 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.610+0000 7efc1ffff640 1 -- 172.21.15.55:0/2893554377 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc140078e0 con 0x7efc2006d3b0 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.610+0000 7efc1ffff640 1 -- 172.21.15.55:0/2893554377 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14007c90 con 0x7efc2006d3b0 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.611+0000 7efc278ca640 1 -- 172.21.15.55:0/2893554377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc2006d3b0 msgr2=0x7efc200d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.611+0000 7efc278ca640 1 --2- 172.21.15.55:0/2893554377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc2006d3b0 0x7efc200d16f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc14009d90 tx=0x7efc1402a660 comp rx=0 tx=0).stop 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.611+0000 7efc278ca640 1 -- 172.21.15.55:0/2893554377 shutdown_connections 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.611+0000 7efc278ca640 1 --2- 172.21.15.55:0/2893554377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc2006d3b0 0x7efc200d16f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.611+0000 7efc278ca640 1 -- 172.21.15.55:0/2893554377 >> 172.21.15.55:0/2893554377 conn(0x7efc200736f0 msgr2=0x7efc200b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.611+0000 7efc278ca640 1 -- 172.21.15.55:0/2893554377 shutdown_connections 2024-05-20T22:08:23.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.611+0000 7efc278ca640 1 -- 172.21.15.55:0/2893554377 wait complete. 2024-05-20T22:08:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.613+0000 7efc278ca640 1 Processor -- start 2024-05-20T22:08:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.613+0000 7efc278ca640 1 -- start start 2024-05-20T22:08:23.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc278ca640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc2006d3b0 0x7efc200ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc278ca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc200d1c30 0x7efc2014c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc278ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc2015d170 0x7efc2014bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc278ca640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efc2009aff0 con 0x7efc2006d3b0 2024-05-20T22:08:23.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc278ca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc200ce3f0 con 0x7efc2015d170 2024-05-20T22:08:23.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc278ca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efc2009b170 con 0x7efc200d1c30 2024-05-20T22:08:23.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc25666640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc2006d3b0 0x7efc200ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc25666640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc2006d3b0 0x7efc200ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35440/0 (socket says 172.21.15.55:35440) 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc25666640 1 -- 172.21.15.55:0/3744049989 learned_addr learned my addr 172.21.15.55:0/3744049989 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc24e65640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc200d1c30 0x7efc2014c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc25e67640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc2015d170 0x7efc2014bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc25666640 1 -- 172.21.15.55:0/3744049989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc200d1c30 msgr2=0x7efc2014c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc25666640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc200d1c30 0x7efc2014c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc25666640 1 -- 172.21.15.55:0/3744049989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc2015d170 msgr2=0x7efc2014bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc25666640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc2015d170 0x7efc2014bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc25666640 1 -- 172.21.15.55:0/3744049989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2009d7b0 con 0x7efc2006d3b0 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.614+0000 7efc24e65640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc200d1c30 0x7efc2014c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.615+0000 7efc25666640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc2006d3b0 0x7efc200ae4b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c3f0 tx=0x7efc1000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.615+0000 7efc1dffb640 1 -- 172.21.15.55:0/3744049989 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10017070 con 0x7efc2006d3b0 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.615+0000 7efc1dffb640 1 -- 172.21.15.55:0/3744049989 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc100052d0 con 0x7efc2006d3b0 2024-05-20T22:08:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.615+0000 7efc1dffb640 1 -- 172.21.15.55:0/3744049989 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10003830 con 0x7efc2006d3b0 2024-05-20T22:08:23.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.615+0000 7efc278ca640 1 -- 172.21.15.55:0/3744049989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2009da90 con 0x7efc2006d3b0 2024-05-20T22:08:23.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.615+0000 7efc278ca640 1 -- 172.21.15.55:0/3744049989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2014f090 con 0x7efc2006d3b0 2024-05-20T22:08:23.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.617+0000 7efc1dffb640 1 -- 172.21.15.55:0/3744049989 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc1001a070 con 0x7efc2006d3b0 2024-05-20T22:08:23.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.617+0000 7efc1dffb640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efc0c06be60 0x7efc0c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.617+0000 7efc1dffb640 1 -- 172.21.15.55:0/3744049989 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7efc1008a510 con 0x7efc2006d3b0 2024-05-20T22:08:23.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.617+0000 7efc24e65640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efc0c06be60 0x7efc0c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.618+0000 7efc24e65640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efc0c06be60 0x7efc0c06e370 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efc14034000 tx=0x7efc14033040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:23.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.618+0000 7efc2e76e740 1 Processor -- start 2024-05-20T22:08:23.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.618+0000 7efc2e76e740 1 -- start start 2024-05-20T22:08:23.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a155a270 0x5624a1925700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a19220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a1539590 0x5624a18fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc2e76e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5624a185e4d0 con 0x5624a155a270 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc2e76e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5624a18063e0 con 0x5624a1539590 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc2e76e740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5624a185a490 con 0x5624a15313b0 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc25666640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a155a270 0x5624a1925700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc25666640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a155a270 0x5624a1925700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35450/0 (socket says 172.21.15.55:35450) 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc25666640 1 -- 172.21.15.55:0/393630299 learned_addr learned my addr 172.21.15.55:0/393630299 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc25e67640 1 --2- 172.21.15.55:0/393630299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a1539590 0x5624a18fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc24e65640 1 --2- 172.21.15.55:0/393630299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a19220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc25666640 1 -- 172.21.15.55:0/393630299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 msgr2=0x5624a19220a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc25666640 1 --2- 172.21.15.55:0/393630299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a19220a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc25666640 1 -- 172.21.15.55:0/393630299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a1539590 msgr2=0x5624a18fb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc25666640 1 --2- 172.21.15.55:0/393630299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a1539590 0x5624a18fb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc25666640 1 -- 172.21.15.55:0/393630299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624a15722d0 con 0x5624a155a270 2024-05-20T22:08:23.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc25666640 1 --2- 172.21.15.55:0/393630299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a155a270 0x5624a1925700 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc10088670 tx=0x7efc10058060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1413db79f73808d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:23.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc06ffd640 1 -- 172.21.15.55:0/393630299 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10061060 con 0x5624a155a270 2024-05-20T22:08:23.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc06ffd640 1 -- 172.21.15.55:0/393630299 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc100566f0 con 0x5624a155a270 2024-05-20T22:08:23.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.620+0000 7efc06ffd640 1 -- 172.21.15.55:0/393630299 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1008a070 con 0x5624a155a270 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.621+0000 7efc2e76e740 1 -- 172.21.15.55:0/393630299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a155a270 msgr2=0x5624a1925700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.621+0000 7efc2e76e740 1 --2- 172.21.15.55:0/393630299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a155a270 0x5624a1925700 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc10088670 tx=0x7efc10058060 comp rx=0 tx=0).stop 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.621+0000 7efc2e76e740 1 -- 172.21.15.55:0/393630299 shutdown_connections 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.621+0000 7efc2e76e740 1 --2- 172.21.15.55:0/393630299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a1539590 0x5624a18fb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.621+0000 7efc2e76e740 1 --2- 172.21.15.55:0/393630299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a19220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.621+0000 7efc2e76e740 1 --2- 172.21.15.55:0/393630299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a155a270 0x5624a1925700 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.621+0000 7efc2e76e740 1 -- 172.21.15.55:0/393630299 >> 172.21.15.55:0/393630299 conn(0x5624a154e3c0 msgr2=0x5624a18faaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.621+0000 7efc2e76e740 1 -- 172.21.15.55:0/393630299 shutdown_connections 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.621+0000 7efc2e76e740 1 -- 172.21.15.55:0/393630299 wait complete. 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.622+0000 7efc2e76e740 1 -- start start 2024-05-20T22:08:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.622+0000 7efc2e76e740 1 Processor -- start 2024-05-20T22:08:23.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a191ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a1928f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 0x5624a1940430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc2e76e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5624a18198e0 con 0x5624a1539590 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc2e76e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5624a193ef20 con 0x5624a155a270 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc2e76e740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5624a194f4e0 con 0x5624a15313b0 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc2e76e740 10 client.0 _ll_register_callbacks cb 0x5624a17e69c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc24e65640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a1928f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc24e65640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a1928f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35462/0 (socket says 172.21.15.55:35462) 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc24e65640 1 -- 172.21.15.55:0/1076350663 learned_addr learned my addr 172.21.15.55:0/1076350663 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc25666640 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a191ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc25e67640 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 0x5624a1940430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc24e65640 1 -- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 msgr2=0x5624a191ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc24e65640 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a191ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc24e65640 1 -- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 msgr2=0x5624a1940430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc24e65640 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 0x5624a1940430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc24e65640 1 -- 172.21.15.55:0/1076350663 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624a163c6b0 con 0x5624a1539590 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc25e67640 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 0x5624a1940430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efc24e65640 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a1928f80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc14009670 tx=0x7efc14004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd7e02b841f7a79 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.624+0000 7efbfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efbfbfff640 1 -- 172.21.15.55:0/1076350663 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1404c030 con 0x5624a1539590 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efbfbfff640 1 -- 172.21.15.55:0/1076350663 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7efc1403cd00 con 0x5624a1539590 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efc2e76e740 1 -- 172.21.15.55:0/1076350663 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc100566f0 con 0x5624a1539590 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efbfbfff640 1 -- 172.21.15.55:0/1076350663 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7efc14058050 con 0x5624a1539590 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efc2e76e740 10 client.4863 fetch_fsmap learned FSMap version 9 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efc2e76e740 1 -- 172.21.15.55:0/1076350663 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5624a1921f20 con 0x5624a1539590 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efbfbfff640 1 -- 172.21.15.55:0/1076350663 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc100566f0 con 0x5624a1539590 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efbfbfff640 1 -- 172.21.15.55:0/1076350663 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7efc1405b020 con 0x5624a1539590 2024-05-20T22:08:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efc2e76e740 10 client.4863 fetch_fsmap finished waiting for FSMap version 9 2024-05-20T22:08:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efc2e76e740 10 client.4863 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-20T22:08:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efc2e76e740 20 client.4863 populate_metadata read hostname 'smithi055' 2024-05-20T22:08:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efc2e76e740 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x5624a1518e40 0x5624a1921490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efc2e76e740 4 client.4863 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T22:08:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efc2e76e740 1 -- 172.21.15.55:0/1076350663 --> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624a186bc00 con 0x5624a1518e40 2024-05-20T22:08:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.625+0000 7efc25666640 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x5624a1518e40 0x5624a1921490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.626+0000 7efc25666640 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x5624a1518e40 0x5624a1921490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4bea8ff2f9a0ce3 server_cookie=31f375af859989c4 in_seq=0 out_seq=0 2024-05-20T22:08:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.626+0000 7efbfbfff640 10 client.4863 ms_handle_connect on v2:172.21.15.138:6833/2341014737 2024-05-20T22:08:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efbfbfff640 1 -- 172.21.15.55:0/1076350663 <== mds.0 v2:172.21.15.138:6833/2341014737 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5624a186bc00 con 0x5624a1518e40 2024-05-20T22:08:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efbfbfff640 10 client.4863 handle_command_reply: tid=0 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 client.4863 shutdown 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 -- 172.21.15.55:0/1076350663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 msgr2=0x5624a1928f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a1928f80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc14009670 tx=0x7efc14004ab0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 -- 172.21.15.55:0/1076350663 shutdown_connections 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x5624a1518e40 0x5624a1921490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 0x5624a1940430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a1928f80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efbfbfff640 0 client.4863 ms_handle_reset on v2:172.21.15.138:6833/2341014737 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 --2- 172.21.15.55:0/1076350663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a191ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 -- 172.21.15.55:0/1076350663 >> 172.21.15.55:0/1076350663 conn(0x5624a154e3c0 msgr2=0x5624a18f9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 -- 172.21.15.55:0/1076350663 shutdown_connections 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 1 -- 172.21.15.55:0/1076350663 wait complete. 2024-05-20T22:08:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.627+0000 7efc2e76e740 20 client.4863 trim_cache size 0 max 16384 2024-05-20T22:08:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.646+0000 7efc2e76e740 1 Processor -- start 2024-05-20T22:08:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.646+0000 7efc2e76e740 1 -- start start 2024-05-20T22:08:23.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.647+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a19258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.647+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a193fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.647+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 0x5624a1940340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.647+0000 7efc2e76e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efc202000e0 con 0x5624a1539590 2024-05-20T22:08:23.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.647+0000 7efc2e76e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc201ff440 con 0x5624a155a270 2024-05-20T22:08:23.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.647+0000 7efc2e76e740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efc201ffa90 con 0x5624a15313b0 2024-05-20T22:08:23.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.647+0000 7efc24e65640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a193fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.647+0000 7efc24e65640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a193fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35476/0 (socket says 172.21.15.55:35476) 2024-05-20T22:08:23.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.647+0000 7efc24e65640 1 -- 172.21.15.55:0/4246263876 learned_addr learned my addr 172.21.15.55:0/4246263876 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:23.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc25e67640 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 0x5624a1940340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc24e65640 1 -- 172.21.15.55:0/4246263876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 msgr2=0x5624a19258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc25666640 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a19258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc24e65640 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a19258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc24e65640 1 -- 172.21.15.55:0/4246263876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 msgr2=0x5624a1940340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc24e65640 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 0x5624a1940340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc24e65640 1 -- 172.21.15.55:0/4246263876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624a18f6d90 con 0x5624a1539590 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc25666640 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a19258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc25e67640 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 0x5624a1940340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc24e65640 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a193fe00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc14000f80 tx=0x7efc14048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc8bc398a5b4814 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc067fc640 1 -- 172.21.15.55:0/4246263876 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1404c030 con 0x5624a1539590 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc067fc640 1 -- 172.21.15.55:0/4246263876 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc14061070 con 0x5624a1539590 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.648+0000 7efc067fc640 1 -- 172.21.15.55:0/4246263876 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14040070 con 0x5624a1539590 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.649+0000 7efc2e76e740 1 -- 172.21.15.55:0/4246263876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 msgr2=0x5624a193fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.649+0000 7efc2e76e740 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a193fe00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc14000f80 tx=0x7efc14048000 comp rx=0 tx=0).stop 2024-05-20T22:08:23.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.649+0000 7efc2e76e740 1 -- 172.21.15.55:0/4246263876 shutdown_connections 2024-05-20T22:08:23.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.649+0000 7efc2e76e740 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a155a270 0x5624a1940340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.649+0000 7efc2e76e740 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a1539590 0x5624a193fe00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.649+0000 7efc2e76e740 1 --2- 172.21.15.55:0/4246263876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a15313b0 0x5624a19258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.649+0000 7efc2e76e740 1 -- 172.21.15.55:0/4246263876 >> 172.21.15.55:0/4246263876 conn(0x5624a154e3c0 msgr2=0x5624a1922590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:23.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.649+0000 7efc2e76e740 1 -- 172.21.15.55:0/4246263876 shutdown_connections 2024-05-20T22:08:23.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.649+0000 7efc2e76e740 1 -- 172.21.15.55:0/4246263876 wait complete. 2024-05-20T22:08:23.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.650+0000 7efc2e76e740 1 -- start start 2024-05-20T22:08:23.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.650+0000 7efc2e76e740 1 Processor -- start 2024-05-20T22:08:23.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.651+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a15313b0 0x5624a19c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.651+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a1539590 0x5624a1943bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.651+0000 7efc2e76e740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a155a270 0x5624a193fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.651+0000 7efc2e76e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5624a1928030 con 0x5624a15313b0 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.651+0000 7efc2e76e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5624a193eb80 con 0x5624a1539590 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.651+0000 7efc2e76e740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5624a1928650 con 0x5624a155a270 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.651+0000 7efc2e76e740 10 client.0 _ll_register_callbacks cb 0x7efc14009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.651+0000 7efc25666640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a15313b0 0x5624a19c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.651+0000 7efc25666640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a15313b0 0x5624a19c05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35482/0 (socket says 172.21.15.55:35482) 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.651+0000 7efc25666640 1 -- 172.21.15.55:0/129762706 learned_addr learned my addr 172.21.15.55:0/129762706 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc25e67640 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a155a270 0x5624a193fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc24e65640 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a1539590 0x5624a1943bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc25666640 1 -- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a155a270 msgr2=0x5624a193fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc25666640 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a155a270 0x5624a193fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc25666640 1 -- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a1539590 msgr2=0x5624a1943bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc25666640 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a1539590 0x5624a1943bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc25666640 1 -- 172.21.15.55:0/129762706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624a191f670 con 0x5624a15313b0 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc25666640 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a15313b0 0x5624a19c05c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efc100585b0 tx=0x7efc10014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff051fcf56c63834 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efbfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efbfbfff640 1 -- 172.21.15.55:0/129762706 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc100581c0 con 0x5624a15313b0 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efbfbfff640 1 -- 172.21.15.55:0/129762706 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7efc10054a60 con 0x5624a15313b0 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc2e76e740 1 -- 172.21.15.55:0/129762706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc14061070 con 0x5624a15313b0 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efbfbfff640 1 -- 172.21.15.55:0/129762706 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7efc10088190 con 0x5624a15313b0 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc2e76e740 10 client.4869 fetch_fsmap learned FSMap version 9 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efc2e76e740 1 -- 172.21.15.55:0/129762706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5624a1922c80 con 0x5624a15313b0 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.652+0000 7efbfbfff640 1 -- 172.21.15.55:0/129762706 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc14061070 con 0x5624a15313b0 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.653+0000 7efbfbfff640 1 -- 172.21.15.55:0/129762706 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7efc10057840 con 0x5624a15313b0 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.653+0000 7efc2e76e740 10 client.4869 fetch_fsmap finished waiting for FSMap version 9 2024-05-20T22:08:23.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.653+0000 7efc2e76e740 10 client.4869 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.653+0000 7efc2e76e740 20 client.4869 populate_metadata read hostname 'smithi055' 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.653+0000 7efc2e76e740 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x5624a1940110 0x5624a191d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.653+0000 7efc2e76e740 4 client.4869 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.653+0000 7efc2e76e740 1 -- 172.21.15.55:0/129762706 --> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] -- command(tid 0: {"prefix": "status"}) -- 0x5624a1865f50 con 0x5624a1940110 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.653+0000 7efc24e65640 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x5624a1940110 0x5624a191d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.653+0000 7efbfbfff640 10 client.4869 ms_handle_connect on v2:172.21.15.138:6833/2341014737 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.653+0000 7efc24e65640 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x5624a1940110 0x5624a191d2d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34172a9ce981611d server_cookie=a15b1e6244e5a107 in_seq=0 out_seq=0 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efbfbfff640 1 -- 172.21.15.55:0/129762706 <== mds.0 v2:172.21.15.138:6833/2341014737 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5624a1865f50 con 0x5624a1940110 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efbfbfff640 10 client.4869 handle_command_reply: tid=0 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 client.4869 shutdown 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 -- 172.21.15.55:0/129762706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a15313b0 msgr2=0x5624a19c05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a15313b0 0x5624a19c05c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efc100585b0 tx=0x7efc10014040 comp rx=0 tx=0).stop 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 -- 172.21.15.55:0/129762706 shutdown_connections 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:6833/2341014737,v1:172.21.15.138:6835/2341014737] conn(0x5624a1940110 0x5624a191d2d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5624a155a270 0x5624a193fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efbfbfff640 0 client.4869 ms_handle_reset on v2:172.21.15.138:6833/2341014737 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a1539590 0x5624a1943bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 --2- 172.21.15.55:0/129762706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5624a15313b0 0x5624a19c05c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 -- 172.21.15.55:0/129762706 >> 172.21.15.55:0/129762706 conn(0x5624a154e3c0 msgr2=0x5624a193ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 -- 172.21.15.55:0/129762706 shutdown_connections 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 1 -- 172.21.15.55:0/129762706 wait complete. 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.654+0000 7efc2e76e740 20 client.4869 trim_cache size 0 max 16384 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "de8ca569-3fd1-4cb2-8bdc-99f367745927", 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 4288, 2024-05-20T22:08:23.660 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-20T22:08:23.661 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-20T22:08:23.661 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-20T22:08:23.661 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 30.012596744, 2024-05-20T22:08:23.661 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 9, 2024-05-20T22:08:23.661 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 24, 2024-05-20T22:08:23.661 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 24, 2024-05-20T22:08:23.661 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 37.030585053000003 2024-05-20T22:08:23.661 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:08:23.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 -- 172.21.15.55:0/3744049989 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efc0c06be60 msgr2=0x7efc0c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efc0c06be60 0x7efc0c06e370 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efc14034000 tx=0x7efc14033040 comp rx=0 tx=0).stop 2024-05-20T22:08:23.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 -- 172.21.15.55:0/3744049989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc2006d3b0 msgr2=0x7efc200ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:23.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc2006d3b0 0x7efc200ae4b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c3f0 tx=0x7efc1000c8b0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 -- 172.21.15.55:0/3744049989 shutdown_connections 2024-05-20T22:08:23.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efc0c06be60 0x7efc0c06e370 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc2015d170 0x7efc2014bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc200d1c30 0x7efc2014c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 --2- 172.21.15.55:0/3744049989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc2006d3b0 0x7efc200ae4b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:23.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 -- 172.21.15.55:0/3744049989 >> 172.21.15.55:0/3744049989 conn(0x7efc200736f0 msgr2=0x7efc20070790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:23.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 -- 172.21.15.55:0/3744049989 shutdown_connections 2024-05-20T22:08:23.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:23.655+0000 7efc278ca640 1 -- 172.21.15.55:0/3744049989 wait complete. 2024-05-20T22:08:23.667 INFO:teuthology.run_tasks:Running task sequential... 2024-05-20T22:08:23.676 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-20T22:08:23.676 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-20T22:08:23.676 INFO:teuthology.task.print:Enabling mgr modules 2024-05-20T22:08:23.676 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-20T22:08:23.741 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-20T22:08:23.741 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-20T22:08:23.742 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-20T22:08:23.742 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T22:08:23.742 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T22:08:23.742 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-20T22:08:23.742 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T22:08:23.742 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-20T22:08:23.742 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-20T22:08:23.742 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T22:08:23.742 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-20T22:08:23.742 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-20T22:08:23.742 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T22:08:23.743 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-20T22:08:23.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:23.743 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-20T22:08:23.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:23.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-05-20T22:08:23.835 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-05-20T22:08:23.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:08:23.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:23.837 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-20T22:08:23.854 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-20T22:08:23.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:23.854 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T22:08:23.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:23.923 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2024-05-20T22:08:24.026 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-20T22:08:24.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T22:08:24.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.142+0000 7fe102e61640 1 Processor -- start 2024-05-20T22:08:24.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.142+0000 7fe102e61640 1 -- start start 2024-05-20T22:08:24.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.143+0000 7fe102e61640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc0cfaf0 0x7fe0fc0ad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:24.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.143+0000 7fe102e61640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0fc0cc970 0x7fe0fc15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:24.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.143+0000 7fe102e61640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0fc0d0480 con 0x7fe0fc15f850 2024-05-20T22:08:24.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.143+0000 7fe102e61640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe0fc0ccd80 con 0x7fe0fc0cc970 2024-05-20T22:08:24.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.143+0000 7fe102e61640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe0fc160370 con 0x7fe0fc0cfaf0 2024-05-20T22:08:24.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe1013fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe0fc15f850 0x7fe0fc15fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46866/0 (socket says 172.21.15.55:46866) 2024-05-20T22:08:24.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe1013fe640 1 -- 172.21.15.55:0/2968963030 learned_addr learned my addr 172.21.15.55:0/2968963030 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:24.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe100bfd640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc0cfaf0 0x7fe0fc0ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f3fff640 1 --2- 172.21.15.55:0/2968963030 >> v2:172.21.15.138:3300/0 conn(0x7fe0fc0cc970 0x7fe0fc15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 205787230 0 0) 0x7fe0f4001490 con 0x7fe0fc15f850 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0e4003520 con 0x7fe0fc15f850 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3010975823 0 0) 0x7fe0e4003520 con 0x7fe0fc15f850 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc0cfaf0 msgr2=0x7fe0fc0ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc0cfaf0 0x7fe0fc0ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 >> v2:172.21.15.138:3300/0 conn(0x7fe0fc0cc970 msgr2=0x7fe0fc15bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 --2- 172.21.15.55:0/2968963030 >> v2:172.21.15.138:3300/0 conn(0x7fe0fc0cc970 0x7fe0fc15bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc1605d0 con 0x7fe0fc15f850 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fe0f4001bc0 con 0x7fe0fc15f850 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe100bfd640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc0cfaf0 0x7fe0fc0ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f3fff640 1 --2- 172.21.15.55:0/2968963030 >> v2:172.21.15.138:3300/0 conn(0x7fe0fc0cc970 0x7fe0fc15bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:24.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 >> v1:172.21.15.55:6789/0 conn(0x7fe0fc15f850 legacy=0x7fe0fc15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 0x7fe0e4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc0cfaf0 0x7fe0e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4004d40 0x7fe0e4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc0ccd80 con 0x7fe0e40039e0 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc160370 con 0x7fe0e4004d40 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e400baf0 con 0x7fe0fc0cfaf0 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.144+0000 7fe100bfd640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 0x7fe0e4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.145+0000 7fe0f3fff640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc0cfaf0 0x7fe0e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.145+0000 7fe1013fe640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4004d40 0x7fe0e4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.145+0000 7fe100bfd640 1 -- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc0cfaf0 msgr2=0x7fe0e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.145+0000 7fe100bfd640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc0cfaf0 0x7fe0e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.145+0000 7fe100bfd640 1 -- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4004d40 msgr2=0x7fe0e4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.145+0000 7fe100bfd640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4004d40 0x7fe0e4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:24.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.145+0000 7fe100bfd640 1 -- 172.21.15.55:0/2968963030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e4003520 con 0x7fe0e40039e0 2024-05-20T22:08:24.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.145+0000 7fe1013fe640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4004d40 0x7fe0e4005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:24.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.145+0000 7fe100bfd640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 0x7fe0e4003e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009c60 tx=0x7fe0e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31ec5cb661ec230 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:24.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.146+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8009860 con 0x7fe0e40039e0 2024-05-20T22:08:24.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.146+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0f8004020 con 0x7fe0e40039e0 2024-05-20T22:08:24.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.146+0000 7fe0f37fe640 1 -- 172.21.15.55:0/2968963030 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80043f0 con 0x7fe0e40039e0 2024-05-20T22:08:24.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.146+0000 7fe102e61640 1 -- 172.21.15.55:0/2968963030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 msgr2=0x7fe0e4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.146+0000 7fe102e61640 1 --2- 172.21.15.55:0/2968963030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 0x7fe0e4003e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009c60 tx=0x7fe0e4005830 comp rx=0 tx=0).stop 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.146+0000 7fe1013fe640 1 -- 172.21.15.55:0/2968963030 reap_dead start 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.146+0000 7fe102e61640 1 -- 172.21.15.55:0/2968963030 shutdown_connections 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.146+0000 7fe102e61640 1 -- 172.21.15.55:0/2968963030 >> 172.21.15.55:0/2968963030 conn(0x7fe0fc0cd4d0 msgr2=0x7fe0fc0cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.146+0000 7fe102e61640 1 -- 172.21.15.55:0/2968963030 shutdown_connections 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.146+0000 7fe102e61640 1 -- 172.21.15.55:0/2968963030 wait complete. 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.148+0000 7fe102e61640 1 Processor -- start 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.148+0000 7fe102e61640 1 -- start start 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe102e61640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 0x7fe0fc1f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe102e61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc0b50e0 0x7fe0fc0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe102e61640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc15d9d0 0x7fe0fc15dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe102e61640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc0cb480 con 0x7fe0e40039e0 2024-05-20T22:08:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe102e61640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc0cb1a0 con 0x7fe0fc0b50e0 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe102e61640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe0fc15c060 con 0x7fe0fc15d9d0 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe100bfd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 0x7fe0fc1f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe100bfd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 0x7fe0fc1f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35506/0 (socket says 172.21.15.55:35506) 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe100bfd640 1 -- 172.21.15.55:0/718677239 learned_addr learned my addr 172.21.15.55:0/718677239 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe0f3fff640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc0b50e0 0x7fe0fc0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe1013fe640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc15d9d0 0x7fe0fc15dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe100bfd640 1 -- 172.21.15.55:0/718677239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc15d9d0 msgr2=0x7fe0fc15dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe100bfd640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc15d9d0 0x7fe0fc15dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe100bfd640 1 -- 172.21.15.55:0/718677239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc0b50e0 msgr2=0x7fe0fc0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe100bfd640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc0b50e0 0x7fe0fc0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe100bfd640 1 -- 172.21.15.55:0/718677239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc15e320 con 0x7fe0e40039e0 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe0f3fff640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc0b50e0 0x7fe0fc0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe100bfd640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 0x7fe0fc1f5280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8033040 tx=0x7fe0f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:24.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe0f17fa640 1 -- 172.21.15.55:0/718677239 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8040070 con 0x7fe0e40039e0 2024-05-20T22:08:24.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe0f17fa640 1 -- 172.21.15.55:0/718677239 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0f80093e0 con 0x7fe0e40039e0 2024-05-20T22:08:24.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe102e61640 1 -- 172.21.15.55:0/718677239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc0afbb0 con 0x7fe0e40039e0 2024-05-20T22:08:24.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.150+0000 7fe0f17fa640 1 -- 172.21.15.55:0/718677239 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8003850 con 0x7fe0e40039e0 2024-05-20T22:08:24.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.151+0000 7fe102e61640 1 -- 172.21.15.55:0/718677239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc0b0170 con 0x7fe0e40039e0 2024-05-20T22:08:24.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.152+0000 7fe102e61640 1 -- 172.21.15.55:0/718677239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0fc0ca940 con 0x7fe0e40039e0 2024-05-20T22:08:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.152+0000 7fe0f17fa640 1 -- 172.21.15.55:0/718677239 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0f8043070 con 0x7fe0e40039e0 2024-05-20T22:08:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.153+0000 7fe0f17fa640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe0e806be30 0x7fe0e806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.153+0000 7fe0f17fa640 1 -- 172.21.15.55:0/718677239 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe0f80ab9c0 con 0x7fe0e40039e0 2024-05-20T22:08:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.153+0000 7fe0f3fff640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe0e806be30 0x7fe0e806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.153+0000 7fe0f3fff640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe0e806be30 0x7fe0e806e340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0b1b50 tx=0x7fe0ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.157+0000 7fe0f17fa640 1 -- 172.21.15.55:0/718677239 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe0fc0ca940 con 0x7fe0e40039e0 2024-05-20T22:08:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.379+0000 7fe102e61640 1 -- 172.21.15.55:0/718677239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe0fc069090 con 0x7fe0e40039e0 2024-05-20T22:08:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.380+0000 7fe0f17fa640 1 -- 172.21.15.55:0/718677239 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fe0f80ade20 con 0x7fe0e40039e0 2024-05-20T22:08:24.381 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T22:08:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 -- 172.21.15.55:0/718677239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe0e806be30 msgr2=0x7fe0e806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe0e806be30 0x7fe0e806e340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0b1b50 tx=0x7fe0ec008040 comp rx=0 tx=0).stop 2024-05-20T22:08:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 -- 172.21.15.55:0/718677239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 msgr2=0x7fe0fc1f5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 0x7fe0fc1f5280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8033040 tx=0x7fe0f8004730 comp rx=0 tx=0).stop 2024-05-20T22:08:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 -- 172.21.15.55:0/718677239 shutdown_connections 2024-05-20T22:08:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe0e806be30 0x7fe0e806e340 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0fc15d9d0 0x7fe0fc15dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc0b50e0 0x7fe0fc0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 --2- 172.21.15.55:0/718677239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0e40039e0 0x7fe0fc1f5280 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 -- 172.21.15.55:0/718677239 >> 172.21.15.55:0/718677239 conn(0x7fe0fc0cd4d0 msgr2=0x7fe0fc0adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 -- 172.21.15.55:0/718677239 shutdown_connections 2024-05-20T22:08:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24.382+0000 7fe102e61640 1 -- 172.21.15.55:0/718677239 wait complete. 2024-05-20T22:08:24.394 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T22:08:24.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T22:08:24.395 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-20T22:08:24.395 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T22:08:24.395 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T22:08:24.395 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T22:08:24.395 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T22:08:24.395 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T22:08:24.395 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T22:08:24.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:24.395 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-20T22:08:24.411 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:08:24.411 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-20T22:08:24.412 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T22:08:24.413 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42687sec preferred_lft 42687sec 2024-05-20T22:08:24.413 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-20T22:08:24.413 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:24.413 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:24.413 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:24.413 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-20T22:08:24.413 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-20T22:08:24.413 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:24.413 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:24.413 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add name ceph-brx type bridge 2024-05-20T22:08:24.414 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr flush dev ceph-brx 2024-05-20T22:08:24.414 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set ceph-brx up 2024-05-20T22:08:24.414 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-20T22:08:24.414 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:24.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:24.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:24.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:24.554 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T22:08:24.580 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-20T22:08:24.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:24.582 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-20T22:08:24.639 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-20T22:08:24.639 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T22:08:24.639 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T22:08:24.639 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T22:08:24.639 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T22:08:24.640 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:24.640 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:24.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T22:08:24.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T22:08:24.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T22:08:24.640 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:24.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:24.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:24.770 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-20T22:08:24.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:24.786 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list-id 2024-05-20T22:08:24.843 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:24.843 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:24.843 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:08:24.843 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-20T22:08:24.843 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:24.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:24.964 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-20T22:08:24.964 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:24.964 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:24.965 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-20T22:08:24.965 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-20T22:08:24.965 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-20T22:08:24.965 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-20T22:08:24.965 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-20T22:08:24.965 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:25.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:25.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:25.084 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:25.084 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:25.085 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-20T22:08:25.085 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set dev brx.0 master ceph-brx 2024-05-20T22:08:25.085 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:25.165 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-20T22:08:25.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:08:25.166 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T22:08:25.181 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T22:08:25.182 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:08:25.182 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T22:08:25.237 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:08:25.304 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:25.361 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:08:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:08:25.363 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:08:25.364 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:08:25.364 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:08:25.364 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:08:25.364 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:08:25.364 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:08:25.364 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:08:25.364 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:08:25.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:25.364 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:08:25.417 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-20T22:08:25.417 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-20T22:08:25.460 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:08:25.485 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:08:25.544 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:08:25.545 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:08:25.546 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:08:25.546 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:08:25.546 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:08:25.546 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:08:25.546 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:08:25.546 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:08:25.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:25.546 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:08:25.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:08:25.553+0000 7fe349d1c480 -1 init, newargv = 0x7fe32c005820 newargc=15 2024-05-20T22:08:25.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:ceph-fuse[47972]: starting ceph client 2024-05-20T22:08:25.560 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:ceph-fuse[47972]: starting fuse 2024-05-20T22:08:25.570 INFO:teuthology.orchestra.run.smithi055.stdout:40 2024-05-20T22:08:25.570 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> sudo stdin-killer -- python3 -c ' 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> import glob 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> import re 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> import subprocess 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> def _find_admin_socket(client_name): 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> files = glob.glob(asok_path) 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> # Given a non-glob path, it better be there 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> if "*" not in asok_path: 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> assert(len(files) == 1) 2024-05-20T22:08:25.571 DEBUG:teuthology.orchestra.run.smithi055:> return files[0] 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> for f in files: 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> contents = proc_f.read() 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> if mountpoint in contents: 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> return f 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> print(_find_admin_socket("client.0")) 2024-05-20T22:08:25.572 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:08:25.693 INFO:teuthology.orchestra.run.smithi055.stdout:/var/run/ceph/ceph-client.0.47972.asok 2024-05-20T22:08:25.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:25.704 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47972.asok 2024-05-20T22:08:25.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:25.704 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47972.asok status 2024-05-20T22:08:25.852 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:08:25.852 INFO:teuthology.orchestra.run.smithi055.stdout: "metadata": { 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_sha1": "d2b507c724ef6c8295ee0a35d185cdab167fd61d", 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "entity_id": "0", 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": "smithi055", 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "pid": "47972", 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "root": "/" 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_count": 0, 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_pinned_count": 0, 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 4881, 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "inst": { 2024-05-20T22:08:25.853 INFO:teuthology.orchestra.run.smithi055.stdout: "name": { 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "client", 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: "num": 4881 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.1:0", 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 1128359002 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.1:0", 2024-05-20T22:08:25.854 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 1128359002 2024-05-20T22:08:25.855 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:25.855 INFO:teuthology.orchestra.run.smithi055.stdout: "inst_str": "client.4881 192.168.0.1:0/1128359002", 2024-05-20T22:08:25.855 INFO:teuthology.orchestra.run.smithi055.stdout: "addr_str": "192.168.0.1:0/1128359002", 2024-05-20T22:08:25.855 INFO:teuthology.orchestra.run.smithi055.stdout: "inode_count": 1, 2024-05-20T22:08:25.855 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_epoch": 9, 2024-05-20T22:08:25.855 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch": 24, 2024-05-20T22:08:25.855 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch_barrier": 0, 2024-05-20T22:08:25.855 INFO:teuthology.orchestra.run.smithi055.stdout: "blocklisted": false, 2024-05-20T22:08:25.855 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs" 2024-05-20T22:08:25.855 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:08:25.862 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T22:08:25.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.975+0000 7fa71d79c640 1 Processor -- start 2024-05-20T22:08:25.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.975+0000 7fa71d79c640 1 -- start start 2024-05-20T22:08:25.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa71d79c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa71815f100 0x7fa7180b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:25.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa71d79c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71806c790 0x7fa7180cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:25.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa71d79c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7180d3dd0 con 0x7fa7180b3b30 2024-05-20T22:08:25.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa71d79c640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa7180d40b0 con 0x7fa71815f100 2024-05-20T22:08:25.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa71d79c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7180d4390 con 0x7fa71806c790 2024-05-20T22:08:25.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa7167fc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa71815f100 0x7fa7180b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:25.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa7167fc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa71815f100 0x7fa7180b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35508/0 (socket says 172.21.15.55:35508) 2024-05-20T22:08:25.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa7167fc640 1 -- 172.21.15.55:0/4135821226 learned_addr learned my addr 172.21.15.55:0/4135821226 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:25.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa7177fe640 1 --2- 172.21.15.55:0/4135821226 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71806c790 0x7fa7180cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:25.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa7167fc640 1 -- 172.21.15.55:0/4135821226 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71806c790 msgr2=0x7fa7180cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:25.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa7167fc640 1 --2- 172.21.15.55:0/4135821226 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71806c790 0x7fa7180cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:25.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa7167fc640 1 -- 172.21.15.55:0/4135821226 >> v1:172.21.15.138:6789/0 conn(0x7fa7180b3b30 legacy=0x7fa71806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:25.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.977+0000 7fa7167fc640 1 -- 172.21.15.55:0/4135821226 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71806b5a0 con 0x7fa71815f100 2024-05-20T22:08:25.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.978+0000 7fa7167fc640 1 --2- 172.21.15.55:0/4135821226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa71815f100 0x7fa7180b7fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa700009870 tx=0x7fa70002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bb408d242c9efa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:25.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.978+0000 7fa715ffb640 1 -- 172.21.15.55:0/4135821226 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700037070 con 0x7fa71815f100 2024-05-20T22:08:25.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.978+0000 7fa715ffb640 1 -- 172.21.15.55:0/4135821226 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa700032440 con 0x7fa71815f100 2024-05-20T22:08:25.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.978+0000 7fa715ffb640 1 -- 172.21.15.55:0/4135821226 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700032810 con 0x7fa71815f100 2024-05-20T22:08:25.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.979+0000 7fa71d79c640 1 -- 172.21.15.55:0/4135821226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa71815f100 msgr2=0x7fa7180b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:25.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.979+0000 7fa71d79c640 1 --2- 172.21.15.55:0/4135821226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa71815f100 0x7fa7180b7fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa700009870 tx=0x7fa70002a210 comp rx=0 tx=0).stop 2024-05-20T22:08:25.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.979+0000 7fa71d79c640 1 -- 172.21.15.55:0/4135821226 shutdown_connections 2024-05-20T22:08:25.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.979+0000 7fa71d79c640 1 --2- 172.21.15.55:0/4135821226 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71806c790 0x7fa7180cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:25.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.979+0000 7fa71d79c640 1 --2- 172.21.15.55:0/4135821226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa71815f100 0x7fa7180b7fe0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:25.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.979+0000 7fa71d79c640 1 -- 172.21.15.55:0/4135821226 >> 172.21.15.55:0/4135821226 conn(0x7fa7180d51d0 msgr2=0x7fa7180b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:25.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.979+0000 7fa71d79c640 1 -- 172.21.15.55:0/4135821226 shutdown_connections 2024-05-20T22:08:25.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.979+0000 7fa71d79c640 1 -- 172.21.15.55:0/4135821226 wait complete. 2024-05-20T22:08:25.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.981+0000 7fa71d79c640 1 Processor -- start 2024-05-20T22:08:25.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.981+0000 7fa71d79c640 1 -- start start 2024-05-20T22:08:25.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.982+0000 7fa71d79c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71806c790 0x7fa7181e97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:25.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.982+0000 7fa71d79c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7180b3b30 0x7fa718161f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:25.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.982+0000 7fa71d79c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71815f100 0x7fa718152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:25.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.982+0000 7fa71d79c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa71806cba0 con 0x7fa7180b3b30 2024-05-20T22:08:25.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.982+0000 7fa71d79c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa71806bb60 con 0x7fa71806c790 2024-05-20T22:08:25.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.982+0000 7fa71d79c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa71806cd20 con 0x7fa71815f100 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.982+0000 7fa7167fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7180b3b30 0x7fa718161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.982+0000 7fa7167fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7180b3b30 0x7fa718161f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35524/0 (socket says 172.21.15.55:35524) 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.982+0000 7fa7167fc640 1 -- 172.21.15.55:0/2030496193 learned_addr learned my addr 172.21.15.55:0/2030496193 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa7177fe640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71815f100 0x7fa718152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa716ffd640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71806c790 0x7fa7181e97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa7167fc640 1 -- 172.21.15.55:0/2030496193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71815f100 msgr2=0x7fa718152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa7167fc640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71815f100 0x7fa718152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa7167fc640 1 -- 172.21.15.55:0/2030496193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71806c790 msgr2=0x7fa7181e97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa7167fc640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71806c790 0x7fa7181e97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa7167fc640 1 -- 172.21.15.55:0/2030496193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71815f510 con 0x7fa7180b3b30 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa7177fe640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71815f100 0x7fa718152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa7167fc640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7180b3b30 0x7fa718161f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa700009870 tx=0x7fa7000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:25.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa70f7fe640 1 -- 172.21.15.55:0/2030496193 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700041020 con 0x7fa7180b3b30 2024-05-20T22:08:25.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa70f7fe640 1 -- 172.21.15.55:0/2030496193 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7000306c0 con 0x7fa7180b3b30 2024-05-20T22:08:25.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa71d79c640 1 -- 172.21.15.55:0/2030496193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71814dd30 con 0x7fa7180b3b30 2024-05-20T22:08:25.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa70f7fe640 1 -- 172.21.15.55:0/2030496193 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700037040 con 0x7fa7180b3b30 2024-05-20T22:08:25.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.983+0000 7fa71d79c640 1 -- 172.21.15.55:0/2030496193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71815fa90 con 0x7fa7180b3b30 2024-05-20T22:08:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.984+0000 7fa71d79c640 1 -- 172.21.15.55:0/2030496193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71814daa0 con 0x7fa7180b3b30 2024-05-20T22:08:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.985+0000 7fa70f7fe640 1 -- 172.21.15.55:0/2030496193 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa700044020 con 0x7fa7180b3b30 2024-05-20T22:08:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.985+0000 7fa70f7fe640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6f806be60 0x7fa6f806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.985+0000 7fa70f7fe640 1 -- 172.21.15.55:0/2030496193 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa7000ab930 con 0x7fa7180b3b30 2024-05-20T22:08:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.985+0000 7fa716ffd640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6f806be60 0x7fa6f806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.986+0000 7fa716ffd640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6f806be60 0x7fa6f806e370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa710006fd0 tx=0x7fa710008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:25.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:25.990+0000 7fa70f7fe640 1 -- 172.21.15.55:0/2030496193 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa70004b030 con 0x7fa7180b3b30 2024-05-20T22:08:26.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.213+0000 7fa71d79c640 1 -- 172.21.15.55:0/2030496193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa718069090 con 0x7fa7180b3b30 2024-05-20T22:08:26.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.213+0000 7fa70f7fe640 1 -- 172.21.15.55:0/2030496193 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fa70007a030 con 0x7fa7180b3b30 2024-05-20T22:08:26.214 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T22:08:26.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 -- 172.21.15.55:0/2030496193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6f806be60 msgr2=0x7fa6f806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:26.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6f806be60 0x7fa6f806e370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa710006fd0 tx=0x7fa710008040 comp rx=0 tx=0).stop 2024-05-20T22:08:26.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 -- 172.21.15.55:0/2030496193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7180b3b30 msgr2=0x7fa718161f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:26.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7180b3b30 0x7fa718161f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa700009870 tx=0x7fa7000046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:26.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 -- 172.21.15.55:0/2030496193 shutdown_connections 2024-05-20T22:08:26.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6f806be60 0x7fa6f806e370 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:26.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa71815f100 0x7fa718152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:26.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7180b3b30 0x7fa718161f90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:26.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 --2- 172.21.15.55:0/2030496193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71806c790 0x7fa7181e97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:26.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 -- 172.21.15.55:0/2030496193 >> 172.21.15.55:0/2030496193 conn(0x7fa7180d51d0 msgr2=0x7fa7180b4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:26.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 -- 172.21.15.55:0/2030496193 shutdown_connections 2024-05-20T22:08:26.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26.215+0000 7fa71d79c640 1 -- 172.21.15.55:0/2030496193 wait complete. 2024-05-20T22:08:26.227 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T22:08:26.227 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T22:08:26.227 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-20T22:08:26.227 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-20T22:08:26.227 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-20T22:08:26.228 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T22:08:26.228 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T22:08:26.228 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T22:08:26.228 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-20T22:08:26.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:26.228 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-20T22:08:26.245 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:08:26.245 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:08:26.245 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:08:26.245 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T22:08:26.246 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42685sec preferred_lft 42685sec 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 52:e2:c2:8c:12:3b brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ba:c7ff:fe99:ba87/64 scope link 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 52:e2:c2:8c:12:3b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:08:26.247 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::50e2:c2ff:fe8c:123b/64 scope link tentative 2024-05-20T22:08:26.248 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:26.248 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T22:08:26.310 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-20T22:08:26.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:26.312 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-20T22:08:26.370 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-20T22:08:26.371 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T22:08:26.371 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T22:08:26.371 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T22:08:26.371 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T22:08:26.371 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:26.371 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:26.371 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T22:08:26.371 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T22:08:26.371 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T22:08:26.371 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:26.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:26.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:26.500 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-20T22:08:26.516 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T22:08:26.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:26.517 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list-id 2024-05-20T22:08:26.573 INFO:teuthology.orchestra.run.smithi055.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-20T22:08:26.573 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:26.573 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:26.573 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:08:26.573 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-20T22:08:26.573 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:26.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:26.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:26.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:26.693 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-20T22:08:26.719 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:08:26.719 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:08:26.719 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:08:26.720 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.720 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-20T22:08:26.720 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.720 INFO:teuthology.orchestra.run.smithi055.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T22:08:26.720 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 4a:4f:cc:c6:8c:22 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-20T22:08:26.720 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-20T22:08:26.720 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.720 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::484f:ccff:fec6:8c22/64 scope link 2024-05-20T22:08:26.720 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:26.721 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-20T22:08:26.787 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:08:26.787 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:08:26.787 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:08:26.787 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.787 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-20T22:08:26.787 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.788 INFO:teuthology.orchestra.run.smithi055.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T22:08:26.788 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 4a:4f:cc:c6:8c:22 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-20T22:08:26.788 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-20T22:08:26.788 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.788 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::484f:ccff:fec6:8c22/64 scope link 2024-05-20T22:08:26.788 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:26.789 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-20T22:08:26.789 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:26.789 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:26.790 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-20T22:08:26.790 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-20T22:08:26.790 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-20T22:08:26.790 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-20T22:08:26.790 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-20T22:08:26.790 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:26.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:26.948 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:26.948 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:08:26.948 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.1 up 2024-05-20T22:08:26.948 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set dev brx.1 master ceph-brx 2024-05-20T22:08:26.948 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:08:26.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:27.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:27.029 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-20T22:08:27.029 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:08:27.029 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-20T22:08:27.045 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-20T22:08:27.045 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:08:27.045 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-20T22:08:27.102 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:08:27.168 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:27.224 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:08:27.225 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:08:27.226 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:08:27.227 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:08:27.227 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:08:27.227 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:08:27.227 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:08:27.227 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:08:27.227 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:08:27.227 INFO:teuthology.orchestra.run.smithi055.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-20T22:08:27.227 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:08:27.227 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:08:27.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:27.227 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:08:27.280 INFO:teuthology.orchestra.run.smithi055.stdout:40 2024-05-20T22:08:27.281 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2024-05-20T22:08:27.281 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-20T22:08:27.323 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:08:27.349 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:27.409 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:08:27.409 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:27.410 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:08:27.411 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:08:27.412 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:08:27.412 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:08:27.412 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:08:27.412 INFO:teuthology.orchestra.run.smithi055.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-20T22:08:27.412 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:08:27.412 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:08:27.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:27.412 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:08:27.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi055.stderr:2024-05-20T22:08:27.418+0000 7f575b7ca480 -1 init, newargv = 0x7f574c0057c0 newargc=15 2024-05-20T22:08:27.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi055.stderr:ceph-fuse[48470]: starting ceph client 2024-05-20T22:08:27.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi055.stderr:ceph-fuse[48470]: starting fuse 2024-05-20T22:08:27.435 INFO:teuthology.orchestra.run.smithi055.stdout:40 2024-05-20T22:08:27.435 INFO:teuthology.orchestra.run.smithi055.stdout:43 2024-05-20T22:08:27.436 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 43] 2024-05-20T22:08:27.436 DEBUG:teuthology.orchestra.run.smithi055:> sudo stdin-killer -- python3 -c ' 2024-05-20T22:08:27.436 DEBUG:teuthology.orchestra.run.smithi055:> import glob 2024-05-20T22:08:27.436 DEBUG:teuthology.orchestra.run.smithi055:> import re 2024-05-20T22:08:27.436 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:08:27.436 DEBUG:teuthology.orchestra.run.smithi055:> import subprocess 2024-05-20T22:08:27.436 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:27.436 DEBUG:teuthology.orchestra.run.smithi055:> def _find_admin_socket(client_name): 2024-05-20T22:08:27.436 DEBUG:teuthology.orchestra.run.smithi055:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-20T22:08:27.436 DEBUG:teuthology.orchestra.run.smithi055:> files = glob.glob(asok_path) 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> # Given a non-glob path, it better be there 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> if "*" not in asok_path: 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> assert(len(files) == 1) 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> return files[0] 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> for f in files: 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> contents = proc_f.read() 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> if mountpoint in contents: 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> return f 2024-05-20T22:08:27.437 DEBUG:teuthology.orchestra.run.smithi055:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-20T22:08:27.438 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:08:27.438 DEBUG:teuthology.orchestra.run.smithi055:> print(_find_admin_socket("client.2")) 2024-05-20T22:08:27.438 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:08:27.557 INFO:teuthology.orchestra.run.smithi055.stdout:/var/run/ceph/ceph-client.2.48470.asok 2024-05-20T22:08:27.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:27.567 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.48470.asok 2024-05-20T22:08:27.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:27.568 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.48470.asok status 2024-05-20T22:08:27.716 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:08:27.716 INFO:teuthology.orchestra.run.smithi055.stdout: "metadata": { 2024-05-20T22:08:27.716 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_sha1": "d2b507c724ef6c8295ee0a35d185cdab167fd61d", 2024-05-20T22:08:27.716 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T22:08:27.717 INFO:teuthology.orchestra.run.smithi055.stdout: "entity_id": "2", 2024-05-20T22:08:27.717 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": "smithi055", 2024-05-20T22:08:27.717 INFO:teuthology.orchestra.run.smithi055.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-20T22:08:27.717 INFO:teuthology.orchestra.run.smithi055.stdout: "pid": "48470", 2024-05-20T22:08:27.717 INFO:teuthology.orchestra.run.smithi055.stdout: "root": "/" 2024-05-20T22:08:27.717 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:27.717 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_count": 0, 2024-05-20T22:08:27.718 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_pinned_count": 0, 2024-05-20T22:08:27.718 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 4890, 2024-05-20T22:08:27.718 INFO:teuthology.orchestra.run.smithi055.stdout: "inst": { 2024-05-20T22:08:27.718 INFO:teuthology.orchestra.run.smithi055.stdout: "name": { 2024-05-20T22:08:27.718 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "client", 2024-05-20T22:08:27.718 INFO:teuthology.orchestra.run.smithi055.stdout: "num": 4890 2024-05-20T22:08:27.718 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:27.719 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-20T22:08:27.719 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-20T22:08:27.719 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.2:0", 2024-05-20T22:08:27.719 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 1583410832 2024-05-20T22:08:27.719 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:08:27.719 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:27.719 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-20T22:08:27.719 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-20T22:08:27.720 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.2:0", 2024-05-20T22:08:27.720 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 1583410832 2024-05-20T22:08:27.720 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:08:27.720 INFO:teuthology.orchestra.run.smithi055.stdout: "inst_str": "client.4890 192.168.0.2:0/1583410832", 2024-05-20T22:08:27.720 INFO:teuthology.orchestra.run.smithi055.stdout: "addr_str": "192.168.0.2:0/1583410832", 2024-05-20T22:08:27.720 INFO:teuthology.orchestra.run.smithi055.stdout: "inode_count": 1, 2024-05-20T22:08:27.721 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_epoch": 9, 2024-05-20T22:08:27.721 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch": 24, 2024-05-20T22:08:27.721 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch_barrier": 0, 2024-05-20T22:08:27.721 INFO:teuthology.orchestra.run.smithi055.stdout: "blocklisted": false, 2024-05-20T22:08:27.721 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs" 2024-05-20T22:08:27.721 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:08:27.724 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T22:08:27.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.838+0000 7f560c557640 1 Processor -- start 2024-05-20T22:08:27.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.838+0000 7f560c557640 1 -- start start 2024-05-20T22:08:27.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.839+0000 7f560c557640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560406cc20 0x7f56040b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.839+0000 7f560c557640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f56040b94b0 0x7f56040b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.839+0000 7f560c557640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56040b7320 con 0x7f56040d4220 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.839+0000 7f560c557640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f56040b4320 con 0x7f56040b94b0 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.839+0000 7f560c557640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56040d0580 con 0x7f560406cc20 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.839+0000 7f560aaf4640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f56040b94b0 0x7f56040b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.839+0000 7f560aaf4640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f56040b94b0 0x7f56040b98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35534/0 (socket says 172.21.15.55:35534) 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.839+0000 7f560aaf4640 1 -- 172.21.15.55:0/1411300022 learned_addr learned my addr 172.21.15.55:0/1411300022 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.840+0000 7f5609af2640 1 --2- 172.21.15.55:0/1411300022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560406cc20 0x7f56040b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.840+0000 7f560aaf4640 1 -- 172.21.15.55:0/1411300022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560406cc20 msgr2=0x7f56040b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.840+0000 7f560aaf4640 1 --2- 172.21.15.55:0/1411300022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560406cc20 0x7f56040b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.840+0000 7f560aaf4640 1 -- 172.21.15.55:0/1411300022 >> v1:172.21.15.138:6789/0 conn(0x7f56040d4220 legacy=0x7f560406c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.840+0000 7f560aaf4640 1 -- 172.21.15.55:0/1411300022 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56040d0860 con 0x7f56040b94b0 2024-05-20T22:08:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.841+0000 7f560aaf4640 1 --2- 172.21.15.55:0/1411300022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040b94b0 0x7f56040b98c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009e50 tx=0x7f55fc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f902dcce7508e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.841+0000 7f56092f1640 1 -- 172.21.15.55:0/1411300022 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc036070 con 0x7f56040b94b0 2024-05-20T22:08:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.841+0000 7f56092f1640 1 -- 172.21.15.55:0/1411300022 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55fc031600 con 0x7f56040b94b0 2024-05-20T22:08:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.841+0000 7f56092f1640 1 -- 172.21.15.55:0/1411300022 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0319d0 con 0x7f56040b94b0 2024-05-20T22:08:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.841+0000 7f560c557640 1 -- 172.21.15.55:0/1411300022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040b94b0 msgr2=0x7f56040b98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.841+0000 7f560c557640 1 --2- 172.21.15.55:0/1411300022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040b94b0 0x7f56040b98c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009e50 tx=0x7f55fc02a710 comp rx=0 tx=0).stop 2024-05-20T22:08:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.842+0000 7f560c557640 1 -- 172.21.15.55:0/1411300022 shutdown_connections 2024-05-20T22:08:27.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.842+0000 7f560c557640 1 --2- 172.21.15.55:0/1411300022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040b94b0 0x7f56040b98c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:27.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.842+0000 7f560c557640 1 --2- 172.21.15.55:0/1411300022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f560406cc20 0x7f56040b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:27.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.842+0000 7f560c557640 1 -- 172.21.15.55:0/1411300022 >> 172.21.15.55:0/1411300022 conn(0x7f56040b6720 msgr2=0x7f56040b6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:27.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.842+0000 7f560c557640 1 -- 172.21.15.55:0/1411300022 shutdown_connections 2024-05-20T22:08:27.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.842+0000 7f560c557640 1 -- 172.21.15.55:0/1411300022 wait complete. 2024-05-20T22:08:27.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.843+0000 7f560c557640 1 Processor -- start 2024-05-20T22:08:27.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.843+0000 7f560c557640 1 -- start start 2024-05-20T22:08:27.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f560c557640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56040b94b0 0x7f560414b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:27.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f560c557640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040d4220 0x7f56040cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:27.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f560c557640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56040ca9e0 0x7f56040cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:27.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f560c557640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56040b8420 con 0x7f56040d4220 2024-05-20T22:08:27.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f560c557640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56040b8a50 con 0x7f56040b94b0 2024-05-20T22:08:27.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f560c557640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56040d4630 con 0x7f56040ca9e0 2024-05-20T22:08:27.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f5609af2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040d4220 0x7f56040cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:27.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f5609af2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040d4220 0x7f56040cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35546/0 (socket says 172.21.15.55:35546) 2024-05-20T22:08:27.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f5609af2640 1 -- 172.21.15.55:0/3324154858 learned_addr learned my addr 172.21.15.55:0/3324154858 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:27.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f560aaf4640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56040ca9e0 0x7f56040cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:27.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f560a2f3640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56040b94b0 0x7f560414b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:27.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f5609af2640 1 -- 172.21.15.55:0/3324154858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56040ca9e0 msgr2=0x7f56040cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:27.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f5609af2640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56040ca9e0 0x7f56040cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:27.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f5609af2640 1 -- 172.21.15.55:0/3324154858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56040b94b0 msgr2=0x7f560414b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:27.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f5609af2640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56040b94b0 0x7f560414b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:27.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f5609af2640 1 -- 172.21.15.55:0/3324154858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560415ba90 con 0x7f56040d4220 2024-05-20T22:08:27.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.845+0000 7f5609af2640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040d4220 0x7f56040cd9d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e870 tx=0x7f55f400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:27.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.846+0000 7f55faffd640 1 -- 172.21.15.55:0/3324154858 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4017070 con 0x7f56040d4220 2024-05-20T22:08:27.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.846+0000 7f55faffd640 1 -- 172.21.15.55:0/3324154858 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f400c940 con 0x7f56040d4220 2024-05-20T22:08:27.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.846+0000 7f560c557640 1 -- 172.21.15.55:0/3324154858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560415bd70 con 0x7f56040d4220 2024-05-20T22:08:27.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.846+0000 7f55faffd640 1 -- 172.21.15.55:0/3324154858 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400cd30 con 0x7f56040d4220 2024-05-20T22:08:27.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.846+0000 7f560c557640 1 -- 172.21.15.55:0/3324154858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560415c330 con 0x7f56040d4220 2024-05-20T22:08:27.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.847+0000 7f560c557640 1 -- 172.21.15.55:0/3324154858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5604069090 con 0x7f56040d4220 2024-05-20T22:08:27.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.847+0000 7f55faffd640 1 -- 172.21.15.55:0/3324154858 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f55f401a070 con 0x7f56040d4220 2024-05-20T22:08:27.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.848+0000 7f55faffd640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55e806be30 0x7f55e806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:27.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.848+0000 7f55faffd640 1 -- 172.21.15.55:0/3324154858 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f55f408b460 con 0x7f56040d4220 2024-05-20T22:08:27.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.848+0000 7f560a2f3640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55e806be30 0x7f55e806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:27.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.848+0000 7f560a2f3640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55e806be30 0x7f55e806e340 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5600000f60 tx=0x7f5600009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:27.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:27.853+0000 7f55faffd640 1 -- 172.21.15.55:0/3324154858 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f55f4059370 con 0x7f56040d4220 2024-05-20T22:08:28.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.075+0000 7f560c557640 1 -- 172.21.15.55:0/3324154858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f56040cfa00 con 0x7f56040d4220 2024-05-20T22:08:28.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.076+0000 7f55faffd640 1 -- 172.21.15.55:0/3324154858 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f55f405d670 con 0x7f56040d4220 2024-05-20T22:08:28.077 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.077+0000 7f560c557640 1 -- 172.21.15.55:0/3324154858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55e806be30 msgr2=0x7f55e806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55e806be30 0x7f55e806e340 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5600000f60 tx=0x7f5600009040 comp rx=0 tx=0).stop 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 -- 172.21.15.55:0/3324154858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040d4220 msgr2=0x7f56040cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040d4220 0x7f56040cd9d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e870 tx=0x7f55f400ed30 comp rx=0 tx=0).stop 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 -- 172.21.15.55:0/3324154858 shutdown_connections 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55e806be30 0x7f55e806e340 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56040ca9e0 0x7f56040cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56040d4220 0x7f56040cd9d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 --2- 172.21.15.55:0/3324154858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56040b94b0 0x7f560414b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 -- 172.21.15.55:0/3324154858 >> 172.21.15.55:0/3324154858 conn(0x7f56040b6720 msgr2=0x7f56040b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 -- 172.21.15.55:0/3324154858 shutdown_connections 2024-05-20T22:08:28.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:28.078+0000 7f560c557640 1 -- 172.21.15.55:0/3324154858 wait complete. 2024-05-20T22:08:28.090 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T22:08:28.090 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T22:08:28.090 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-20T22:08:28.090 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-20T22:08:28.090 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-20T22:08:28.090 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T22:08:28.090 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T22:08:28.090 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T22:08:28.091 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-20T22:08:28.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:28.091 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:28.111 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-20T22:08:28.112 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:28.112 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:28.112 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T22:08:28.112 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:28.112 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-20T22:08:28.112 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42685sec preferred_lft 42685sec 2024-05-20T22:08:28.112 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-20T22:08:28.112 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:28.112 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-20T22:08:28.112 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:28.113 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-20T22:08:28.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-05-20T22:08:28.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-05-20T22:08:28.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-05-20T22:08:28.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-20T22:08:28.113 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-20T22:08:28.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:28.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:28.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:28.454 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T22:08:28.479 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-20T22:08:28.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:28.481 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-20T22:08:28.549 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-20T22:08:28.549 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T22:08:28.549 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-20T22:08:28.549 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-20T22:08:28.549 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T22:08:28.550 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:28.550 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-20T22:08:28.550 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-20T22:08:28.550 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-20T22:08:28.550 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-20T22:08:28.550 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-20T22:08:28.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:28.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:28.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:28.855 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-20T22:08:28.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:28.870 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-05-20T22:08:28.927 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:28.927 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-20T22:08:28.927 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:08:28.927 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-20T22:08:28.927 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-20T22:08:29.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:29.047 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-20T22:08:29.048 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:29.048 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-20T22:08:29.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-20T22:08:29.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-20T22:08:29.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-20T22:08:29.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-20T22:08:29.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-20T22:08:29.048 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-20T22:08:29.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:29.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:29.177 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:29.177 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-20T22:08:29.178 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-20T22:08:29.178 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-05-20T22:08:29.178 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-20T22:08:29.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:29.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:29.254 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-20T22:08:29.254 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:08:29.254 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-20T22:08:29.278 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-20T22:08:29.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:08:29.279 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-20T22:08:29.333 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-20T22:08:29.399 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:29.456 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-20T22:08:29.456 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-20T22:08:29.456 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-20T22:08:29.456 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-20T22:08:29.456 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-20T22:08:29.457 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-05-20T22:08:29.458 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-20T22:08:29.459 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:08:29.459 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:08:29.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:29.459 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-20T22:08:29.511 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-20T22:08:29.512 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-20T22:08:29.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-20T22:08:29.578 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:29.634 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-20T22:08:29.635 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-05-20T22:08:29.636 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-05-20T22:08:29.637 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-05-20T22:08:29.637 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-20T22:08:29.637 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:08:29.637 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:08:29.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:29.638 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-20T22:08:29.656 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:2024-05-20T22:08:29.655+0000 7ffafce88480 -1 init, newargv = 0x7ffae0001f00 newargc=15 2024-05-20T22:08:29.656 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:ceph-fuse[44342]: starting ceph client 2024-05-20T22:08:29.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:ceph-fuse[44342]: starting fuse 2024-05-20T22:08:29.672 INFO:teuthology.orchestra.run.smithi138.stdout:40 2024-05-20T22:08:29.673 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-20T22:08:29.673 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.1")) 2024-05-20T22:08:29.674 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-20T22:08:29.793 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.1.44342.asok 2024-05-20T22:08:29.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:29.803 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44342.asok 2024-05-20T22:08:29.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:29.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44342.asok status 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "d2b507c724ef6c8295ee0a35d185cdab167fd61d", 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "1", 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "44342", 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-20T22:08:29.951 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4834, 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 4834 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 4099009249 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-20T22:08:29.952 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 4099009249 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.4834 192.168.0.1:0/4099009249", 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/4099009249", 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 9, 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 24, 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-20T22:08:29.953 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-20T22:08:29.959 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T22:08:30.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.088+0000 7f3469661640 1 Processor -- start 2024-05-20T22:08:30.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.088+0000 7f3469661640 1 -- start start 2024-05-20T22:08:30.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.089+0000 7f3469661640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f34640b8140 0x7f34640d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:30.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f3469661640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34640b7c90 con 0x7f34640ce520 2024-05-20T22:08:30.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f3469661640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34640d1740 con 0x7f34640d1190 2024-05-20T22:08:30.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f3469661640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f34640ce930 con 0x7f34640b8140 2024-05-20T22:08:30.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34637fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f34640ce520 0x7f34640b67c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56778/0 (socket says 172.21.15.55:56778) 2024-05-20T22:08:30.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34637fe640 1 -- 172.21.15.55:0/1485906062 learned_addr learned my addr 172.21.15.55:0/1485906062 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:30.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f3462ffd640 1 --2- 172.21.15.55:0/1485906062 >> v2:172.21.15.138:3300/0 conn(0x7f34640b8140 0x7f34640d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:30.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 918602671 0 0) 0x7f34640b7c90 con 0x7f34640ce520 2024-05-20T22:08:30.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3450003520 con 0x7f34640ce520 2024-05-20T22:08:30.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2574985165 0 0) 0x7f34640d1740 con 0x7f34640d1190 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34640b7c90 con 0x7f34640d1190 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1709855769 0 0) 0x7f3450003520 con 0x7f34640ce520 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 >> v2:172.21.15.138:3300/0 conn(0x7f34640b8140 msgr2=0x7f34640d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 --2- 172.21.15.55:0/1485906062 >> v2:172.21.15.138:3300/0 conn(0x7f34640b8140 0x7f34640d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 >> v1:172.21.15.138:6789/0 conn(0x7f34640d1190 legacy=0x7f34640b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346409bc10 con 0x7f34640ce520 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f345c0019f0 con 0x7f34640ce520 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f3462ffd640 1 --2- 172.21.15.55:0/1485906062 >> v2:172.21.15.138:3300/0 conn(0x7f34640b8140 0x7f34640d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 >> v1:172.21.15.55:6789/0 conn(0x7f34640ce520 legacy=0x7f34640b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3450003b70 0x7f3450003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3450004800 0x7f3450004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f345000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:30.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f34640ce930 con 0x7f3450005150 2024-05-20T22:08:30.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f345000c0f0 con 0x7f3450003b70 2024-05-20T22:08:30.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.090+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f345000c2f0 con 0x7f3450004800 2024-05-20T22:08:30.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.091+0000 7f34637fe640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f345000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:30.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.091+0000 7f3462ffd640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3450003b70 0x7f3450003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:30.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.091+0000 7f345bfff640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3450004800 0x7f3450004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:30.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.091+0000 7f34637fe640 1 -- 172.21.15.55:0/1485906062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3450004800 msgr2=0x7f3450004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:30.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.091+0000 7f34637fe640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3450004800 0x7f3450004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:30.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.091+0000 7f34637fe640 1 -- 172.21.15.55:0/1485906062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3450003b70 msgr2=0x7f3450003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:30.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.091+0000 7f34637fe640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3450003b70 0x7f3450003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:30.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.091+0000 7f34637fe640 1 -- 172.21.15.55:0/1485906062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3450003520 con 0x7f3450005150 2024-05-20T22:08:30.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.092+0000 7f34637fe640 1 -- 172.21.15.55:0/1485906062 reap_dead start 2024-05-20T22:08:30.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.092+0000 7f34637fe640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f345000b9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f34640d24c0 tx=0x7f3450005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57775e34ddb7baa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:30.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.092+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c00a450 con 0x7f3450005150 2024-05-20T22:08:30.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.092+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345c02f070 con 0x7f3450005150 2024-05-20T22:08:30.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.092+0000 7f34627fc640 1 -- 172.21.15.55:0/1485906062 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c02bc00 con 0x7f3450005150 2024-05-20T22:08:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.093+0000 7f3469661640 1 -- 172.21.15.55:0/1485906062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 msgr2=0x7f345000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.093+0000 7f3469661640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f345000b9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f34640d24c0 tx=0x7f3450005830 comp rx=0 tx=0).stop 2024-05-20T22:08:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.093+0000 7f3469661640 1 -- 172.21.15.55:0/1485906062 shutdown_connections 2024-05-20T22:08:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.093+0000 7f3469661640 1 --2- 172.21.15.55:0/1485906062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f345000b9c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.093+0000 7f3469661640 1 -- 172.21.15.55:0/1485906062 >> 172.21.15.55:0/1485906062 conn(0x7f34640b4340 msgr2=0x7f34640b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.093+0000 7f3469661640 1 -- 172.21.15.55:0/1485906062 shutdown_connections 2024-05-20T22:08:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.093+0000 7f3469661640 1 -- 172.21.15.55:0/1485906062 wait complete. 2024-05-20T22:08:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.095+0000 7f3469661640 1 Processor -- start 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.095+0000 7f3469661640 1 -- start start 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3469661640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f34641d7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3469661640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34640a0210 0x7f346409aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3469661640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346409afa0 0x7f34640b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3469661640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f34640d06a0 con 0x7f3450005150 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3469661640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34640d0520 con 0x7f346409afa0 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3469661640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34640b8140 con 0x7f34640a0210 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3462ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f34641d7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3462ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f34641d7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38414/0 (socket says 172.21.15.55:38414) 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3462ffd640 1 -- 172.21.15.55:0/1946036433 learned_addr learned my addr 172.21.15.55:0/1946036433 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f345bfff640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34640a0210 0x7f346409aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:30.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f34637fe640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346409afa0 0x7f34640b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3462ffd640 1 -- 172.21.15.55:0/1946036433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34640a0210 msgr2=0x7f346409aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3462ffd640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34640a0210 0x7f346409aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3462ffd640 1 -- 172.21.15.55:0/1946036433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346409afa0 msgr2=0x7f34640b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3462ffd640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346409afa0 0x7f34640b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f3462ffd640 1 -- 172.21.15.55:0/1946036433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346409b3b0 con 0x7f3450005150 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.096+0000 7f34637fe640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346409afa0 0x7f34640b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.097+0000 7f3462ffd640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f34641d7900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3454011a50 tx=0x7f3454011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.097+0000 7f345b7fe640 1 -- 172.21.15.55:0/1946036433 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345400fb90 con 0x7f3450005150 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.097+0000 7f345b7fe640 1 -- 172.21.15.55:0/1946036433 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345400fd30 con 0x7f3450005150 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.097+0000 7f3469661640 1 -- 172.21.15.55:0/1946036433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346409eb00 con 0x7f3450005150 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.097+0000 7f345b7fe640 1 -- 172.21.15.55:0/1946036433 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454014680 con 0x7f3450005150 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.097+0000 7f3469661640 1 -- 172.21.15.55:0/1946036433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346409f0c0 con 0x7f3450005150 2024-05-20T22:08:30.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.098+0000 7f3469661640 1 -- 172.21.15.55:0/1946036433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f346406be80 con 0x7f3450005150 2024-05-20T22:08:30.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.099+0000 7f345b7fe640 1 -- 172.21.15.55:0/1946036433 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3454012040 con 0x7f3450005150 2024-05-20T22:08:30.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.099+0000 7f345b7fe640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f344406be30 0x7f344406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:30.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.099+0000 7f345b7fe640 1 -- 172.21.15.55:0/1946036433 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f345408b8e0 con 0x7f3450005150 2024-05-20T22:08:30.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.099+0000 7f345bfff640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f344406be30 0x7f344406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:30.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.100+0000 7f345bfff640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f344406be30 0x7f344406e340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f34640b3ca0 tx=0x7f344c001210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:30.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.104+0000 7f345b7fe640 1 -- 172.21.15.55:0/1946036433 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f346406be80 con 0x7f3450005150 2024-05-20T22:08:30.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.324+0000 7f3469661640 1 -- 172.21.15.55:0/1946036433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f34640b0c60 con 0x7f3450005150 2024-05-20T22:08:30.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.324+0000 7f345b7fe640 1 -- 172.21.15.55:0/1946036433 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f34640b0c60 con 0x7f3450005150 2024-05-20T22:08:30.325 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T22:08:30.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 -- 172.21.15.55:0/1946036433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f344406be30 msgr2=0x7f344406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:30.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f344406be30 0x7f344406e340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f34640b3ca0 tx=0x7f344c001210 comp rx=0 tx=0).stop 2024-05-20T22:08:30.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 -- 172.21.15.55:0/1946036433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 msgr2=0x7f34641d7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:30.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f34641d7900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3454011a50 tx=0x7f3454011e50 comp rx=0 tx=0).stop 2024-05-20T22:08:30.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 -- 172.21.15.55:0/1946036433 shutdown_connections 2024-05-20T22:08:30.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f344406be30 0x7f344406e340 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:30.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f346409afa0 0x7f34640b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:30.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34640a0210 0x7f346409aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:30.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 --2- 172.21.15.55:0/1946036433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3450005150 0x7f34641d7900 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:30.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 -- 172.21.15.55:0/1946036433 >> 172.21.15.55:0/1946036433 conn(0x7f34640b4340 msgr2=0x7f34640b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:30.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 -- 172.21.15.55:0/1946036433 shutdown_connections 2024-05-20T22:08:30.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:30.326+0000 7f3469661640 1 -- 172.21.15.55:0/1946036433 wait complete. 2024-05-20T22:08:30.338 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T22:08:30.338 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T22:08:30.339 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-20T22:08:30.339 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-20T22:08:30.339 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-20T22:08:30.339 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T22:08:30.339 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T22:08:30.339 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T22:08:30.339 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-20T22:08:30.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:30.339 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-20T22:08:30.355 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:08:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42683sec preferred_lft 42683sec 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether ba:08:83:1b:79:3b brd ff:ff:ff:ff:ff:ff 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ccec:7fff:fe1c:8044/64 scope link 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether ba:08:83:1b:79:3b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::b808:83ff:fe1b:793b/64 scope link tentative 2024-05-20T22:08:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:30.358 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T22:08:30.419 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-20T22:08:30.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:30.421 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-20T22:08:30.478 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-20T22:08:30.478 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T22:08:30.479 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-20T22:08:30.479 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-20T22:08:30.479 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T22:08:30.479 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:30.479 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-20T22:08:30.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-20T22:08:30.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-20T22:08:30.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-20T22:08:30.479 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-20T22:08:30.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:30.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:30.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:30.606 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-20T22:08:30.622 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-20T22:08:30.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:30.623 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-05-20T22:08:30.678 INFO:teuthology.orchestra.run.smithi138.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-20T22:08:30.679 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:30.679 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-20T22:08:30.679 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-20T22:08:30.679 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-20T22:08:30.679 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-20T22:08:30.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:30.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:30.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:30.796 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:d8:58:4b:99:8b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::64d8:58ff:fe4b:998b/64 scope link 2024-05-20T22:08:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:30.823 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:d8:58:4b:99:8b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.887 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::64d8:58ff:fe4b:998b/64 scope link 2024-05-20T22:08:30.888 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:08:30.888 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-20T22:08:30.889 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:30.889 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-20T22:08:30.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-20T22:08:30.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-20T22:08:30.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-20T22:08:30.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-20T22:08:30.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-20T22:08:30.889 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-20T22:08:30.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:31.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:31.041 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:08:31.041 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-20T22:08:31.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.1 up 2024-05-20T22:08:31.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.1 master ceph-brx 2024-05-20T22:08:31.042 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-20T22:08:31.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:08:31.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:31.118 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-20T22:08:31.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:08:31.119 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-20T22:08:31.146 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-20T22:08:31.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:08:31.147 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-20T22:08:31.202 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-20T22:08:31.266 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:31.324 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-20T22:08:31.324 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-20T22:08:31.324 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-20T22:08:31.324 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-20T22:08:31.325 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-20T22:08:31.325 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-20T22:08:31.325 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-20T22:08:31.325 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-20T22:08:31.325 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-20T22:08:31.325 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-20T22:08:31.326 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-20T22:08:31.326 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-20T22:08:31.326 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:31.326 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-20T22:08:31.326 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-20T22:08:31.326 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-20T22:08:31.327 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-20T22:08:31.327 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:31.327 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-20T22:08:31.327 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:31.327 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:31.327 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:31.328 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:31.328 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-20T22:08:31.328 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-20T22:08:31.328 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-05-20T22:08:31.328 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-05-20T22:08:31.328 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-05-20T22:08:31.329 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-05-20T22:08:31.329 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-20T22:08:31.329 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:08:31.329 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:08:31.329 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-20T22:08:31.329 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-20T22:08:31.329 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-20T22:08:31.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:31.330 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-20T22:08:31.380 INFO:teuthology.orchestra.run.smithi138.stdout:40 2024-05-20T22:08:31.380 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2024-05-20T22:08:31.381 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-20T22:08:31.423 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-20T22:08:31.447 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:08:31.505 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-20T22:08:31.505 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-20T22:08:31.505 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-20T22:08:31.505 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-20T22:08:31.505 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-20T22:08:31.505 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-20T22:08:31.506 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:08:31.507 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-20T22:08:31.508 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-20T22:08:31.508 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-20T22:08:31.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:31.508 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-20T22:08:31.514 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi138.stderr:2024-05-20T22:08:31.514+0000 7fc8ac557480 -1 init, newargv = 0x7fc89c001f00 newargc=15 2024-05-20T22:08:31.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi138.stderr:ceph-fuse[44796]: starting ceph client 2024-05-20T22:08:31.519 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi138.stderr:ceph-fuse[44796]: starting fuse 2024-05-20T22:08:31.529 INFO:teuthology.orchestra.run.smithi138.stdout:40 2024-05-20T22:08:31.530 INFO:teuthology.orchestra.run.smithi138.stdout:43 2024-05-20T22:08:31.530 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 43] 2024-05-20T22:08:31.530 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-20T22:08:31.530 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-20T22:08:31.530 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-20T22:08:31.530 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-20T22:08:31.530 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-20T22:08:31.530 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-20T22:08:31.530 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-20T22:08:31.531 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-20T22:08:31.532 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-20T22:08:31.532 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-20T22:08:31.532 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-20T22:08:31.532 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-20T22:08:31.532 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-20T22:08:31.532 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-20T22:08:31.532 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-20T22:08:31.532 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.3")) 2024-05-20T22:08:31.532 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-20T22:08:31.648 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.3.44796.asok 2024-05-20T22:08:31.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-20T22:08:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:08:31.658 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.44796.asok 2024-05-20T22:08:31.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:31.659 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.44796.asok status 2024-05-20T22:08:31.802 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-20T22:08:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-20T22:08:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "d2b507c724ef6c8295ee0a35d185cdab167fd61d", 2024-05-20T22:08:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "3", 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "44796", 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 34595, 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-20T22:08:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 34595 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.2:0", 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 4125173231 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.2:0", 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 4125173231 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.34595 192.168.0.2:0/4125173231", 2024-05-20T22:08:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.2:0/4125173231", 2024-05-20T22:08:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-20T22:08:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 9, 2024-05-20T22:08:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 24, 2024-05-20T22:08:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-20T22:08:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-20T22:08:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-20T22:08:31.805 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-20T22:08:31.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:31.811 DEBUG:teuthology.orchestra.run.smithi055:> stat --file-system '--printf=%T 2024-05-20T22:08:31.811 DEBUG:teuthology.orchestra.run.smithi055:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-20T22:08:31.832 INFO:teuthology.orchestra.run.smithi055.stdout:fuseblk 2024-05-20T22:08:31.832 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-20T22:08:31.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:31.832 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T22:08:31.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:31.904 DEBUG:teuthology.orchestra.run.smithi055:> stat --file-system '--printf=%T 2024-05-20T22:08:31.904 DEBUG:teuthology.orchestra.run.smithi055:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-20T22:08:31.963 INFO:teuthology.orchestra.run.smithi055.stdout:fuseblk 2024-05-20T22:08:31.963 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-20T22:08:31.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:31.963 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-20T22:08:32.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:32.033 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-20T22:08:32.033 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-20T22:08:32.049 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-20T22:08:32.050 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-20T22:08:32.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:32.050 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-20T22:08:32.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:32.115 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-20T22:08:32.115 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-20T22:08:32.170 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-20T22:08:32.171 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-20T22:08:32.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:32.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-20T22:08:32.235 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-20T22:08:32.304 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:32.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.433+0000 7f31ee1aa640 1 Processor -- start 2024-05-20T22:08:32.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.433+0000 7f31ee1aa640 1 -- start start 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.434+0000 7f31ee1aa640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31e806d6b0 0x7f31e806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.434+0000 7f31ee1aa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f31e80d2700 0x7f31e80d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.434+0000 7f31ee1aa640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e806e320 con 0x7f31e80cd030 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.434+0000 7f31ee1aa640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f31e80d1d70 con 0x7f31e806d6b0 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.434+0000 7f31ee1aa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f31e80b2fa0 con 0x7f31e80d2700 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.435+0000 7f31dffff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31e806d6b0 0x7f31e806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.435+0000 7f31e77fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f31e80cd030 0x7f31e8162aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56792/0 (socket says 172.21.15.55:56792) 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.435+0000 7f31e77fe640 1 -- 172.21.15.55:0/668252495 learned_addr learned my addr 172.21.15.55:0/668252495 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.435+0000 7f31e7fff640 1 --2- 172.21.15.55:0/668252495 >> v2:172.21.15.138:3300/0 conn(0x7f31e80d2700 0x7f31e80d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.435+0000 7f31dffff640 1 -- 172.21.15.55:0/668252495 >> v2:172.21.15.138:3300/0 conn(0x7f31e80d2700 msgr2=0x7f31e80d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:32.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.435+0000 7f31dffff640 1 --2- 172.21.15.55:0/668252495 >> v2:172.21.15.138:3300/0 conn(0x7f31e80d2700 0x7f31e80d2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.435+0000 7f31dffff640 1 -- 172.21.15.55:0/668252495 >> v1:172.21.15.55:6789/0 conn(0x7f31e80cd030 legacy=0x7f31e8162aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.435+0000 7f31e6ffd640 1 -- 172.21.15.55:0/668252495 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4223589641 0 0) 0x7f31e806e320 con 0x7f31e80cd030 2024-05-20T22:08:32.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.435+0000 7f31dffff640 1 -- 172.21.15.55:0/668252495 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e806e600 con 0x7f31e806d6b0 2024-05-20T22:08:32.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.436+0000 7f31dffff640 1 --2- 172.21.15.55:0/668252495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e806d6b0 0x7f31e806dac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31e808d1b0 tx=0x7f31d002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e070910a7b1815 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:32.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.436+0000 7f31e6ffd640 1 -- 172.21.15.55:0/668252495 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0037070 con 0x7f31e806d6b0 2024-05-20T22:08:32.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.436+0000 7f31e6ffd640 1 -- 172.21.15.55:0/668252495 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d0032440 con 0x7f31e806d6b0 2024-05-20T22:08:32.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.436+0000 7f31e6ffd640 1 -- 172.21.15.55:0/668252495 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d002ae60 con 0x7f31e806d6b0 2024-05-20T22:08:32.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.437+0000 7f31ee1aa640 1 -- 172.21.15.55:0/668252495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e806d6b0 msgr2=0x7f31e806dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.437+0000 7f31ee1aa640 1 --2- 172.21.15.55:0/668252495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e806d6b0 0x7f31e806dac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31e808d1b0 tx=0x7f31d002a110 comp rx=0 tx=0).stop 2024-05-20T22:08:32.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.437+0000 7f31ee1aa640 1 -- 172.21.15.55:0/668252495 shutdown_connections 2024-05-20T22:08:32.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.437+0000 7f31ee1aa640 1 --2- 172.21.15.55:0/668252495 >> v2:172.21.15.138:3300/0 conn(0x7f31e80d2700 0x7f31e80d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.437+0000 7f31ee1aa640 1 --2- 172.21.15.55:0/668252495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e806d6b0 0x7f31e806dac0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.437+0000 7f31ee1aa640 1 -- 172.21.15.55:0/668252495 >> 172.21.15.55:0/668252495 conn(0x7f31e808bff0 msgr2=0x7f31e808c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:32.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.437+0000 7f31ee1aa640 1 -- 172.21.15.55:0/668252495 shutdown_connections 2024-05-20T22:08:32.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.437+0000 7f31ee1aa640 1 -- 172.21.15.55:0/668252495 wait complete. 2024-05-20T22:08:32.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.439+0000 7f31ee1aa640 1 Processor -- start 2024-05-20T22:08:32.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.439+0000 7f31ee1aa640 1 -- start start 2024-05-20T22:08:32.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31ee1aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e806d6b0 0x7f31e809c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31ee1aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e80cd030 0x7f31e80b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31ee1aa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e80d2700 0x7f31e80b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31ee1aa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f31e80b2890 con 0x7f31e80d2700 2024-05-20T22:08:32.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31ee1aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31e80b3280 con 0x7f31e806d6b0 2024-05-20T22:08:32.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31ee1aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f31e806e000 con 0x7f31e80cd030 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31e7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e80d2700 0x7f31e80b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31e7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e80d2700 0x7f31e80b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38438/0 (socket says 172.21.15.55:38438) 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31e7fff640 1 -- 172.21.15.55:0/1079910634 learned_addr learned my addr 172.21.15.55:0/1079910634 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31e77fe640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e806d6b0 0x7f31e809c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31dffff640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e80cd030 0x7f31e80b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.440+0000 7f31e7fff640 1 -- 172.21.15.55:0/1079910634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e80cd030 msgr2=0x7f31e80b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.441+0000 7f31e7fff640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e80cd030 0x7f31e80b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.441+0000 7f31e7fff640 1 -- 172.21.15.55:0/1079910634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e806d6b0 msgr2=0x7f31e809c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.441+0000 7f31e7fff640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e806d6b0 0x7f31e809c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.441+0000 7f31e7fff640 1 -- 172.21.15.55:0/1079910634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e80ca070 con 0x7f31e80d2700 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.441+0000 7f31e7fff640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e80d2700 0x7f31e80b5d70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f31e0007e90 tx=0x7f31e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.441+0000 7f31e4ff9640 1 -- 172.21.15.55:0/1079910634 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0014420 con 0x7f31e80d2700 2024-05-20T22:08:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.441+0000 7f31e4ff9640 1 -- 172.21.15.55:0/1079910634 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e0019070 con 0x7f31e80d2700 2024-05-20T22:08:32.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.441+0000 7f31ee1aa640 1 -- 172.21.15.55:0/1079910634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e80ca350 con 0x7f31e80d2700 2024-05-20T22:08:32.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.441+0000 7f31e4ff9640 1 -- 172.21.15.55:0/1079910634 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0014800 con 0x7f31e80d2700 2024-05-20T22:08:32.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.441+0000 7f31ee1aa640 1 -- 172.21.15.55:0/1079910634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e80ca910 con 0x7f31e80d2700 2024-05-20T22:08:32.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.442+0000 7f31ee1aa640 1 -- 172.21.15.55:0/1079910634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e8069090 con 0x7f31e80d2700 2024-05-20T22:08:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.443+0000 7f31e4ff9640 1 -- 172.21.15.55:0/1079910634 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31e00149a0 con 0x7f31e80d2700 2024-05-20T22:08:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.443+0000 7f31e4ff9640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c806be60 0x7f31c806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.443+0000 7f31e4ff9640 1 -- 172.21.15.55:0/1079910634 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f31e008b830 con 0x7f31e80d2700 2024-05-20T22:08:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.443+0000 7f31e77fe640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c806be60 0x7f31c806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.444+0000 7f31e77fe640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c806be60 0x7f31c806e370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31d80010e0 tx=0x7f31d8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:32.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.448+0000 7f31e4ff9640 1 -- 172.21.15.55:0/1079910634 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f31e0059850 con 0x7f31e80d2700 2024-05-20T22:08:32.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.633+0000 7f31ee1aa640 1 -- 172.21.15.55:0/1079910634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31e8072420 con 0x7f31e80d2700 2024-05-20T22:08:32.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.634+0000 7f31e4ff9640 1 -- 172.21.15.55:0/1079910634 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f31e005db50 con 0x7f31e80d2700 2024-05-20T22:08:32.635 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:32.636 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:07:58.091506+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:32.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.636+0000 7f31ee1aa640 1 -- 172.21.15.55:0/1079910634 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c806be60 msgr2=0x7f31c806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.636+0000 7f31ee1aa640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c806be60 0x7f31c806e370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31d80010e0 tx=0x7f31d8009040 comp rx=0 tx=0).stop 2024-05-20T22:08:32.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.636+0000 7f31ee1aa640 1 -- 172.21.15.55:0/1079910634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e80d2700 msgr2=0x7f31e80b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.636+0000 7f31ee1aa640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e80d2700 0x7f31e80b5d70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f31e0007e90 tx=0x7f31e000e4b0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.636+0000 7f31ee1aa640 1 -- 172.21.15.55:0/1079910634 shutdown_connections 2024-05-20T22:08:32.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.636+0000 7f31ee1aa640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f31c806be60 0x7f31c806e370 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.636+0000 7f31ee1aa640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31e80d2700 0x7f31e80b5d70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.636+0000 7f31ee1aa640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31e80cd030 0x7f31e80b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.636+0000 7f31ee1aa640 1 --2- 172.21.15.55:0/1079910634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31e806d6b0 0x7f31e809c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.636+0000 7f31ee1aa640 1 -- 172.21.15.55:0/1079910634 >> 172.21.15.55:0/1079910634 conn(0x7f31e808bff0 msgr2=0x7f31e80ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:32.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.637+0000 7f31ee1aa640 1 -- 172.21.15.55:0/1079910634 shutdown_connections 2024-05-20T22:08:32.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.637+0000 7f31ee1aa640 1 -- 172.21.15.55:0/1079910634 wait complete. 2024-05-20T22:08:32.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:32.649 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:32.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.767+0000 7f5f6473e640 1 Processor -- start 2024-05-20T22:08:32.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.767+0000 7f5f6473e640 1 -- start start 2024-05-20T22:08:32.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f6473e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f5c0d32a0 0x7f5f5c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f6473e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5f5c0cf690 0x7f5f5c0cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f6473e640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f5c0d3780 con 0x7f5f5c0cffe0 2024-05-20T22:08:32.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f6473e640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f5f5c06c510 con 0x7f5f5c0cf690 2024-05-20T22:08:32.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f6473e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5f5c0d3a60 con 0x7f5f5c0d32a0 2024-05-20T22:08:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f61cd9640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5f5c0cf690 0x7f5f5c0cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f62cdb640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5f5c0cffe0 0x7f5f5c162cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56800/0 (socket says 172.21.15.55:56800) 2024-05-20T22:08:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f62cdb640 1 -- 172.21.15.55:0/3511315532 learned_addr learned my addr 172.21.15.55:0/3511315532 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f624da640 1 --2- 172.21.15.55:0/3511315532 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f5c0d32a0 0x7f5f5c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f61cd9640 1 -- 172.21.15.55:0/3511315532 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f5c0d32a0 msgr2=0x7f5f5c15f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f61cd9640 1 --2- 172.21.15.55:0/3511315532 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f5c0d32a0 0x7f5f5c15f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f61cd9640 1 -- 172.21.15.55:0/3511315532 >> v1:172.21.15.55:6789/0 conn(0x7f5f5c0cffe0 legacy=0x7f5f5c162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f614d8640 1 -- 172.21.15.55:0/3511315532 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343624115 0 0) 0x7f5f5c0d3780 con 0x7f5f5c0cffe0 2024-05-20T22:08:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.769+0000 7f5f61cd9640 1 -- 172.21.15.55:0/3511315532 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c0d3d40 con 0x7f5f5c0cf690 2024-05-20T22:08:32.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.770+0000 7f5f61cd9640 1 --2- 172.21.15.55:0/3511315532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 0x7f5f5c0cfaa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0097a0 tx=0x7f5f4c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5274f188ad680622 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:32.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.770+0000 7f5f614d8640 1 -- 172.21.15.55:0/3511315532 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c037070 con 0x7f5f5c0cf690 2024-05-20T22:08:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.770+0000 7f5f614d8640 1 -- 172.21.15.55:0/3511315532 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f4c032440 con 0x7f5f5c0cf690 2024-05-20T22:08:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.770+0000 7f5f614d8640 1 -- 172.21.15.55:0/3511315532 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c032810 con 0x7f5f5c0cf690 2024-05-20T22:08:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.771+0000 7f5f6473e640 1 -- 172.21.15.55:0/3511315532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 msgr2=0x7f5f5c0cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.771+0000 7f5f6473e640 1 --2- 172.21.15.55:0/3511315532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 0x7f5f5c0cfaa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0097a0 tx=0x7f5f4c02a1a0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.771+0000 7f5f6473e640 1 -- 172.21.15.55:0/3511315532 shutdown_connections 2024-05-20T22:08:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.771+0000 7f5f6473e640 1 --2- 172.21.15.55:0/3511315532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 0x7f5f5c0cfaa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.771+0000 7f5f6473e640 1 --2- 172.21.15.55:0/3511315532 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f5c0d32a0 0x7f5f5c15f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.771+0000 7f5f6473e640 1 -- 172.21.15.55:0/3511315532 >> 172.21.15.55:0/3511315532 conn(0x7f5f5c0b6de0 msgr2=0x7f5f5c0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.771+0000 7f5f6473e640 1 -- 172.21.15.55:0/3511315532 shutdown_connections 2024-05-20T22:08:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.771+0000 7f5f6473e640 1 -- 172.21.15.55:0/3511315532 wait complete. 2024-05-20T22:08:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.773+0000 7f5f6473e640 1 Processor -- start 2024-05-20T22:08:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.773+0000 7f5f6473e640 1 -- start start 2024-05-20T22:08:32.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f6473e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 0x7f5f5c161290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f6473e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f5c0cffe0 0x7f5f5c161c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f6473e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f5c0d32a0 0x7f5f5c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f6473e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c0d03f0 con 0x7f5f5c0cf690 2024-05-20T22:08:32.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f6473e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c0d4300 con 0x7f5f5c0d32a0 2024-05-20T22:08:32.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f6473e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5f5c0d4820 con 0x7f5f5c0cffe0 2024-05-20T22:08:32.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f624da640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 0x7f5f5c161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f624da640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 0x7f5f5c161290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38444/0 (socket says 172.21.15.55:38444) 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f624da640 1 -- 172.21.15.55:0/2806353097 learned_addr learned my addr 172.21.15.55:0/2806353097 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f61cd9640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f5c0cffe0 0x7f5f5c161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.774+0000 7f5f62cdb640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f5c0d32a0 0x7f5f5c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f624da640 1 -- 172.21.15.55:0/2806353097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f5c0cffe0 msgr2=0x7f5f5c161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f624da640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f5c0cffe0 0x7f5f5c161c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f624da640 1 -- 172.21.15.55:0/2806353097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f5c0d32a0 msgr2=0x7f5f5c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f624da640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f5c0d32a0 0x7f5f5c0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f624da640 1 -- 172.21.15.55:0/2806353097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c1601f0 con 0x7f5f5c0cf690 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f624da640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 0x7f5f5c161290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800d620 tx=0x7f5f5800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f52ffd640 1 -- 172.21.15.55:0/2806353097 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800b620 con 0x7f5f5c0cf690 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f52ffd640 1 -- 172.21.15.55:0/2806353097 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f58019070 con 0x7f5f5c0cf690 2024-05-20T22:08:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f6473e640 1 -- 172.21.15.55:0/2806353097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5c1604d0 con 0x7f5f5c0cf690 2024-05-20T22:08:32.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f52ffd640 1 -- 172.21.15.55:0/2806353097 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800ba00 con 0x7f5f5c0cf690 2024-05-20T22:08:32.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.775+0000 7f5f6473e640 1 -- 172.21.15.55:0/2806353097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5c160a90 con 0x7f5f5c0cf690 2024-05-20T22:08:32.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.776+0000 7f5f6473e640 1 -- 172.21.15.55:0/2806353097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5c069090 con 0x7f5f5c0cf690 2024-05-20T22:08:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.777+0000 7f5f52ffd640 1 -- 172.21.15.55:0/2806353097 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f5800bba0 con 0x7f5f5c0cf690 2024-05-20T22:08:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.777+0000 7f5f52ffd640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5f4006be60 0x7f5f4006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.777+0000 7f5f52ffd640 1 -- 172.21.15.55:0/2806353097 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5f5808b690 con 0x7f5f5c0cf690 2024-05-20T22:08:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.777+0000 7f5f61cd9640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5f4006be60 0x7f5f4006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.778+0000 7f5f61cd9640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5f4006be60 0x7f5f4006e370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c034000 tx=0x7f5f4c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:32.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.782+0000 7f5f52ffd640 1 -- 172.21.15.55:0/2806353097 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5f58059bf0 con 0x7f5f5c0cf690 2024-05-20T22:08:32.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.971+0000 7f5f6473e640 1 -- 172.21.15.55:0/2806353097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f5c0cec80 con 0x7f5f5c0cf690 2024-05-20T22:08:32.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.972+0000 7f5f52ffd640 1 -- 172.21.15.55:0/2806353097 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f5f580594f0 con 0x7f5f5c0cf690 2024-05-20T22:08:32.973 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.973+0000 7f5f6473e640 1 -- 172.21.15.55:0/2806353097 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5f4006be60 msgr2=0x7f5f4006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.973+0000 7f5f6473e640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5f4006be60 0x7f5f4006e370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c034000 tx=0x7f5f4c02e000 comp rx=0 tx=0).stop 2024-05-20T22:08:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.974+0000 7f5f6473e640 1 -- 172.21.15.55:0/2806353097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 msgr2=0x7f5f5c161290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.974+0000 7f5f6473e640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 0x7f5f5c161290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800d620 tx=0x7f5f5800dae0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.974+0000 7f5f6473e640 1 -- 172.21.15.55:0/2806353097 shutdown_connections 2024-05-20T22:08:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.974+0000 7f5f6473e640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5f4006be60 0x7f5f4006e370 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.974+0000 7f5f6473e640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f5c0d32a0 0x7f5f5c0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.974+0000 7f5f6473e640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5f5c0cffe0 0x7f5f5c161c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.974+0000 7f5f6473e640 1 --2- 172.21.15.55:0/2806353097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f5c0cf690 0x7f5f5c161290 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.974+0000 7f5f6473e640 1 -- 172.21.15.55:0/2806353097 >> 172.21.15.55:0/2806353097 conn(0x7f5f5c0b6de0 msgr2=0x7f5f5c15d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:32.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.974+0000 7f5f6473e640 1 -- 172.21.15.55:0/2806353097 shutdown_connections 2024-05-20T22:08:32.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:32.974+0000 7f5f6473e640 1 -- 172.21.15.55:0/2806353097 wait complete. 2024-05-20T22:08:32.987 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:33.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.103+0000 7f6c7bcbc640 1 Processor -- start 2024-05-20T22:08:33.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.103+0000 7f6c7bcbc640 1 -- start start 2024-05-20T22:08:33.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.104+0000 7f6c7bcbc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c740b2470 0x7f6c740b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.104+0000 7f6c7bcbc640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6c7406d0b0 0x7f6c740cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.104+0000 7f6c7bcbc640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c740d1b70 con 0x7f6c740cf900 2024-05-20T22:08:33.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.104+0000 7f6c7bcbc640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f6c740b2880 con 0x7f6c740b2470 2024-05-20T22:08:33.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.104+0000 7f6c7bcbc640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6c740b37b0 con 0x7f6c7406d0b0 2024-05-20T22:08:33.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.105+0000 7f6c79a58640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c740b2470 0x7f6c740b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.105+0000 7f6c79a58640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c740b2470 0x7f6c740b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38460/0 (socket says 172.21.15.55:38460) 2024-05-20T22:08:33.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.105+0000 7f6c79a58640 1 -- 172.21.15.55:0/2192855877 learned_addr learned my addr 172.21.15.55:0/2192855877 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:33.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.105+0000 7f6c79257640 1 --2- 172.21.15.55:0/2192855877 >> v2:172.21.15.138:3300/0 conn(0x7f6c7406d0b0 0x7f6c740cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.105+0000 7f6c79a58640 1 -- 172.21.15.55:0/2192855877 >> v2:172.21.15.138:3300/0 conn(0x7f6c7406d0b0 msgr2=0x7f6c740cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:33.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.105+0000 7f6c79a58640 1 --2- 172.21.15.55:0/2192855877 >> v2:172.21.15.138:3300/0 conn(0x7f6c7406d0b0 0x7f6c740cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.105+0000 7f6c79a58640 1 -- 172.21.15.55:0/2192855877 >> v1:172.21.15.138:6789/0 conn(0x7f6c740cf900 legacy=0x7f6c7408bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:33.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.105+0000 7f6c79a58640 1 -- 172.21.15.55:0/2192855877 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c740cfd10 con 0x7f6c740b2470 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.106+0000 7f6c79a58640 1 --2- 172.21.15.55:0/2192855877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740b2470 0x7f6c740b1eb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6c740b56d0 tx=0x7f6c7002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c04542d705554c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.106+0000 7f6c78a56640 1 -- 172.21.15.55:0/2192855877 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70037070 con 0x7f6c740b2470 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.106+0000 7f6c78a56640 1 -- 172.21.15.55:0/2192855877 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c70032440 con 0x7f6c740b2470 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.106+0000 7f6c78a56640 1 -- 172.21.15.55:0/2192855877 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70032810 con 0x7f6c740b2470 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.107+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2192855877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740b2470 msgr2=0x7f6c740b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.107+0000 7f6c7bcbc640 1 --2- 172.21.15.55:0/2192855877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740b2470 0x7f6c740b1eb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6c740b56d0 tx=0x7f6c7002a230 comp rx=0 tx=0).stop 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.107+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2192855877 shutdown_connections 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.107+0000 7f6c7bcbc640 1 --2- 172.21.15.55:0/2192855877 >> v2:172.21.15.138:3300/0 conn(0x7f6c7406d0b0 0x7f6c740cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.107+0000 7f6c7bcbc640 1 --2- 172.21.15.55:0/2192855877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740b2470 0x7f6c740b1eb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.107+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2192855877 >> 172.21.15.55:0/2192855877 conn(0x7f6c7409be10 msgr2=0x7f6c740d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:33.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.107+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2192855877 shutdown_connections 2024-05-20T22:08:33.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.107+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2192855877 wait complete. 2024-05-20T22:08:33.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.108+0000 7f6c7bcbc640 1 Processor -- start 2024-05-20T22:08:33.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.109+0000 7f6c7bcbc640 1 -- start start 2024-05-20T22:08:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7bcbc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c7406d0b0 0x7f6c7414f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7bcbc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c740b2470 0x7f6c741617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7bcbc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740cf900 0x7f6c740b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7bcbc640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7406de70 con 0x7f6c740cf900 2024-05-20T22:08:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7bcbc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7406db90 con 0x7f6c740b2470 2024-05-20T22:08:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7bcbc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c740b3a30 con 0x7f6c7406d0b0 2024-05-20T22:08:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7a259640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740cf900 0x7f6c740b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7a259640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740cf900 0x7f6c740b0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38472/0 (socket says 172.21.15.55:38472) 2024-05-20T22:08:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7a259640 1 -- 172.21.15.55:0/2531066969 learned_addr learned my addr 172.21.15.55:0/2531066969 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c79a58640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c7406d0b0 0x7f6c7414f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c79257640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c740b2470 0x7f6c741617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7a259640 1 -- 172.21.15.55:0/2531066969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c7406d0b0 msgr2=0x7f6c7414f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7a259640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c7406d0b0 0x7f6c7414f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7a259640 1 -- 172.21.15.55:0/2531066969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c740b2470 msgr2=0x7f6c741617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7a259640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c740b2470 0x7f6c741617c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7a259640 1 -- 172.21.15.55:0/2531066969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7414fa80 con 0x7f6c740cf900 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.110+0000 7f6c7a259640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740cf900 0x7f6c740b0850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00b9a0 tx=0x7f6c6c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.111+0000 7f6c6a7fc640 1 -- 172.21.15.55:0/2531066969 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c019070 con 0x7f6c740cf900 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.111+0000 7f6c6a7fc640 1 -- 172.21.15.55:0/2531066969 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c6c00cba0 con 0x7f6c740cf900 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.111+0000 7f6c6a7fc640 1 -- 172.21.15.55:0/2531066969 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c014440 con 0x7f6c740cf900 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.111+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2531066969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7414f770 con 0x7f6c740cf900 2024-05-20T22:08:33.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.111+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2531066969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c74152360 con 0x7f6c740cf900 2024-05-20T22:08:33.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.112+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2531066969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c74069090 con 0x7f6c740cf900 2024-05-20T22:08:33.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.112+0000 7f6c6a7fc640 1 -- 172.21.15.55:0/2531066969 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c6c01c070 con 0x7f6c740cf900 2024-05-20T22:08:33.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.113+0000 7f6c6a7fc640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6c5806be30 0x7f6c5806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.113+0000 7f6c6a7fc640 1 -- 172.21.15.55:0/2531066969 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7f6c6c08af80 con 0x7f6c740cf900 2024-05-20T22:08:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.113+0000 7f6c79a58640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6c5806be30 0x7f6c5806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.113+0000 7f6c79a58640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6c5806be30 0x7f6c5806e340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c70034000 tx=0x7f6c7002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:33.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.118+0000 7f6c6a7fc640 1 -- 172.21.15.55:0/2531066969 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6c6c059400 con 0x7f6c740cf900 2024-05-20T22:08:33.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.307+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2531066969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6c740d35a0 con 0x7f6c740cf900 2024-05-20T22:08:33.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.307+0000 7f6c6a7fc640 1 -- 172.21.15.55:0/2531066969 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f6c6c058d00 con 0x7f6c740cf900 2024-05-20T22:08:33.308 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2531066969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6c5806be30 msgr2=0x7f6c5806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6c5806be30 0x7f6c5806e340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c70034000 tx=0x7f6c7002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2531066969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740cf900 msgr2=0x7f6c740b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740cf900 0x7f6c740b0850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00b9a0 tx=0x7f6c6c00be60 comp rx=0 tx=0).stop 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2531066969 shutdown_connections 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6c5806be30 0x7f6c5806e340 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c740cf900 0x7f6c740b0850 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c740b2470 0x7f6c741617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 --2- 172.21.15.55:0/2531066969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c7406d0b0 0x7f6c7414f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2531066969 >> 172.21.15.55:0/2531066969 conn(0x7f6c7409be10 msgr2=0x7f6c7408bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2531066969 shutdown_connections 2024-05-20T22:08:33.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.310+0000 7f6c7bcbc640 1 -- 172.21.15.55:0/2531066969 wait complete. 2024-05-20T22:08:33.322 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:33.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.438+0000 7fb50a06e640 1 Processor -- start 2024-05-20T22:08:33.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.438+0000 7fb50a06e640 1 -- start start 2024-05-20T22:08:33.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb50a06e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb50415d220 0x7fb50406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb50a06e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb5040b2ca0 0x7fb5040b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb50a06e640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5040b35f0 con 0x7fb50406cae0 2024-05-20T22:08:33.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb50a06e640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fb50406b500 con 0x7fb5040b2ca0 2024-05-20T22:08:33.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb50a06e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb50406b7e0 con 0x7fb50415d220 2024-05-20T22:08:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb503fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb5040b2ca0 0x7fb5040b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb502ffd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb50406cae0 0x7fb5040c4c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56814/0 (socket says 172.21.15.55:56814) 2024-05-20T22:08:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb502ffd640 1 -- 172.21.15.55:0/3812073510 learned_addr learned my addr 172.21.15.55:0/3812073510 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb5037fe640 1 --2- 172.21.15.55:0/3812073510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb50415d220 0x7fb50406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb503fff640 1 -- 172.21.15.55:0/3812073510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb50415d220 msgr2=0x7fb50406c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb503fff640 1 --2- 172.21.15.55:0/3812073510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb50415d220 0x7fb50406c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb503fff640 1 -- 172.21.15.55:0/3812073510 >> v1:172.21.15.55:6789/0 conn(0x7fb50406cae0 legacy=0x7fb5040c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb503fff640 1 -- 172.21.15.55:0/3812073510 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50406bac0 con 0x7fb5040b2ca0 2024-05-20T22:08:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.440+0000 7fb5027fc640 1 -- 172.21.15.55:0/3812073510 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3877146022 0 0) 0x7fb4f8001110 con 0x7fb50406cae0 2024-05-20T22:08:33.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.441+0000 7fb503fff640 1 --2- 172.21.15.55:0/3812073510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5040b2ca0 0x7fb5040b30b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009d90 tx=0x7fb4fc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef321c27c9210f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:33.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.441+0000 7fb5027fc640 1 -- 172.21.15.55:0/3812073510 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc007740 con 0x7fb5040b2ca0 2024-05-20T22:08:33.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.441+0000 7fb5027fc640 1 -- 172.21.15.55:0/3812073510 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc0078e0 con 0x7fb5040b2ca0 2024-05-20T22:08:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.441+0000 7fb5027fc640 1 -- 172.21.15.55:0/3812073510 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc007cb0 con 0x7fb5040b2ca0 2024-05-20T22:08:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.442+0000 7fb50a06e640 1 -- 172.21.15.55:0/3812073510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5040b2ca0 msgr2=0x7fb5040b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.442+0000 7fb50a06e640 1 --2- 172.21.15.55:0/3812073510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5040b2ca0 0x7fb5040b30b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009d90 tx=0x7fb4fc02a710 comp rx=0 tx=0).stop 2024-05-20T22:08:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.442+0000 7fb50a06e640 1 -- 172.21.15.55:0/3812073510 shutdown_connections 2024-05-20T22:08:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.442+0000 7fb50a06e640 1 --2- 172.21.15.55:0/3812073510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5040b2ca0 0x7fb5040b30b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.442+0000 7fb50a06e640 1 --2- 172.21.15.55:0/3812073510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb50415d220 0x7fb50406c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.442+0000 7fb50a06e640 1 -- 172.21.15.55:0/3812073510 >> 172.21.15.55:0/3812073510 conn(0x7fb5040b62c0 msgr2=0x7fb50409a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.442+0000 7fb50a06e640 1 -- 172.21.15.55:0/3812073510 shutdown_connections 2024-05-20T22:08:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.442+0000 7fb50a06e640 1 -- 172.21.15.55:0/3812073510 wait complete. 2024-05-20T22:08:33.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.444+0000 7fb50a06e640 1 Processor -- start 2024-05-20T22:08:33.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.444+0000 7fb50a06e640 1 -- start start 2024-05-20T22:08:33.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb50a06e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb50406cae0 0x7fb5040b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb50a06e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5040b2ca0 0x7fb50414d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb50a06e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50415d220 0x7fb5040c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb50a06e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb5040c5370 con 0x7fb50415d220 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb50a06e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb50406c080 con 0x7fb5040b2ca0 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb50a06e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb5040c54f0 con 0x7fb50406cae0 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb503fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50415d220 0x7fb5040c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb503fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50415d220 0x7fb5040c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38484/0 (socket says 172.21.15.55:38484) 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb503fff640 1 -- 172.21.15.55:0/1428111125 learned_addr learned my addr 172.21.15.55:0/1428111125 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb5037fe640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb50406cae0 0x7fb5040b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.445+0000 7fb502ffd640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5040b2ca0 0x7fb50414d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb503fff640 1 -- 172.21.15.55:0/1428111125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb50406cae0 msgr2=0x7fb5040b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb503fff640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb50406cae0 0x7fb5040b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb503fff640 1 -- 172.21.15.55:0/1428111125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5040b2ca0 msgr2=0x7fb50414d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb503fff640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5040b2ca0 0x7fb50414d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb503fff640 1 -- 172.21.15.55:0/1428111125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5040b0ba0 con 0x7fb50415d220 2024-05-20T22:08:33.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb503fff640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50415d220 0x7fb5040c94e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009eb0 tx=0x7fb4fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:33.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb4ebfff640 1 -- 172.21.15.55:0/1428111125 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc03f070 con 0x7fb50415d220 2024-05-20T22:08:33.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb4ebfff640 1 -- 172.21.15.55:0/1428111125 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc031be0 con 0x7fb50415d220 2024-05-20T22:08:33.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb4ebfff640 1 -- 172.21.15.55:0/1428111125 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc043410 con 0x7fb50415d220 2024-05-20T22:08:33.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb50a06e640 1 -- 172.21.15.55:0/1428111125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5040b0e80 con 0x7fb50415d220 2024-05-20T22:08:33.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.446+0000 7fb50a06e640 1 -- 172.21.15.55:0/1428111125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5040c9ea0 con 0x7fb50415d220 2024-05-20T22:08:33.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.448+0000 7fb50a06e640 1 -- 172.21.15.55:0/1428111125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5040ca100 con 0x7fb50415d220 2024-05-20T22:08:33.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.448+0000 7fb4ebfff640 1 -- 172.21.15.55:0/1428111125 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4fc03b410 con 0x7fb50415d220 2024-05-20T22:08:33.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.448+0000 7fb4ebfff640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4f006be30 0x7fb4f006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.449+0000 7fb4ebfff640 1 -- 172.21.15.55:0/1428111125 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7fb4fc0ac5e0 con 0x7fb50415d220 2024-05-20T22:08:33.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.449+0000 7fb5037fe640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4f006be30 0x7fb4f006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.449+0000 7fb5037fe640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4f006be30 0x7fb4f006e340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40079a0 tx=0x7fb4f4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:33.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.453+0000 7fb4ebfff640 1 -- 172.21.15.55:0/1428111125 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb4fc07a540 con 0x7fb50415d220 2024-05-20T22:08:33.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.637+0000 7fb50a06e640 1 -- 172.21.15.55:0/1428111125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb504069090 con 0x7fb50415d220 2024-05-20T22:08:33.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.637+0000 7fb4ebfff640 1 -- 172.21.15.55:0/1428111125 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fb4fc07e840 con 0x7fb50415d220 2024-05-20T22:08:33.638 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 -- 172.21.15.55:0/1428111125 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4f006be30 msgr2=0x7fb4f006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4f006be30 0x7fb4f006e340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40079a0 tx=0x7fb4f4008040 comp rx=0 tx=0).stop 2024-05-20T22:08:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 -- 172.21.15.55:0/1428111125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50415d220 msgr2=0x7fb5040c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50415d220 0x7fb5040c94e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009eb0 tx=0x7fb4fc004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 -- 172.21.15.55:0/1428111125 shutdown_connections 2024-05-20T22:08:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4f006be30 0x7fb4f006e340 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50415d220 0x7fb5040c94e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5040b2ca0 0x7fb50414d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 --2- 172.21.15.55:0/1428111125 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb50406cae0 0x7fb5040b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 -- 172.21.15.55:0/1428111125 >> 172.21.15.55:0/1428111125 conn(0x7fb5040b62c0 msgr2=0x7fb5040c5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 -- 172.21.15.55:0/1428111125 shutdown_connections 2024-05-20T22:08:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.639+0000 7fb50a06e640 1 -- 172.21.15.55:0/1428111125 wait complete. 2024-05-20T22:08:33.652 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:33.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.770+0000 7f03be738640 1 Processor -- start 2024-05-20T22:08:33.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.770+0000 7f03be738640 1 -- start start 2024-05-20T22:08:33.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.771+0000 7f03be738640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b80cd520 0x7f03b808c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.771+0000 7f03be738640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f03b80d0860 0x7f03b80b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.771+0000 7f03be738640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b80b1f30 con 0x7f03b80cf270 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.771+0000 7f03be738640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f03b80d1020 con 0x7f03b80d0860 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.771+0000 7f03be738640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03b80d1300 con 0x7f03b80cd520 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.771+0000 7f03bccd5640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f03b80cf270 0x7f03b80cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56826/0 (socket says 172.21.15.55:56826) 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.771+0000 7f03bccd5640 1 -- 172.21.15.55:0/3863500360 learned_addr learned my addr 172.21.15.55:0/3863500360 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b7fff640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b80cd520 0x7f03b808c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b77fe640 1 --2- 172.21.15.55:0/3863500360 >> v2:172.21.15.138:3300/0 conn(0x7f03b80d0860 0x7f03b80b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127540853 0 0) 0x7f03b80b1f30 con 0x7f03b80cf270 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03a0003520 con 0x7f03b80cf270 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2810531064 0 0) 0x7f03a0003520 con 0x7f03b80cf270 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b80cd520 msgr2=0x7f03b808c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b80cd520 0x7f03b808c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 >> v2:172.21.15.138:3300/0 conn(0x7f03b80d0860 msgr2=0x7f03b80b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 --2- 172.21.15.55:0/3863500360 >> v2:172.21.15.138:3300/0 conn(0x7f03b80d0860 0x7f03b80b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b80d15e0 con 0x7f03b80cf270 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f03b00019f0 con 0x7f03b80cf270 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 >> v1:172.21.15.55:6789/0 conn(0x7f03b80cf270 legacy=0x7f03b80cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a00039b0 0x7f03a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b80cd520 0x7f03a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a0004d40 0x7f03a0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f03b80d1020 con 0x7f03a00039b0 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03b80d1300 con 0x7f03a0004d40 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03a000ba40 con 0x7f03b80cd520 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b7fff640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a00039b0 0x7f03a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03bccd5640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a0004d40 0x7f03a0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.772+0000 7f03b77fe640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b80cd520 0x7f03a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.773+0000 7f03b7fff640 1 -- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b80cd520 msgr2=0x7f03a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.773+0000 7f03b7fff640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b80cd520 0x7f03a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.773+0000 7f03b7fff640 1 -- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a0004d40 msgr2=0x7f03a0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.773+0000 7f03b7fff640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a0004d40 0x7f03a0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.773+0000 7f03b7fff640 1 -- 172.21.15.55:0/3863500360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a0003520 con 0x7f03a00039b0 2024-05-20T22:08:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.773+0000 7f03bccd5640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a0004d40 0x7f03a0005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.773+0000 7f03b7fff640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a00039b0 0x7f03a0003e20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009ac0 tx=0x7f03a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19b990bd64ed960 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.773+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8036070 con 0x7f03a00039b0 2024-05-20T22:08:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.773+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a8031400 con 0x7f03a00039b0 2024-05-20T22:08:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.773+0000 7f03b6ffd640 1 -- 172.21.15.55:0/3863500360 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8009690 con 0x7f03a00039b0 2024-05-20T22:08:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.774+0000 7f03be738640 1 -- 172.21.15.55:0/3863500360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a00039b0 msgr2=0x7f03a0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.774+0000 7f03be738640 1 --2- 172.21.15.55:0/3863500360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a00039b0 0x7f03a0003e20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009ac0 tx=0x7f03a0005830 comp rx=0 tx=0).stop 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.774+0000 7f03bccd5640 1 -- 172.21.15.55:0/3863500360 reap_dead start 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.774+0000 7f03be738640 1 -- 172.21.15.55:0/3863500360 shutdown_connections 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.774+0000 7f03be738640 1 -- 172.21.15.55:0/3863500360 >> 172.21.15.55:0/3863500360 conn(0x7f03b80d2860 msgr2=0x7f03b80d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.774+0000 7f03be738640 1 -- 172.21.15.55:0/3863500360 shutdown_connections 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.774+0000 7f03be738640 1 -- 172.21.15.55:0/3863500360 wait complete. 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.776+0000 7f03be738640 1 Processor -- start 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.776+0000 7f03be738640 1 -- start start 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.777+0000 7f03be738640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a00039b0 0x7f03b8151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.777+0000 7f03be738640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b81554a0 0x7f03b80b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.777+0000 7f03be738640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b815fe80 0x7f03b8160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.777+0000 7f03be738640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f03b80d0c70 con 0x7f03b81554a0 2024-05-20T22:08:33.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.777+0000 7f03be738640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03b80b2cb0 con 0x7f03a00039b0 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.777+0000 7f03be738640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03b80b6870 con 0x7f03b815fe80 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b81554a0 0x7f03b80b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b81554a0 0x7f03b80b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38514/0 (socket says 172.21.15.55:38514) 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b77fe640 1 -- 172.21.15.55:0/2162018512 learned_addr learned my addr 172.21.15.55:0/2162018512 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03bccd5640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b815fe80 0x7f03b8160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b7fff640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a00039b0 0x7f03b8151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b77fe640 1 -- 172.21.15.55:0/2162018512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b815fe80 msgr2=0x7f03b8160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b77fe640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b815fe80 0x7f03b8160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b77fe640 1 -- 172.21.15.55:0/2162018512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a00039b0 msgr2=0x7f03b8151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b77fe640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a00039b0 0x7f03b8151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b77fe640 1 -- 172.21.15.55:0/2162018512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b815ede0 con 0x7f03b81554a0 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b77fe640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b81554a0 0x7f03b80b5f20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0115b0 tx=0x7f03ac011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b4ff9640 1 -- 172.21.15.55:0/2162018512 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac015870 con 0x7f03b81554a0 2024-05-20T22:08:33.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b4ff9640 1 -- 172.21.15.55:0/2162018512 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac015a10 con 0x7f03b81554a0 2024-05-20T22:08:33.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03be738640 1 -- 172.21.15.55:0/2162018512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b815f0c0 con 0x7f03b81554a0 2024-05-20T22:08:33.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03b4ff9640 1 -- 172.21.15.55:0/2162018512 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac015dc0 con 0x7f03b81554a0 2024-05-20T22:08:33.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.778+0000 7f03be738640 1 -- 172.21.15.55:0/2162018512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b815f680 con 0x7f03b81554a0 2024-05-20T22:08:33.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.779+0000 7f03be738640 1 -- 172.21.15.55:0/2162018512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b8069090 con 0x7f03b81554a0 2024-05-20T22:08:33.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.780+0000 7f03b4ff9640 1 -- 172.21.15.55:0/2162018512 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03ac00f480 con 0x7f03b81554a0 2024-05-20T22:08:33.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.780+0000 7f03b4ff9640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03a406be60 0x7f03a406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:33.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.780+0000 7f03b4ff9640 1 -- 172.21.15.55:0/2162018512 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7f03ac08bc40 con 0x7f03b81554a0 2024-05-20T22:08:33.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.780+0000 7f03b7fff640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03a406be60 0x7f03a406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:33.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.781+0000 7f03b7fff640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03a406be60 0x7f03a406e370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f03a8033000 tx=0x7f03a8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:33.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.785+0000 7f03b4ff9640 1 -- 172.21.15.55:0/2162018512 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f03ac019030 con 0x7f03b81554a0 2024-05-20T22:08:33.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.976+0000 7f03be738640 1 -- 172.21.15.55:0/2162018512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f03b80b27c0 con 0x7f03b81554a0 2024-05-20T22:08:33.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.977+0000 7f03b4ff9640 1 -- 172.21.15.55:0/2162018512 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f03ac05a280 con 0x7f03b81554a0 2024-05-20T22:08:33.978 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:33.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 -- 172.21.15.55:0/2162018512 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03a406be60 msgr2=0x7f03a406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03a406be60 0x7f03a406e370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f03a8033000 tx=0x7f03a8008030 comp rx=0 tx=0).stop 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 -- 172.21.15.55:0/2162018512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b81554a0 msgr2=0x7f03b80b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b81554a0 0x7f03b80b5f20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0115b0 tx=0x7f03ac011a70 comp rx=0 tx=0).stop 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 -- 172.21.15.55:0/2162018512 shutdown_connections 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03a406be60 0x7f03a406e370 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b815fe80 0x7f03b8160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b81554a0 0x7f03b80b5f20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 --2- 172.21.15.55:0/2162018512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03a00039b0 0x7f03b8151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 -- 172.21.15.55:0/2162018512 >> 172.21.15.55:0/2162018512 conn(0x7f03b80d2860 msgr2=0x7f03b80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 -- 172.21.15.55:0/2162018512 shutdown_connections 2024-05-20T22:08:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:33.979+0000 7f03be738640 1 -- 172.21.15.55:0/2162018512 wait complete. 2024-05-20T22:08:33.993 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:34.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.107+0000 7f61631f5640 1 Processor -- start 2024-05-20T22:08:34.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.107+0000 7f61631f5640 1 -- start start 2024-05-20T22:08:34.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.108+0000 7f61631f5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f615c0b3b30 0x7f615c0b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.108+0000 7f61631f5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f615c15f100 0x7f615c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.108+0000 7f61631f5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f615c06c5c0 0x7f615c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.108+0000 7f61631f5640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f615c0d3dd0 con 0x7f615c06c5c0 2024-05-20T22:08:34.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.108+0000 7f61631f5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f615c0d40b0 con 0x7f615c15f100 2024-05-20T22:08:34.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.108+0000 7f61631f5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f615c0d4390 con 0x7f615c0b3b30 2024-05-20T22:08:34.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.108+0000 7f6161792640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f615c06c5c0 0x7f615c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.109+0000 7f6161792640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f615c06c5c0 0x7f615c06c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38518/0 (socket says 172.21.15.55:38518) 2024-05-20T22:08:34.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.109+0000 7f6161792640 1 -- 172.21.15.55:0/3738632494 learned_addr learned my addr 172.21.15.55:0/3738632494 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:34.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.109+0000 7f6160f91640 1 --2- 172.21.15.55:0/3738632494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f615c0b3b30 0x7f615c0b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.109+0000 7f615bfff640 1 --2- 172.21.15.55:0/3738632494 >> v2:172.21.15.138:3300/0 conn(0x7f615c15f100 0x7f615c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.109+0000 7f6161792640 1 -- 172.21.15.55:0/3738632494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f615c0b3b30 msgr2=0x7f615c0b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:34.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.109+0000 7f6161792640 1 --2- 172.21.15.55:0/3738632494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f615c0b3b30 0x7f615c0b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.109+0000 7f6161792640 1 -- 172.21.15.55:0/3738632494 >> v2:172.21.15.138:3300/0 conn(0x7f615c15f100 msgr2=0x7f615c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:34.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.109+0000 7f6161792640 1 --2- 172.21.15.55:0/3738632494 >> v2:172.21.15.138:3300/0 conn(0x7f615c15f100 0x7f615c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.109+0000 7f6161792640 1 -- 172.21.15.55:0/3738632494 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c06b5a0 con 0x7f615c06c5c0 2024-05-20T22:08:34.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.110+0000 7f6161792640 1 --2- 172.21.15.55:0/3738632494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 0x7f615c06c9d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f615c0b6c90 tx=0x7f615402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708e74746ec702e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:34.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.110+0000 7f615b7fe640 1 -- 172.21.15.55:0/3738632494 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154036070 con 0x7f615c06c5c0 2024-05-20T22:08:34.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.110+0000 7f615b7fe640 1 -- 172.21.15.55:0/3738632494 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6154031600 con 0x7f615c06c5c0 2024-05-20T22:08:34.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.110+0000 7f615b7fe640 1 -- 172.21.15.55:0/3738632494 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61540319d0 con 0x7f615c06c5c0 2024-05-20T22:08:34.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.110+0000 7f61631f5640 1 -- 172.21.15.55:0/3738632494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 msgr2=0x7f615c06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.110+0000 7f61631f5640 1 --2- 172.21.15.55:0/3738632494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 0x7f615c06c9d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f615c0b6c90 tx=0x7f615402a6a0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.111+0000 7f61631f5640 1 -- 172.21.15.55:0/3738632494 shutdown_connections 2024-05-20T22:08:34.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.111+0000 7f61631f5640 1 --2- 172.21.15.55:0/3738632494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 0x7f615c06c9d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.111+0000 7f61631f5640 1 --2- 172.21.15.55:0/3738632494 >> v2:172.21.15.138:3300/0 conn(0x7f615c15f100 0x7f615c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.111+0000 7f61631f5640 1 --2- 172.21.15.55:0/3738632494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f615c0b3b30 0x7f615c0b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.111+0000 7f61631f5640 1 -- 172.21.15.55:0/3738632494 >> 172.21.15.55:0/3738632494 conn(0x7f615c0d51d0 msgr2=0x7f615c0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:34.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.111+0000 7f61631f5640 1 -- 172.21.15.55:0/3738632494 shutdown_connections 2024-05-20T22:08:34.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.111+0000 7f61631f5640 1 -- 172.21.15.55:0/3738632494 wait complete. 2024-05-20T22:08:34.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.112+0000 7f61631f5640 1 Processor -- start 2024-05-20T22:08:34.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.112+0000 7f61631f5640 1 -- start start 2024-05-20T22:08:34.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f61631f5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 0x7f615c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f61631f5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f615c0b3b30 0x7f615c1f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f61631f5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f615c15f100 0x7f615c160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f61631f5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f615c162b70 con 0x7f615c06c5c0 2024-05-20T22:08:34.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f61631f5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f615c06bb60 con 0x7f615c15f100 2024-05-20T22:08:34.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f61631f5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f615c162cf0 con 0x7f615c0b3b30 2024-05-20T22:08:34.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f6160f91640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 0x7f615c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f6160f91640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 0x7f615c1533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38522/0 (socket says 172.21.15.55:38522) 2024-05-20T22:08:34.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f6160f91640 1 -- 172.21.15.55:0/1329756501 learned_addr learned my addr 172.21.15.55:0/1329756501 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:34.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f615bfff640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f615c0b3b30 0x7f615c1f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f6161792640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f615c15f100 0x7f615c160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f6160f91640 1 -- 172.21.15.55:0/1329756501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f615c0b3b30 msgr2=0x7f615c1f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f6160f91640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f615c0b3b30 0x7f615c1f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f6160f91640 1 -- 172.21.15.55:0/1329756501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f615c15f100 msgr2=0x7f615c160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f6160f91640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f615c15f100 0x7f615c160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f6160f91640 1 -- 172.21.15.55:0/1329756501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c15ac00 con 0x7f615c06c5c0 2024-05-20T22:08:34.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.114+0000 7f6161792640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f615c15f100 0x7f615c160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:34.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.115+0000 7f6160f91640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 0x7f615c1533c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f615000e840 tx=0x7f615000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:34.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.115+0000 7f61597fa640 1 -- 172.21.15.55:0/1329756501 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150017070 con 0x7f615c06c5c0 2024-05-20T22:08:34.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.115+0000 7f61597fa640 1 -- 172.21.15.55:0/1329756501 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615000c730 con 0x7f615c06c5c0 2024-05-20T22:08:34.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.115+0000 7f61597fa640 1 -- 172.21.15.55:0/1329756501 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615000cb00 con 0x7f615c06c5c0 2024-05-20T22:08:34.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.115+0000 7f61631f5640 1 -- 172.21.15.55:0/1329756501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615c15aea0 con 0x7f615c06c5c0 2024-05-20T22:08:34.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.115+0000 7f61631f5640 1 -- 172.21.15.55:0/1329756501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615c15b460 con 0x7f615c06c5c0 2024-05-20T22:08:34.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.116+0000 7f61631f5640 1 -- 172.21.15.55:0/1329756501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f615c069090 con 0x7f615c06c5c0 2024-05-20T22:08:34.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.116+0000 7f61597fa640 1 -- 172.21.15.55:0/1329756501 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f615001a070 con 0x7f615c06c5c0 2024-05-20T22:08:34.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.117+0000 7f61597fa640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f614006be60 0x7f614006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.117+0000 7f61597fa640 1 -- 172.21.15.55:0/1329756501 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f615008ad60 con 0x7f615c06c5c0 2024-05-20T22:08:34.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.117+0000 7f615bfff640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f614006be60 0x7f614006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.118+0000 7f615bfff640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f614006be60 0x7f614006e370 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f615c151d30 tx=0x7f6148008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:34.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.122+0000 7f61597fa640 1 -- 172.21.15.55:0/1329756501 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6150059280 con 0x7f615c06c5c0 2024-05-20T22:08:34.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.308+0000 7f61631f5640 1 -- 172.21.15.55:0/1329756501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f615c0d2ee0 con 0x7f615c06c5c0 2024-05-20T22:08:34.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.309+0000 7f61597fa640 1 -- 172.21.15.55:0/1329756501 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7f6150058b80 con 0x7f615c06c5c0 2024-05-20T22:08:34.311 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:34.311 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:34.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 -- 172.21.15.55:0/1329756501 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f614006be60 msgr2=0x7f614006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f614006be60 0x7f614006e370 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f615c151d30 tx=0x7f6148008040 comp rx=0 tx=0).stop 2024-05-20T22:08:34.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 -- 172.21.15.55:0/1329756501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 msgr2=0x7f615c1533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 0x7f615c1533c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f615000e840 tx=0x7f615000ed00 comp rx=0 tx=0).stop 2024-05-20T22:08:34.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 -- 172.21.15.55:0/1329756501 shutdown_connections 2024-05-20T22:08:34.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f614006be60 0x7f614006e370 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f615c15f100 0x7f615c160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f615c0b3b30 0x7f615c1f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 --2- 172.21.15.55:0/1329756501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06c5c0 0x7f615c1533c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 -- 172.21.15.55:0/1329756501 >> 172.21.15.55:0/1329756501 conn(0x7f615c0d51d0 msgr2=0x7f615c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:34.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 -- 172.21.15.55:0/1329756501 shutdown_connections 2024-05-20T22:08:34.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.311+0000 7f61631f5640 1 -- 172.21.15.55:0/1329756501 wait complete. 2024-05-20T22:08:34.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:34.325 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:34.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.444+0000 7faa6e28b640 1 Processor -- start 2024-05-20T22:08:34.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.444+0000 7faa6e28b640 1 -- start start 2024-05-20T22:08:34.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa6e28b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faa6806e630 0x7faa680afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa6e28b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6806c5a0 0x7faa6815f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa6e28b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa6806ebd0 con 0x7faa6815c4e0 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa6e28b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7faa6806eeb0 con 0x7faa6806e630 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa6e28b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa6806f190 con 0x7faa6806c5a0 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa67fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faa6806e630 0x7faa680afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa67fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faa6806e630 0x7faa680afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38532/0 (socket says 172.21.15.55:38532) 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa67fff640 1 -- 172.21.15.55:0/2248049433 learned_addr learned my addr 172.21.15.55:0/2248049433 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa67fff640 1 -- 172.21.15.55:0/2248049433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6806c5a0 msgr2=0x7faa6815f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa6c828640 1 --2- 172.21.15.55:0/2248049433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6806c5a0 0x7faa6815f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.445+0000 7faa67fff640 1 --2- 172.21.15.55:0/2248049433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6806c5a0 0x7faa6815f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.446+0000 7faa67fff640 1 -- 172.21.15.55:0/2248049433 >> v1:172.21.15.138:6789/0 conn(0x7faa6815c4e0 legacy=0x7faa6815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:34.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.446+0000 7faa67fff640 1 -- 172.21.15.55:0/2248049433 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6806f470 con 0x7faa6806e630 2024-05-20T22:08:34.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.446+0000 7faa67fff640 1 --2- 172.21.15.55:0/2248049433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 0x7faa680afcd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa58009870 tx=0x7faa5802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650f50968cabfee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.446+0000 7faa677fe640 1 -- 172.21.15.55:0/2248049433 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58037070 con 0x7faa6806e630 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.446+0000 7faa677fe640 1 -- 172.21.15.55:0/2248049433 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa58032440 con 0x7faa6806e630 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.446+0000 7faa677fe640 1 -- 172.21.15.55:0/2248049433 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58032810 con 0x7faa6806e630 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.447+0000 7faa6e28b640 1 -- 172.21.15.55:0/2248049433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 msgr2=0x7faa680afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.447+0000 7faa6e28b640 1 --2- 172.21.15.55:0/2248049433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 0x7faa680afcd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa58009870 tx=0x7faa5802a210 comp rx=0 tx=0).stop 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.447+0000 7faa6e28b640 1 -- 172.21.15.55:0/2248049433 shutdown_connections 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.447+0000 7faa6e28b640 1 --2- 172.21.15.55:0/2248049433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6806c5a0 0x7faa6815f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.447+0000 7faa6e28b640 1 --2- 172.21.15.55:0/2248049433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 0x7faa680afcd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.447+0000 7faa6e28b640 1 -- 172.21.15.55:0/2248049433 >> 172.21.15.55:0/2248049433 conn(0x7faa680b3980 msgr2=0x7faa680b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.447+0000 7faa6e28b640 1 -- 172.21.15.55:0/2248049433 shutdown_connections 2024-05-20T22:08:34.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.447+0000 7faa6e28b640 1 -- 172.21.15.55:0/2248049433 wait complete. 2024-05-20T22:08:34.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.449+0000 7faa6e28b640 1 Processor -- start 2024-05-20T22:08:34.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.449+0000 7faa6e28b640 1 -- start start 2024-05-20T22:08:34.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.450+0000 7faa6e28b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6806c5a0 0x7faa680cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.450+0000 7faa6e28b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 0x7faa680ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.450+0000 7faa6e28b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6815c4e0 0x7faa680caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.450+0000 7faa6e28b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7faa6815d000 con 0x7faa6806e630 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.450+0000 7faa6e28b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa680b0210 con 0x7faa6806c5a0 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.450+0000 7faa6e28b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa6815d180 con 0x7faa6815c4e0 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.450+0000 7faa5ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 0x7faa680ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.450+0000 7faa5ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 0x7faa680ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38536/0 (socket says 172.21.15.55:38536) 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.450+0000 7faa5ffff640 1 -- 172.21.15.55:0/1810520935 learned_addr learned my addr 172.21.15.55:0/1810520935 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa6c828640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6815c4e0 0x7faa680caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa67fff640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6806c5a0 0x7faa680cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa5ffff640 1 -- 172.21.15.55:0/1810520935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6815c4e0 msgr2=0x7faa680caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa5ffff640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6815c4e0 0x7faa680caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa5ffff640 1 -- 172.21.15.55:0/1810520935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6806c5a0 msgr2=0x7faa680cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa5ffff640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6806c5a0 0x7faa680cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa5ffff640 1 -- 172.21.15.55:0/1810520935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa680ca1e0 con 0x7faa6806e630 2024-05-20T22:08:34.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa6c828640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6815c4e0 0x7faa680caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:34.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa5ffff640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 0x7faa680ca9f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b420 tx=0x7faa5000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:34.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa657fa640 1 -- 172.21.15.55:0/1810520935 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50011830 con 0x7faa6806e630 2024-05-20T22:08:34.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa657fa640 1 -- 172.21.15.55:0/1810520935 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa50016070 con 0x7faa6806e630 2024-05-20T22:08:34.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa6e28b640 1 -- 172.21.15.55:0/1810520935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa68143f80 con 0x7faa6806e630 2024-05-20T22:08:34.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa657fa640 1 -- 172.21.15.55:0/1810520935 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50011c10 con 0x7faa6806e630 2024-05-20T22:08:34.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.451+0000 7faa6e28b640 1 -- 172.21.15.55:0/1810520935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa68144540 con 0x7faa6806e630 2024-05-20T22:08:34.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.452+0000 7faa6e28b640 1 -- 172.21.15.55:0/1810520935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa68069090 con 0x7faa6806e630 2024-05-20T22:08:34.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.453+0000 7faa657fa640 1 -- 172.21.15.55:0/1810520935 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa5001b460 con 0x7faa6806e630 2024-05-20T22:08:34.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.453+0000 7faa657fa640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa4806be30 0x7faa4806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.453+0000 7faa657fa640 1 -- 172.21.15.55:0/1810520935 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7faa5008b010 con 0x7faa6806e630 2024-05-20T22:08:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.453+0000 7faa67fff640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa4806be30 0x7faa4806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.454+0000 7faa67fff640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa4806be30 0x7faa4806e340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa58034000 tx=0x7faa5802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:34.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.458+0000 7faa657fa640 1 -- 172.21.15.55:0/1810520935 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faa500594b0 con 0x7faa6806e630 2024-05-20T22:08:34.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.642+0000 7faa6e28b640 1 -- 172.21.15.55:0/1810520935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa680c9f70 con 0x7faa6806e630 2024-05-20T22:08:34.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.642+0000 7faa657fa640 1 -- 172.21.15.55:0/1810520935 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7faa50058db0 con 0x7faa6806e630 2024-05-20T22:08:34.645 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:34.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 -- 172.21.15.55:0/1810520935 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa4806be30 msgr2=0x7faa4806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa4806be30 0x7faa4806e340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa58034000 tx=0x7faa5802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:34.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 -- 172.21.15.55:0/1810520935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 msgr2=0x7faa680ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 0x7faa680ca9f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b420 tx=0x7faa5000b8e0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 -- 172.21.15.55:0/1810520935 shutdown_connections 2024-05-20T22:08:34.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa4806be30 0x7faa4806e340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6815c4e0 0x7faa680caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa6806e630 0x7faa680ca9f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 --2- 172.21.15.55:0/1810520935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6806c5a0 0x7faa680cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 -- 172.21.15.55:0/1810520935 >> 172.21.15.55:0/1810520935 conn(0x7faa680b3980 msgr2=0x7faa6815c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:34.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 -- 172.21.15.55:0/1810520935 shutdown_connections 2024-05-20T22:08:34.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.644+0000 7faa6e28b640 1 -- 172.21.15.55:0/1810520935 wait complete. 2024-05-20T22:08:34.657 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:34.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.773+0000 7f76a5302640 1 Processor -- start 2024-05-20T22:08:34.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.773+0000 7f76a5302640 1 -- start start 2024-05-20T22:08:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f76a5302640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76a0161140 0x7f76a0072690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f76a5302640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a0157870 con 0x7f76a015e960 2024-05-20T22:08:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f76a5302640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a015ed70 con 0x7f76a015e550 2024-05-20T22:08:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f76a5302640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f76a0073310 con 0x7f76a0161140 2024-05-20T22:08:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f769e7fc640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f76a015e960 0x7f76a0160a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56840/0 (socket says 172.21.15.55:56840) 2024-05-20T22:08:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f769f7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76a0161140 0x7f76a0072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f769e7fc640 1 -- 172.21.15.55:0/1558235063 learned_addr learned my addr 172.21.15.55:0/1558235063 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f769f7fe640 1 -- 172.21.15.55:0/1558235063 >> v1:172.21.15.138:6789/0 conn(0x7f76a015e550 legacy=0x7f76a006bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f769f7fe640 1 -- 172.21.15.55:0/1558235063 >> v1:172.21.15.55:6789/0 conn(0x7f76a015e960 legacy=0x7f76a0160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f769f7fe640 1 -- 172.21.15.55:0/1558235063 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a00735f0 con 0x7f76a0161140 2024-05-20T22:08:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.775+0000 7f769dffb640 1 -- 172.21.15.55:0/1558235063 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2586607944 0 0) 0x7f76a0157870 con 0x7f76a015e960 2024-05-20T22:08:34.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.776+0000 7f769f7fe640 1 --2- 172.21.15.55:0/1558235063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a0161140 0x7f76a0072690 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7690009d90 tx=0x7f769002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac53327662ebf80 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:34.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.776+0000 7f769dffb640 1 -- 172.21.15.55:0/1558235063 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690007740 con 0x7f76a0161140 2024-05-20T22:08:34.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.776+0000 7f769dffb640 1 -- 172.21.15.55:0/1558235063 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76900078e0 con 0x7f76a0161140 2024-05-20T22:08:34.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.776+0000 7f769dffb640 1 -- 172.21.15.55:0/1558235063 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690007cb0 con 0x7f76a0161140 2024-05-20T22:08:34.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.777+0000 7f76a5302640 1 -- 172.21.15.55:0/1558235063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a0161140 msgr2=0x7f76a0072690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.777+0000 7f76a5302640 1 --2- 172.21.15.55:0/1558235063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a0161140 0x7f76a0072690 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7690009d90 tx=0x7f769002a6a0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.777+0000 7f76a5302640 1 -- 172.21.15.55:0/1558235063 shutdown_connections 2024-05-20T22:08:34.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.777+0000 7f76a5302640 1 --2- 172.21.15.55:0/1558235063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a0161140 0x7f76a0072690 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.777+0000 7f76a5302640 1 -- 172.21.15.55:0/1558235063 >> 172.21.15.55:0/1558235063 conn(0x7f76a00b3980 msgr2=0x7f76a00b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:34.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.777+0000 7f76a5302640 1 -- 172.21.15.55:0/1558235063 shutdown_connections 2024-05-20T22:08:34.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.777+0000 7f76a5302640 1 -- 172.21.15.55:0/1558235063 wait complete. 2024-05-20T22:08:34.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.779+0000 7f76a5302640 1 Processor -- start 2024-05-20T22:08:34.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.779+0000 7f76a5302640 1 -- start start 2024-05-20T22:08:34.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.780+0000 7f76a5302640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a015e550 0x7f76a014c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.780+0000 7f76a5302640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a015e960 0x7f76a01515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.780+0000 7f76a5302640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76a00cf9d0 0x7f76a0151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.780+0000 7f76a5302640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f76a006bac0 con 0x7f76a015e550 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.780+0000 7f76a5302640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76a006b7e0 con 0x7f76a015e960 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.780+0000 7f76a5302640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f76a006c6f0 con 0x7f76a00cf9d0 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.780+0000 7f769effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a015e550 0x7f76a014c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.780+0000 7f769effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a015e550 0x7f76a014c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38554/0 (socket says 172.21.15.55:38554) 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.780+0000 7f769effd640 1 -- 172.21.15.55:0/2743753273 learned_addr learned my addr 172.21.15.55:0/2743753273 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f769f7fe640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76a00cf9d0 0x7f76a0151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f769e7fc640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a015e960 0x7f76a01515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f769effd640 1 -- 172.21.15.55:0/2743753273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76a00cf9d0 msgr2=0x7f76a0151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f769effd640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76a00cf9d0 0x7f76a0151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f769effd640 1 -- 172.21.15.55:0/2743753273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a015e960 msgr2=0x7f76a01515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f769effd640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a015e960 0x7f76a01515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f769effd640 1 -- 172.21.15.55:0/2743753273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a00d0140 con 0x7f76a015e550 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f769effd640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a015e550 0x7f76a014c460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7698009fd0 tx=0x7f769800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:34.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f76977fe640 1 -- 172.21.15.55:0/2743753273 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698015020 con 0x7f76a015e550 2024-05-20T22:08:34.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f76977fe640 1 -- 172.21.15.55:0/2743753273 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7698011830 con 0x7f76a015e550 2024-05-20T22:08:34.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f76a5302640 1 -- 172.21.15.55:0/2743753273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a00d0420 con 0x7f76a015e550 2024-05-20T22:08:34.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f76977fe640 1 -- 172.21.15.55:0/2743753273 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698011c00 con 0x7f76a015e550 2024-05-20T22:08:34.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.781+0000 7f76a5302640 1 -- 172.21.15.55:0/2743753273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a00d09e0 con 0x7f76a015e550 2024-05-20T22:08:34.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.782+0000 7f76a5302640 1 -- 172.21.15.55:0/2743753273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a0069090 con 0x7f76a015e550 2024-05-20T22:08:34.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.783+0000 7f76977fe640 1 -- 172.21.15.55:0/2743753273 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7698018020 con 0x7f76a015e550 2024-05-20T22:08:34.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.783+0000 7f76977fe640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f768006be30 0x7f768006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:34.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.783+0000 7f76977fe640 1 -- 172.21.15.55:0/2743753273 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f769808b200 con 0x7f76a015e550 2024-05-20T22:08:34.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.783+0000 7f769e7fc640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f768006be30 0x7f768006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:34.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.784+0000 7f769e7fc640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f768006be30 0x7f768006e340 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f76a01501e0 tx=0x7f7688009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:34.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.788+0000 7f76977fe640 1 -- 172.21.15.55:0/2743753273 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f769801f020 con 0x7f76a015e550 2024-05-20T22:08:34.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.971+0000 7f76a5302640 1 -- 172.21.15.55:0/2743753273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76a0156150 con 0x7f76a015e550 2024-05-20T22:08:34.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.971+0000 7f76977fe640 1 -- 172.21.15.55:0/2743753273 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f76980172c0 con 0x7f76a015e550 2024-05-20T22:08:34.973 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:34.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.973+0000 7f76a5302640 1 -- 172.21.15.55:0/2743753273 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f768006be30 msgr2=0x7f768006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.973+0000 7f76a5302640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f768006be30 0x7f768006e340 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f76a01501e0 tx=0x7f7688009040 comp rx=0 tx=0).stop 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.973+0000 7f76a5302640 1 -- 172.21.15.55:0/2743753273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a015e550 msgr2=0x7f76a014c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.973+0000 7f76a5302640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a015e550 0x7f76a014c460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7698009fd0 tx=0x7f769800b870 comp rx=0 tx=0).stop 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.974+0000 7f76a5302640 1 -- 172.21.15.55:0/2743753273 shutdown_connections 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.974+0000 7f76a5302640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f768006be30 0x7f768006e340 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.974+0000 7f76a5302640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76a00cf9d0 0x7f76a0151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.974+0000 7f76a5302640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a015e960 0x7f76a01515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.974+0000 7f76a5302640 1 --2- 172.21.15.55:0/2743753273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a015e550 0x7f76a014c460 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.974+0000 7f76a5302640 1 -- 172.21.15.55:0/2743753273 >> 172.21.15.55:0/2743753273 conn(0x7f76a00b3980 msgr2=0x7f76a00b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.974+0000 7f76a5302640 1 -- 172.21.15.55:0/2743753273 shutdown_connections 2024-05-20T22:08:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:34.974+0000 7f76a5302640 1 -- 172.21.15.55:0/2743753273 wait complete. 2024-05-20T22:08:34.986 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:35.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.102+0000 7f05cd3b1640 1 Processor -- start 2024-05-20T22:08:35.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.103+0000 7f05cd3b1640 1 -- start start 2024-05-20T22:08:35.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05cd3b1640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f05c80d32a0 0x7f05c815f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05cd3b1640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f05c80d01b0 0x7f05c806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05cd3b1640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c80d3780 con 0x7f05c80cf690 2024-05-20T22:08:35.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05cd3b1640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f05c80d3a60 con 0x7f05c80d32a0 2024-05-20T22:08:35.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05cd3b1640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f05c80d3d40 con 0x7f05c80d01b0 2024-05-20T22:08:35.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05c6ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f05c80d32a0 0x7f05c815f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05c6ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f05c80d32a0 0x7f05c815f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38570/0 (socket says 172.21.15.55:38570) 2024-05-20T22:08:35.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05c6ffd640 1 -- 172.21.15.55:0/450735667 learned_addr learned my addr 172.21.15.55:0/450735667 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:35.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05c77fe640 1 --2- 172.21.15.55:0/450735667 >> v2:172.21.15.138:3300/0 conn(0x7f05c80d01b0 0x7f05c806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05c6ffd640 1 -- 172.21.15.55:0/450735667 >> v2:172.21.15.138:3300/0 conn(0x7f05c80d01b0 msgr2=0x7f05c806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:35.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05c6ffd640 1 --2- 172.21.15.55:0/450735667 >> v2:172.21.15.138:3300/0 conn(0x7f05c80d01b0 0x7f05c806b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.104+0000 7f05c6ffd640 1 -- 172.21.15.55:0/450735667 >> v1:172.21.15.138:6789/0 conn(0x7f05c80cf690 legacy=0x7f05c80cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.105+0000 7f05c6ffd640 1 -- 172.21.15.55:0/450735667 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c80d4020 con 0x7f05c80d32a0 2024-05-20T22:08:35.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.105+0000 7f05c77fe640 1 --2- 172.21.15.55:0/450735667 >> v2:172.21.15.138:3300/0 conn(0x7f05c80d01b0 0x7f05c806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.105+0000 7f05c6ffd640 1 --2- 172.21.15.55:0/450735667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 0x7f05c815f150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f05c8163d70 tx=0x7f05b802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c474b43481fbb15 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.105+0000 7f05c5ffb640 1 -- 172.21.15.55:0/450735667 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8036070 con 0x7f05c80d32a0 2024-05-20T22:08:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.105+0000 7f05c5ffb640 1 -- 172.21.15.55:0/450735667 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b8031600 con 0x7f05c80d32a0 2024-05-20T22:08:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.105+0000 7f05c5ffb640 1 -- 172.21.15.55:0/450735667 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b80319d0 con 0x7f05c80d32a0 2024-05-20T22:08:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.106+0000 7f05cd3b1640 1 -- 172.21.15.55:0/450735667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 msgr2=0x7f05c815f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.106+0000 7f05cd3b1640 1 --2- 172.21.15.55:0/450735667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 0x7f05c815f150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f05c8163d70 tx=0x7f05b802a730 comp rx=0 tx=0).stop 2024-05-20T22:08:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.106+0000 7f05cd3b1640 1 -- 172.21.15.55:0/450735667 shutdown_connections 2024-05-20T22:08:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.106+0000 7f05cd3b1640 1 --2- 172.21.15.55:0/450735667 >> v2:172.21.15.138:3300/0 conn(0x7f05c80d01b0 0x7f05c806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.106+0000 7f05cd3b1640 1 --2- 172.21.15.55:0/450735667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 0x7f05c815f150 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.106+0000 7f05cd3b1640 1 -- 172.21.15.55:0/450735667 >> 172.21.15.55:0/450735667 conn(0x7f05c80b6de0 msgr2=0x7f05c80d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:35.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.106+0000 7f05cd3b1640 1 -- 172.21.15.55:0/450735667 shutdown_connections 2024-05-20T22:08:35.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.106+0000 7f05cd3b1640 1 -- 172.21.15.55:0/450735667 wait complete. 2024-05-20T22:08:35.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.108+0000 7f05cd3b1640 1 Processor -- start 2024-05-20T22:08:35.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.108+0000 7f05cd3b1640 1 -- start start 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.109+0000 7f05cd3b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80cf690 0x7f05c8154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.109+0000 7f05cd3b1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05c80d01b0 0x7f05c80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.109+0000 7f05cd3b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 0x7f05c815fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.109+0000 7f05cd3b1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f05c80d4820 con 0x7f05c80d32a0 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.109+0000 7f05cd3b1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05c80b4350 con 0x7f05c80cf690 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.109+0000 7f05cd3b1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05c80c93e0 con 0x7f05c80d01b0 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 0x7f05c815fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 0x7f05c815fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38582/0 (socket says 172.21.15.55:38582) 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c77fe640 1 -- 172.21.15.55:0/3385872318 learned_addr learned my addr 172.21.15.55:0/3385872318 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c6ffd640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80cf690 0x7f05c8154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c67fc640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05c80d01b0 0x7f05c80ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c77fe640 1 -- 172.21.15.55:0/3385872318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05c80d01b0 msgr2=0x7f05c80ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c77fe640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05c80d01b0 0x7f05c80ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c77fe640 1 -- 172.21.15.55:0/3385872318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80cf690 msgr2=0x7f05c8154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c77fe640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80cf690 0x7f05c8154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c77fe640 1 -- 172.21.15.55:0/3385872318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c80ce550 con 0x7f05c80d32a0 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c6ffd640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80cf690 0x7f05c8154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05c77fe640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 0x7f05c815fbb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f05c000d610 tx=0x7f05c000dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05af7fe640 1 -- 172.21.15.55:0/3385872318 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0018070 con 0x7f05c80d32a0 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05af7fe640 1 -- 172.21.15.55:0/3385872318 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c0004030 con 0x7f05c80d32a0 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05cd3b1640 1 -- 172.21.15.55:0/3385872318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c80ccf70 con 0x7f05c80d32a0 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05af7fe640 1 -- 172.21.15.55:0/3385872318 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0004400 con 0x7f05c80d32a0 2024-05-20T22:08:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.110+0000 7f05cd3b1640 1 -- 172.21.15.55:0/3385872318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c80cd530 con 0x7f05c80d32a0 2024-05-20T22:08:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.112+0000 7f05cd3b1640 1 -- 172.21.15.55:0/3385872318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c8069090 con 0x7f05c80d32a0 2024-05-20T22:08:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.112+0000 7f05af7fe640 1 -- 172.21.15.55:0/3385872318 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05c0014620 con 0x7f05c80d32a0 2024-05-20T22:08:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.112+0000 7f05af7fe640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b406be60 0x7f05b406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.113+0000 7f05af7fe640 1 -- 172.21.15.55:0/3385872318 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f05c008b470 con 0x7f05c80d32a0 2024-05-20T22:08:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.113+0000 7f05c6ffd640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b406be60 0x7f05b406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.113+0000 7f05c6ffd640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b406be60 0x7f05b406e370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f05b8033000 tx=0x7f05b8009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:35.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.117+0000 7f05af7fe640 1 -- 172.21.15.55:0/3385872318 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f05c0059a30 con 0x7f05c80d32a0 2024-05-20T22:08:35.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.303+0000 7f05cd3b1640 1 -- 172.21.15.55:0/3385872318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f05c80d2e50 con 0x7f05c80d32a0 2024-05-20T22:08:35.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.303+0000 7f05af7fe640 1 -- 172.21.15.55:0/3385872318 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f05c0059330 con 0x7f05c80d32a0 2024-05-20T22:08:35.304 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 -- 172.21.15.55:0/3385872318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b406be60 msgr2=0x7f05b406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b406be60 0x7f05b406e370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f05b8033000 tx=0x7f05b8009000 comp rx=0 tx=0).stop 2024-05-20T22:08:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 -- 172.21.15.55:0/3385872318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 msgr2=0x7f05c815fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 0x7f05c815fbb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f05c000d610 tx=0x7f05c000dad0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 -- 172.21.15.55:0/3385872318 shutdown_connections 2024-05-20T22:08:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f05b406be60 0x7f05b406e370 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05c80d32a0 0x7f05c815fbb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05c80d01b0 0x7f05c80ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 --2- 172.21.15.55:0/3385872318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80cf690 0x7f05c8154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 -- 172.21.15.55:0/3385872318 >> 172.21.15.55:0/3385872318 conn(0x7f05c80b6de0 msgr2=0x7f05c815d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:35.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 -- 172.21.15.55:0/3385872318 shutdown_connections 2024-05-20T22:08:35.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.306+0000 7f05cd3b1640 1 -- 172.21.15.55:0/3385872318 wait complete. 2024-05-20T22:08:35.318 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.435+0000 7fb4e327a640 1 Processor -- start 2024-05-20T22:08:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.435+0000 7fb4e327a640 1 -- start start 2024-05-20T22:08:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e327a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0d32a0 0x7fb4dc15f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e327a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4dc0d01b0 0x7fb4dc06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e327a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4dc0d3780 con 0x7fb4dc0cf690 2024-05-20T22:08:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e327a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb4dc0d3a60 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e327a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4dc0d3d40 con 0x7fb4dc0d32a0 2024-05-20T22:08:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e1817640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4dc0d01b0 0x7fb4dc06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e1016640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0d32a0 0x7fb4dc15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e1817640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4dc0d01b0 0x7fb4dc06b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:41708/0 (socket says 172.21.15.55:41708) 2024-05-20T22:08:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e1817640 1 -- 172.21.15.55:0/194940471 learned_addr learned my addr 172.21.15.55:0/194940471 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4d3fff640 1 -- 172.21.15.55:0/194940471 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1750595961 0 0) 0x7fb4dc0d3780 con 0x7fb4dc0cf690 2024-05-20T22:08:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4d3fff640 1 -- 172.21.15.55:0/194940471 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4c8003520 con 0x7fb4dc0cf690 2024-05-20T22:08:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e1016640 1 -- 172.21.15.55:0/194940471 >> v2:172.21.15.138:3300/0 conn(0x7fb4dc0d01b0 msgr2=0x7fb4dc06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e1016640 1 --2- 172.21.15.55:0/194940471 >> v2:172.21.15.138:3300/0 conn(0x7fb4dc0d01b0 0x7fb4dc06b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e1016640 1 -- 172.21.15.55:0/194940471 >> v1:172.21.15.138:6789/0 conn(0x7fb4dc0cf690 legacy=0x7fb4dc0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e1016640 1 -- 172.21.15.55:0/194940471 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc0d4020 con 0x7fb4dc0d32a0 2024-05-20T22:08:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.437+0000 7fb4e1817640 1 --2- 172.21.15.55:0/194940471 >> v2:172.21.15.138:3300/0 conn(0x7fb4dc0d01b0 0x7fb4dc06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:35.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.438+0000 7fb4e1016640 1 --2- 172.21.15.55:0/194940471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0d32a0 0x7fb4dc15f150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80097a0 tx=0x7fb4d802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38fa34be8151784f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:35.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.438+0000 7fb4d3fff640 1 -- 172.21.15.55:0/194940471 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8037070 con 0x7fb4dc0d32a0 2024-05-20T22:08:35.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.438+0000 7fb4d3fff640 1 -- 172.21.15.55:0/194940471 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d8032440 con 0x7fb4dc0d32a0 2024-05-20T22:08:35.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.438+0000 7fb4d3fff640 1 -- 172.21.15.55:0/194940471 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80327f0 con 0x7fb4dc0d32a0 2024-05-20T22:08:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.439+0000 7fb4e327a640 1 -- 172.21.15.55:0/194940471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0d32a0 msgr2=0x7fb4dc15f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.439+0000 7fb4e327a640 1 --2- 172.21.15.55:0/194940471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0d32a0 0x7fb4dc15f150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80097a0 tx=0x7fb4d802a180 comp rx=0 tx=0).stop 2024-05-20T22:08:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.439+0000 7fb4e327a640 1 -- 172.21.15.55:0/194940471 shutdown_connections 2024-05-20T22:08:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.439+0000 7fb4e327a640 1 --2- 172.21.15.55:0/194940471 >> v2:172.21.15.138:3300/0 conn(0x7fb4dc0d01b0 0x7fb4dc06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.439+0000 7fb4e327a640 1 --2- 172.21.15.55:0/194940471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0d32a0 0x7fb4dc15f150 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.439+0000 7fb4e327a640 1 -- 172.21.15.55:0/194940471 >> 172.21.15.55:0/194940471 conn(0x7fb4dc0b6de0 msgr2=0x7fb4dc0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.439+0000 7fb4e327a640 1 -- 172.21.15.55:0/194940471 shutdown_connections 2024-05-20T22:08:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.439+0000 7fb4e327a640 1 -- 172.21.15.55:0/194940471 wait complete. 2024-05-20T22:08:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.441+0000 7fb4e327a640 1 Processor -- start 2024-05-20T22:08:35.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.441+0000 7fb4e327a640 1 -- start start 2024-05-20T22:08:35.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.442+0000 7fb4e327a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0cf690 0x7fb4dc156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.442+0000 7fb4e327a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4dc0d01b0 0x7fb4dc06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.442+0000 7fb4e327a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4dc0d32a0 0x7fb4dc06c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.442+0000 7fb4e327a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc0d4820 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.442+0000 7fb4e327a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc0b4350 con 0x7fb4dc0d32a0 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.442+0000 7fb4e327a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4dc158260 con 0x7fb4dc0cf690 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.442+0000 7fb4e0815640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4dc0d01b0 0x7fb4dc06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.442+0000 7fb4e0815640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4dc0d01b0 0x7fb4dc06bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38596/0 (socket says 172.21.15.55:38596) 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.442+0000 7fb4e0815640 1 -- 172.21.15.55:0/1387637565 learned_addr learned my addr 172.21.15.55:0/1387637565 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.442+0000 7fb4e1016640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0cf690 0x7fb4dc156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4e1817640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4dc0d32a0 0x7fb4dc06c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4e0815640 1 -- 172.21.15.55:0/1387637565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0cf690 msgr2=0x7fb4dc156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4e0815640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0cf690 0x7fb4dc156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4e0815640 1 -- 172.21.15.55:0/1387637565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4dc0d32a0 msgr2=0x7fb4dc06c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4e0815640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4dc0d32a0 0x7fb4dc06c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4e0815640 1 -- 172.21.15.55:0/1387637565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc0ccb40 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4e1817640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4dc0d32a0 0x7fb4dc06c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4e0815640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4dc0d01b0 0x7fb4dc06bd30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00c430 tx=0x7fb4cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:35.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4d1ffb640 1 -- 172.21.15.55:0/1387637565 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc017070 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4d1ffb640 1 -- 172.21.15.55:0/1387637565 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc0052f0 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4e327a640 1 -- 172.21.15.55:0/1387637565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4dc0cce20 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4d1ffb640 1 -- 172.21.15.55:0/1387637565 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc003830 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.443+0000 7fb4e327a640 1 -- 172.21.15.55:0/1387637565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4dc0cd3e0 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.444+0000 7fb4e327a640 1 -- 172.21.15.55:0/1387637565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4dc0cc8b0 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.445+0000 7fb4d1ffb640 1 -- 172.21.15.55:0/1387637565 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4cc01a070 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.446+0000 7fb4d1ffb640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be60 0x7fb4c006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.446+0000 7fb4d1ffb640 1 -- 172.21.15.55:0/1387637565 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb4cc08a6a0 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.446+0000 7fb4e1016640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be60 0x7fb4c006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.446+0000 7fb4e1016640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be60 0x7fb4c006e370 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8034000 tx=0x7fb4d802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:35.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.450+0000 7fb4d1ffb640 1 -- 172.21.15.55:0/1387637565 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb4cc0584d0 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.631+0000 7fb4e327a640 1 -- 172.21.15.55:0/1387637565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb4dc069090 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.632+0000 7fb4d1ffb640 1 -- 172.21.15.55:0/1387637565 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb4cc05c7d0 con 0x7fb4dc0d01b0 2024-05-20T22:08:35.633 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:35.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 -- 172.21.15.55:0/1387637565 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be60 msgr2=0x7fb4c006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be60 0x7fb4c006e370 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8034000 tx=0x7fb4d802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:35.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 -- 172.21.15.55:0/1387637565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4dc0d01b0 msgr2=0x7fb4dc06bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4dc0d01b0 0x7fb4dc06bd30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00c430 tx=0x7fb4cc00c8f0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 -- 172.21.15.55:0/1387637565 shutdown_connections 2024-05-20T22:08:35.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be60 0x7fb4c006e370 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4dc0d32a0 0x7fb4dc06c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4dc0d01b0 0x7fb4dc06bd30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 --2- 172.21.15.55:0/1387637565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4dc0cf690 0x7fb4dc156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 -- 172.21.15.55:0/1387637565 >> 172.21.15.55:0/1387637565 conn(0x7fb4dc0b6de0 msgr2=0x7fb4dc15e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:35.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 -- 172.21.15.55:0/1387637565 shutdown_connections 2024-05-20T22:08:35.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.634+0000 7fb4e327a640 1 -- 172.21.15.55:0/1387637565 wait complete. 2024-05-20T22:08:35.646 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:35.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.760+0000 7fca07157640 1 Processor -- start 2024-05-20T22:08:35.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.760+0000 7fca07157640 1 -- start start 2024-05-20T22:08:35.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.761+0000 7fca07157640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fca000cd840 0x7fca000b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.761+0000 7fca07157640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000b5830 0x7fca000d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.761+0000 7fca07157640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca0015da90 con 0x7fca000d2700 2024-05-20T22:08:35.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.761+0000 7fca07157640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fca000b6990 con 0x7fca000cd840 2024-05-20T22:08:35.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.761+0000 7fca07157640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca000b5d80 con 0x7fca000b5830 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fc9fffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000b5830 0x7fca000d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fca04ef3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fca000cd840 0x7fca000b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fc9fffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000b5830 0x7fca000d1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:59150/0 (socket says 172.21.15.55:59150) 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fc9fffff640 1 -- 172.21.15.55:0/1203140385 learned_addr learned my addr 172.21.15.55:0/1203140385 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fca04ef3640 1 --2- 172.21.15.55:0/1203140385 >> v2:172.21.15.138:3300/0 conn(0x7fca000cd840 0x7fca000b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:41714/0 (socket says 172.21.15.55:41714) 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fca056f4640 1 --1- 172.21.15.55:0/1203140385 >> v1:172.21.15.138:6789/0 conn(0x7fca000d2700 0x7fca000d2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:60476/0 (socket says 172.21.15.55:60476) 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fc9fffff640 1 -- 172.21.15.55:0/1203140385 >> v2:172.21.15.138:3300/0 conn(0x7fca000cd840 msgr2=0x7fca000b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fc9fffff640 1 --2- 172.21.15.55:0/1203140385 >> v2:172.21.15.138:3300/0 conn(0x7fca000cd840 0x7fca000b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fc9fffff640 1 -- 172.21.15.55:0/1203140385 >> v1:172.21.15.138:6789/0 conn(0x7fca000d2700 legacy=0x7fca000d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fc9fffff640 1 -- 172.21.15.55:0/1203140385 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca000b6060 con 0x7fca000b5830 2024-05-20T22:08:35.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.762+0000 7fca04ef3640 1 --2- 172.21.15.55:0/1203140385 >> v2:172.21.15.138:3300/0 conn(0x7fca000cd840 0x7fca000b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:35.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.763+0000 7fc9fffff640 1 --2- 172.21.15.55:0/1203140385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000b5830 0x7fca000d1890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009e50 tx=0x7fc9f002a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a46cebbd459a90 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:35.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.763+0000 7fc9ff7fe640 1 -- 172.21.15.55:0/1203140385 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0036070 con 0x7fca000b5830 2024-05-20T22:08:35.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.763+0000 7fc9ff7fe640 1 -- 172.21.15.55:0/1203140385 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f0031600 con 0x7fca000b5830 2024-05-20T22:08:35.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.763+0000 7fc9ff7fe640 1 -- 172.21.15.55:0/1203140385 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00319d0 con 0x7fca000b5830 2024-05-20T22:08:35.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.764+0000 7fca07157640 1 -- 172.21.15.55:0/1203140385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000b5830 msgr2=0x7fca000d1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.764+0000 7fca07157640 1 --2- 172.21.15.55:0/1203140385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000b5830 0x7fca000d1890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009e50 tx=0x7fc9f002a6a0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.764+0000 7fca07157640 1 -- 172.21.15.55:0/1203140385 shutdown_connections 2024-05-20T22:08:35.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.764+0000 7fca07157640 1 --2- 172.21.15.55:0/1203140385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000b5830 0x7fca000d1890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.764+0000 7fca07157640 1 --2- 172.21.15.55:0/1203140385 >> v2:172.21.15.138:3300/0 conn(0x7fca000cd840 0x7fca000b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.764+0000 7fca07157640 1 -- 172.21.15.55:0/1203140385 >> 172.21.15.55:0/1203140385 conn(0x7fca0008bff0 msgr2=0x7fca0008c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:35.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.764+0000 7fca07157640 1 -- 172.21.15.55:0/1203140385 shutdown_connections 2024-05-20T22:08:35.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.764+0000 7fca07157640 1 -- 172.21.15.55:0/1203140385 wait complete. 2024-05-20T22:08:35.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.766+0000 7fca07157640 1 Processor -- start 2024-05-20T22:08:35.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.766+0000 7fca07157640 1 -- start start 2024-05-20T22:08:35.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.767+0000 7fca07157640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca000b5830 0x7fca000c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.767+0000 7fca07157640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000cd840 0x7fca001529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.767+0000 7fca07157640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca000d2700 0x7fca000c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.767+0000 7fca07157640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fca000b8160 con 0x7fca000b5830 2024-05-20T22:08:35.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.767+0000 7fca07157640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca000b6620 con 0x7fca000d2700 2024-05-20T22:08:35.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.767+0000 7fca07157640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca000b6e20 con 0x7fca000cd840 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.767+0000 7fca04ef3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca000b5830 0x7fca000c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca04ef3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca000b5830 0x7fca000c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38612/0 (socket says 172.21.15.55:38612) 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca04ef3640 1 -- 172.21.15.55:0/150171838 learned_addr learned my addr 172.21.15.55:0/150171838 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fc9fffff640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000cd840 0x7fca001529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca056f4640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca000d2700 0x7fca000c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca04ef3640 1 -- 172.21.15.55:0/150171838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000cd840 msgr2=0x7fca001529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca04ef3640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000cd840 0x7fca001529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca04ef3640 1 -- 172.21.15.55:0/150171838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca000d2700 msgr2=0x7fca000c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca04ef3640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca000d2700 0x7fca000c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca04ef3640 1 -- 172.21.15.55:0/150171838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca001f72e0 con 0x7fca000b5830 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca04ef3640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca000b5830 0x7fca000c6ec0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800e900 tx=0x7fc9e800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fc9fd7fa640 1 -- 172.21.15.55:0/150171838 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800ca00 con 0x7fca000b5830 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fc9fd7fa640 1 -- 172.21.15.55:0/150171838 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e800cba0 con 0x7fca000b5830 2024-05-20T22:08:35.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca07157640 1 -- 172.21.15.55:0/150171838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca001f75c0 con 0x7fca000b5830 2024-05-20T22:08:35.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fc9fd7fa640 1 -- 172.21.15.55:0/150171838 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8005990 con 0x7fca000b5830 2024-05-20T22:08:35.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.768+0000 7fca07157640 1 -- 172.21.15.55:0/150171838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca00152ee0 con 0x7fca000b5830 2024-05-20T22:08:35.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.769+0000 7fca07157640 1 -- 172.21.15.55:0/150171838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca00069090 con 0x7fca000b5830 2024-05-20T22:08:35.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.770+0000 7fc9fd7fa640 1 -- 172.21.15.55:0/150171838 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9e8005b30 con 0x7fca000b5830 2024-05-20T22:08:35.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.771+0000 7fc9fd7fa640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc9e006be30 0x7fc9e006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:35.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.771+0000 7fc9fffff640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc9e006be30 0x7fc9e006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:35.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.771+0000 7fc9fd7fa640 1 -- 172.21.15.55:0/150171838 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc9e808b650 con 0x7fca000b5830 2024-05-20T22:08:35.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.771+0000 7fc9fffff640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc9e006be30 0x7fc9e006e340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0033000 tx=0x7fc9f0009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:35.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.775+0000 7fc9fd7fa640 1 -- 172.21.15.55:0/150171838 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc9e8017020 con 0x7fca000b5830 2024-05-20T22:08:35.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.964+0000 7fca07157640 1 -- 172.21.15.55:0/150171838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca000d1750 con 0x7fca000b5830 2024-05-20T22:08:35.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.965+0000 7fc9fd7fa640 1 -- 172.21.15.55:0/150171838 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7fc9e8059510 con 0x7fca000b5830 2024-05-20T22:08:35.966 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:35.966 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:35.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 -- 172.21.15.55:0/150171838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc9e006be30 msgr2=0x7fc9e006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc9e006be30 0x7fc9e006e340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0033000 tx=0x7fc9f0009000 comp rx=0 tx=0).stop 2024-05-20T22:08:35.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 -- 172.21.15.55:0/150171838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca000b5830 msgr2=0x7fca000c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:35.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca000b5830 0x7fca000c6ec0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800e900 tx=0x7fc9e800edc0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 -- 172.21.15.55:0/150171838 shutdown_connections 2024-05-20T22:08:35.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc9e006be30 0x7fc9e006e340 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca000d2700 0x7fca000c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca000cd840 0x7fca001529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 --2- 172.21.15.55:0/150171838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca000b5830 0x7fca000c6ec0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:35.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 -- 172.21.15.55:0/150171838 >> 172.21.15.55:0/150171838 conn(0x7fca0008bff0 msgr2=0x7fca000d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:35.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 -- 172.21.15.55:0/150171838 shutdown_connections 2024-05-20T22:08:35.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:35.967+0000 7fca07157640 1 -- 172.21.15.55:0/150171838 wait complete. 2024-05-20T22:08:35.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:35.980 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:36.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.097+0000 7f374c565640 1 Processor -- start 2024-05-20T22:08:36.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.097+0000 7f374c565640 1 -- start start 2024-05-20T22:08:36.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f374c565640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f37440cf680 0x7f374415f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f374c565640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f37440b7fb0 0x7f37440b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f374c565640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f374406c460 0x7f374406c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f374c565640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f374406b6a0 con 0x7f37440b7fb0 2024-05-20T22:08:36.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f374c565640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f374406b980 con 0x7f37440cf680 2024-05-20T22:08:36.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f374c565640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f374406bc60 con 0x7f374406c460 2024-05-20T22:08:36.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f3749b00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f37440b7fb0 0x7f37440b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f3749b00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f37440b7fb0 0x7f37440b83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38622/0 (socket says 172.21.15.55:38622) 2024-05-20T22:08:36.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f3749b00640 1 -- 172.21.15.55:0/391944034 learned_addr learned my addr 172.21.15.55:0/391944034 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:36.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f374a301640 1 --2- 172.21.15.55:0/391944034 >> v2:172.21.15.138:3300/0 conn(0x7f37440cf680 0x7f374415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f374ab02640 1 --2- 172.21.15.55:0/391944034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f374406c460 0x7f374406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f3749b00640 1 -- 172.21.15.55:0/391944034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f374406c460 msgr2=0x7f374406c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:36.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f3749b00640 1 --2- 172.21.15.55:0/391944034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f374406c460 0x7f374406c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f3749b00640 1 -- 172.21.15.55:0/391944034 >> v2:172.21.15.138:3300/0 conn(0x7f37440cf680 msgr2=0x7f374415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:36.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f3749b00640 1 --2- 172.21.15.55:0/391944034 >> v2:172.21.15.138:3300/0 conn(0x7f37440cf680 0x7f374415f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.099+0000 7f3749b00640 1 -- 172.21.15.55:0/391944034 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374406bf40 con 0x7f37440b7fb0 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.100+0000 7f3749b00640 1 --2- 172.21.15.55:0/391944034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 0x7f37440b83c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f37440b6f50 tx=0x7f373402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d363c327295e6b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.100+0000 7f37492ff640 1 -- 172.21.15.55:0/391944034 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734037070 con 0x7f37440b7fb0 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.100+0000 7f37492ff640 1 -- 172.21.15.55:0/391944034 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3734032440 con 0x7f37440b7fb0 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.100+0000 7f37492ff640 1 -- 172.21.15.55:0/391944034 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734032810 con 0x7f37440b7fb0 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.101+0000 7f374c565640 1 -- 172.21.15.55:0/391944034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 msgr2=0x7f37440b83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.101+0000 7f374c565640 1 --2- 172.21.15.55:0/391944034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 0x7f37440b83c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f37440b6f50 tx=0x7f373402a140 comp rx=0 tx=0).stop 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.101+0000 7f374c565640 1 -- 172.21.15.55:0/391944034 shutdown_connections 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.101+0000 7f374c565640 1 --2- 172.21.15.55:0/391944034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f374406c460 0x7f374406c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.101+0000 7f374c565640 1 --2- 172.21.15.55:0/391944034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 0x7f37440b83c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.101+0000 7f374c565640 1 --2- 172.21.15.55:0/391944034 >> v2:172.21.15.138:3300/0 conn(0x7f37440cf680 0x7f374415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.101+0000 7f374c565640 1 -- 172.21.15.55:0/391944034 >> 172.21.15.55:0/391944034 conn(0x7f37440d5450 msgr2=0x7f37440b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.101+0000 7f374c565640 1 -- 172.21.15.55:0/391944034 shutdown_connections 2024-05-20T22:08:36.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.101+0000 7f374c565640 1 -- 172.21.15.55:0/391944034 wait complete. 2024-05-20T22:08:36.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.103+0000 7f374c565640 1 Processor -- start 2024-05-20T22:08:36.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.103+0000 7f374c565640 1 -- start start 2024-05-20T22:08:36.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f374c565640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f374406c460 0x7f37441f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f374c565640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 0x7f374415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f374c565640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37440cf680 0x7f3744161290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f374c565640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f37440b42a0 con 0x7f37440b7fb0 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f374c565640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37440b4120 con 0x7f37440cf680 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f374c565640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3744162b50 con 0x7f374406c460 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f3749b00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 0x7f374415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f3749b00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 0x7f374415f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38634/0 (socket says 172.21.15.55:38634) 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f3749b00640 1 -- 172.21.15.55:0/2727565948 learned_addr learned my addr 172.21.15.55:0/2727565948 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f3749b00640 1 -- 172.21.15.55:0/2727565948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f374406c460 msgr2=0x7f37441f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f3749b00640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f374406c460 0x7f37441f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f3749b00640 1 -- 172.21.15.55:0/2727565948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37440cf680 msgr2=0x7f3744161290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f3749b00640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37440cf680 0x7f3744161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.104+0000 7f3749b00640 1 -- 172.21.15.55:0/2727565948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37441f89b0 con 0x7f37440b7fb0 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.105+0000 7f3749b00640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 0x7f374415f080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f373402a180 tx=0x7f37340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.105+0000 7f373affd640 1 -- 172.21.15.55:0/2727565948 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734037040 con 0x7f37440b7fb0 2024-05-20T22:08:36.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.105+0000 7f373affd640 1 -- 172.21.15.55:0/2727565948 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3734042070 con 0x7f37440b7fb0 2024-05-20T22:08:36.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.105+0000 7f374c565640 1 -- 172.21.15.55:0/2727565948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374415fd70 con 0x7f37440b7fb0 2024-05-20T22:08:36.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.105+0000 7f373affd640 1 -- 172.21.15.55:0/2727565948 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373403d880 con 0x7f37440b7fb0 2024-05-20T22:08:36.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.105+0000 7f374c565640 1 -- 172.21.15.55:0/2727565948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3744160330 con 0x7f37440b7fb0 2024-05-20T22:08:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.106+0000 7f374c565640 1 -- 172.21.15.55:0/2727565948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374415fae0 con 0x7f37440b7fb0 2024-05-20T22:08:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.106+0000 7f373affd640 1 -- 172.21.15.55:0/2727565948 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f373403da20 con 0x7f37440b7fb0 2024-05-20T22:08:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.107+0000 7f373affd640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f372806be60 0x7f372806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.107+0000 7f373affd640 1 -- 172.21.15.55:0/2727565948 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f37340abb80 con 0x7f37440b7fb0 2024-05-20T22:08:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.107+0000 7f374a301640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f372806be60 0x7f372806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.107+0000 7f374a301640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f372806be60 0x7f372806e370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3740006a90 tx=0x7f3740006a20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.111+0000 7f373affd640 1 -- 172.21.15.55:0/2727565948 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f373407a000 con 0x7f37440b7fb0 2024-05-20T22:08:36.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.294+0000 7f374c565640 1 -- 172.21.15.55:0/2727565948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3744069090 con 0x7f37440b7fb0 2024-05-20T22:08:36.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.294+0000 7f373affd640 1 -- 172.21.15.55:0/2727565948 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f3734079900 con 0x7f37440b7fb0 2024-05-20T22:08:36.295 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 -- 172.21.15.55:0/2727565948 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f372806be60 msgr2=0x7f372806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f372806be60 0x7f372806e370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3740006a90 tx=0x7f3740006a20 comp rx=0 tx=0).stop 2024-05-20T22:08:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 -- 172.21.15.55:0/2727565948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 msgr2=0x7f374415f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 0x7f374415f080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f373402a180 tx=0x7f37340046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 -- 172.21.15.55:0/2727565948 shutdown_connections 2024-05-20T22:08:36.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f372806be60 0x7f372806e370 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37440cf680 0x7f3744161290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37440b7fb0 0x7f374415f080 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 --2- 172.21.15.55:0/2727565948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f374406c460 0x7f37441f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 -- 172.21.15.55:0/2727565948 >> 172.21.15.55:0/2727565948 conn(0x7f37440d5450 msgr2=0x7f37440d35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:36.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 -- 172.21.15.55:0/2727565948 shutdown_connections 2024-05-20T22:08:36.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.296+0000 7f374c565640 1 -- 172.21.15.55:0/2727565948 wait complete. 2024-05-20T22:08:36.308 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:36.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.425+0000 7fc3c39c3640 1 Processor -- start 2024-05-20T22:08:36.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.425+0000 7fc3c39c3640 1 -- start start 2024-05-20T22:08:36.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c39c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc0cfad0 0x7fc3bc0b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c39c3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3bc06d0b0 0x7fc3bc0d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c39c3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3bc0b37b0 con 0x7fc3bc0b2470 2024-05-20T22:08:36.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c39c3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc3bc0b2880 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c39c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3bc06d8b0 con 0x7fc3bc0cfad0 2024-05-20T22:08:36.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c0f5e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc0cfad0 0x7fc3bc0b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c1f60640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3bc06d0b0 0x7fc3bc0d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c175f640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc3bc0b2470 0x7fc3bc0cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:60478/0 (socket says 172.21.15.55:60478) 2024-05-20T22:08:36.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c175f640 1 -- 172.21.15.55:0/1828301164 learned_addr learned my addr 172.21.15.55:0/1828301164 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:36.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c1f60640 1 -- 172.21.15.55:0/1828301164 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc0cfad0 msgr2=0x7fc3bc0b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:36.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c1f60640 1 --2- 172.21.15.55:0/1828301164 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc0cfad0 0x7fc3bc0b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c1f60640 1 -- 172.21.15.55:0/1828301164 >> v1:172.21.15.138:6789/0 conn(0x7fc3bc0b2470 legacy=0x7fc3bc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c1f60640 1 -- 172.21.15.55:0/1828301164 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3bc06db90 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3b3fff640 1 -- 172.21.15.55:0/1828301164 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330450026 0 0) 0x7fc3b8001110 con 0x7fc3bc0b2470 2024-05-20T22:08:36.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.427+0000 7fc3c0f5e640 1 --2- 172.21.15.55:0/1828301164 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc0cfad0 0x7fc3bc0b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.428+0000 7fc3c1f60640 1 --2- 172.21.15.55:0/1828301164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3bc06d0b0 0x7fc3bc0d0b60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4009d90 tx=0x7fc3b402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0daa689626ff761 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.428+0000 7fc3b3fff640 1 -- 172.21.15.55:0/1828301164 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4007740 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.428+0000 7fc3b3fff640 1 -- 172.21.15.55:0/1828301164 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b40078e0 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.428+0000 7fc3b3fff640 1 -- 172.21.15.55:0/1828301164 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4007cb0 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.429+0000 7fc3c39c3640 1 -- 172.21.15.55:0/1828301164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3bc06d0b0 msgr2=0x7fc3bc0d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.429+0000 7fc3c39c3640 1 --2- 172.21.15.55:0/1828301164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3bc06d0b0 0x7fc3bc0d0b60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4009d90 tx=0x7fc3b402a660 comp rx=0 tx=0).stop 2024-05-20T22:08:36.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.429+0000 7fc3c39c3640 1 -- 172.21.15.55:0/1828301164 shutdown_connections 2024-05-20T22:08:36.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.429+0000 7fc3c39c3640 1 --2- 172.21.15.55:0/1828301164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3bc06d0b0 0x7fc3bc0d0b60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.429+0000 7fc3c39c3640 1 --2- 172.21.15.55:0/1828301164 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc0cfad0 0x7fc3bc0b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.429+0000 7fc3c39c3640 1 -- 172.21.15.55:0/1828301164 >> 172.21.15.55:0/1828301164 conn(0x7fc3bc09be10 msgr2=0x7fc3bc0d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:36.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.429+0000 7fc3c39c3640 1 -- 172.21.15.55:0/1828301164 shutdown_connections 2024-05-20T22:08:36.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.429+0000 7fc3c39c3640 1 -- 172.21.15.55:0/1828301164 wait complete. 2024-05-20T22:08:36.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.431+0000 7fc3c39c3640 1 Processor -- start 2024-05-20T22:08:36.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.431+0000 7fc3c39c3640 1 -- start start 2024-05-20T22:08:36.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.432+0000 7fc3c39c3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3bc06d0b0 0x7fc3bc15e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.432+0000 7fc3c39c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3bc0b2470 0x7fc3bc09e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.432+0000 7fc3c39c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc09e750 0x7fc3bc09eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.432+0000 7fc3c39c3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc06d4c0 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.432+0000 7fc3c39c3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc0b3a30 con 0x7fc3bc0b2470 2024-05-20T22:08:36.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.432+0000 7fc3c39c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3bc0b3f50 con 0x7fc3bc09e750 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.432+0000 7fc3c175f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3bc06d0b0 0x7fc3bc15e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.432+0000 7fc3c175f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3bc06d0b0 0x7fc3bc15e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38650/0 (socket says 172.21.15.55:38650) 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.432+0000 7fc3c175f640 1 -- 172.21.15.55:0/3254081487 learned_addr learned my addr 172.21.15.55:0/3254081487 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3c0f5e640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3bc0b2470 0x7fc3bc09e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3c1f60640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc09e750 0x7fc3bc09eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3c175f640 1 -- 172.21.15.55:0/3254081487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc09e750 msgr2=0x7fc3bc09eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3c175f640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc09e750 0x7fc3bc09eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3c175f640 1 -- 172.21.15.55:0/3254081487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3bc0b2470 msgr2=0x7fc3bc09e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3c175f640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3bc0b2470 0x7fc3bc09e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3c175f640 1 -- 172.21.15.55:0/3254081487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3bc09f0a0 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3c175f640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3bc06d0b0 0x7fc3bc15e220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800c410 tx=0x7fc3b800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3b1ffb640 1 -- 172.21.15.55:0/3254081487 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b80074e0 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3b1ffb640 1 -- 172.21.15.55:0/3254081487 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b80047e0 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3b1ffb640 1 -- 172.21.15.55:0/3254081487 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8004bd0 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3c39c3640 1 -- 172.21.15.55:0/3254081487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3bc15e760 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.433+0000 7fc3c39c3640 1 -- 172.21.15.55:0/3254081487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3bc0ce650 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.434+0000 7fc3c39c3640 1 -- 172.21.15.55:0/3254081487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3bc069090 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.435+0000 7fc3b1ffb640 1 -- 172.21.15.55:0/3254081487 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3b8011070 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.435+0000 7fc3b1ffb640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc3a006be30 0x7fc3a006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.435+0000 7fc3b1ffb640 1 -- 172.21.15.55:0/3254081487 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc3b808afc0 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.436+0000 7fc3c0f5e640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc3a006be30 0x7fc3a006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.436+0000 7fc3c0f5e640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc3a006be30 0x7fc3a006e340 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc1e8cc0 tx=0x7fc3ac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:36.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.440+0000 7fc3b1ffb640 1 -- 172.21.15.55:0/3254081487 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc3bc0b37b0 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.622+0000 7fc3c39c3640 1 -- 172.21.15.55:0/3254081487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc3bc0d35a0 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.622+0000 7fc3b1ffb640 1 -- 172.21.15.55:0/3254081487 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fc3b800e280 con 0x7fc3bc06d0b0 2024-05-20T22:08:36.624 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.624+0000 7fc3c39c3640 1 -- 172.21.15.55:0/3254081487 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc3a006be30 msgr2=0x7fc3a006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.624+0000 7fc3c39c3640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc3a006be30 0x7fc3a006e340 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc1e8cc0 tx=0x7fc3ac008040 comp rx=0 tx=0).stop 2024-05-20T22:08:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.625+0000 7fc3c39c3640 1 -- 172.21.15.55:0/3254081487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3bc06d0b0 msgr2=0x7fc3bc15e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.625+0000 7fc3c39c3640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3bc06d0b0 0x7fc3bc15e220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800c410 tx=0x7fc3b800c8d0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.625+0000 7fc3c39c3640 1 -- 172.21.15.55:0/3254081487 shutdown_connections 2024-05-20T22:08:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.625+0000 7fc3c39c3640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc3a006be30 0x7fc3a006e340 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.625+0000 7fc3c39c3640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3bc09e750 0x7fc3bc09eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.625+0000 7fc3c39c3640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3bc0b2470 0x7fc3bc09e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.625+0000 7fc3c39c3640 1 --2- 172.21.15.55:0/3254081487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3bc06d0b0 0x7fc3bc15e220 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.625+0000 7fc3c39c3640 1 -- 172.21.15.55:0/3254081487 >> 172.21.15.55:0/3254081487 conn(0x7fc3bc09be10 msgr2=0x7fc3bc0d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:36.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.625+0000 7fc3c39c3640 1 -- 172.21.15.55:0/3254081487 shutdown_connections 2024-05-20T22:08:36.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.625+0000 7fc3c39c3640 1 -- 172.21.15.55:0/3254081487 wait complete. 2024-05-20T22:08:36.637 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:36.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.754+0000 7f8c6db37640 1 Processor -- start 2024-05-20T22:08:36.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.754+0000 7f8c6db37640 1 -- start start 2024-05-20T22:08:36.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c6db37640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6815f150 0x7f8c680cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c6db37640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8c6806b500 0x7f8c6806b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c6db37640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c680d3780 con 0x7f8c680d32a0 2024-05-20T22:08:36.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c6db37640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8c680d3a60 con 0x7f8c6806b500 2024-05-20T22:08:36.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c6db37640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8c680d3d40 con 0x7f8c6815f150 2024-05-20T22:08:36.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c677fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8c680d32a0 0x7f8c680cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56842/0 (socket says 172.21.15.55:56842) 2024-05-20T22:08:36.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c677fe640 1 -- 172.21.15.55:0/1004521825 learned_addr learned my addr 172.21.15.55:0/1004521825 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:36.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c67fff640 1 --2- 172.21.15.55:0/1004521825 >> v2:172.21.15.138:3300/0 conn(0x7f8c6806b500 0x7f8c6806b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c66ffd640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6815f150 0x7f8c680cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2928074431 0 0) 0x7f8c680d3780 con 0x7f8c680d32a0 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.755+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c50003520 con 0x7f8c680d32a0 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1809135525 0 0) 0x7f8c50003520 con 0x7f8c680d32a0 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6815f150 msgr2=0x7f8c680cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6815f150 0x7f8c680cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 >> v2:172.21.15.138:3300/0 conn(0x7f8c6806b500 msgr2=0x7f8c6806b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 --2- 172.21.15.55:0/1004521825 >> v2:172.21.15.138:3300/0 conn(0x7f8c6806b500 0x7f8c6806b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c680d4020 con 0x7f8c680d32a0 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f8c58001670 con 0x7f8c680d32a0 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c66ffd640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6815f150 0x7f8c680cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 >> v1:172.21.15.55:6789/0 conn(0x7f8c680d32a0 legacy=0x7f8c680cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6815f150 0x7f8c500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c50003f10 0x7f8c50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c50004d40 0x7f8c50005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8c680d3a60 con 0x7f8c50004d40 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c680d3d40 con 0x7f8c50003f10 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5000b9f0 con 0x7f8c6815f150 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c67fff640 1 --2- 172.21.15.55:0/1004521825 >> v2:172.21.15.138:3300/0 conn(0x7f8c6806b500 0x7f8c6806b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c67fff640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c50004d40 0x7f8c50005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c66ffd640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c50003f10 0x7f8c50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c677fe640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6815f150 0x7f8c500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c67fff640 1 -- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6815f150 msgr2=0x7f8c500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c67fff640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c6815f150 0x7f8c500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c67fff640 1 -- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c50003f10 msgr2=0x7f8c50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c67fff640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c50003f10 0x7f8c50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.756+0000 7f8c67fff640 1 -- 172.21.15.55:0/1004521825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c50003520 con 0x7f8c50004d40 2024-05-20T22:08:36.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.757+0000 7f8c67fff640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c50004d40 0x7f8c50005150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8c680b7830 tx=0x7f8c50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f982c41a0a3038d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:36.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.757+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60033070 con 0x7f8c50004d40 2024-05-20T22:08:36.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.757+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6002f650 con 0x7f8c50004d40 2024-05-20T22:08:36.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.757+0000 7f8c667fc640 1 -- 172.21.15.55:0/1004521825 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60009de0 con 0x7f8c50004d40 2024-05-20T22:08:36.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.758+0000 7f8c6db37640 1 -- 172.21.15.55:0/1004521825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c50004d40 msgr2=0x7f8c50005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.758+0000 7f8c6db37640 1 --2- 172.21.15.55:0/1004521825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c50004d40 0x7f8c50005150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8c680b7830 tx=0x7f8c50005830 comp rx=0 tx=0).stop 2024-05-20T22:08:36.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.758+0000 7f8c67fff640 1 -- 172.21.15.55:0/1004521825 reap_dead start 2024-05-20T22:08:36.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.758+0000 7f8c6db37640 1 -- 172.21.15.55:0/1004521825 shutdown_connections 2024-05-20T22:08:36.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.758+0000 7f8c6db37640 1 -- 172.21.15.55:0/1004521825 >> 172.21.15.55:0/1004521825 conn(0x7f8c680b6de0 msgr2=0x7f8c680d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:36.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.758+0000 7f8c6db37640 1 -- 172.21.15.55:0/1004521825 shutdown_connections 2024-05-20T22:08:36.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.758+0000 7f8c6db37640 1 -- 172.21.15.55:0/1004521825 wait complete. 2024-05-20T22:08:36.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.760+0000 7f8c6db37640 1 Processor -- start 2024-05-20T22:08:36.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.760+0000 7f8c6db37640 1 -- start start 2024-05-20T22:08:36.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.761+0000 7f8c6db37640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c50004d40 0x7f8c681559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.761+0000 7f8c6db37640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c681f5130 0x7f8c680c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.761+0000 7f8c6db37640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c680c9eb0 0x7f8c680c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.761+0000 7f8c6db37640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8c680d4820 con 0x7f8c680c9eb0 2024-05-20T22:08:36.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.761+0000 7f8c6db37640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6815f560 con 0x7f8c50004d40 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.761+0000 7f8c6db37640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8c680ca5b0 con 0x7f8c681f5130 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.761+0000 7f8c67fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c680c9eb0 0x7f8c680c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.761+0000 7f8c67fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c680c9eb0 0x7f8c680c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38670/0 (socket says 172.21.15.55:38670) 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.761+0000 7f8c67fff640 1 -- 172.21.15.55:0/2065115452 learned_addr learned my addr 172.21.15.55:0/2065115452 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c66ffd640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c681f5130 0x7f8c680c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c67fff640 1 -- 172.21.15.55:0/2065115452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c681f5130 msgr2=0x7f8c680c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c67fff640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c681f5130 0x7f8c680c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c67fff640 1 -- 172.21.15.55:0/2065115452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c50004d40 msgr2=0x7f8c681559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c677fe640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c50004d40 0x7f8c681559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c67fff640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c50004d40 0x7f8c681559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c67fff640 1 -- 172.21.15.55:0/2065115452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c68154bf0 con 0x7f8c680c9eb0 2024-05-20T22:08:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c67fff640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c680c9eb0 0x7f8c680c89c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000e3e0 tx=0x7f8c60009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c4ffff640 1 -- 172.21.15.55:0/2065115452 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60033040 con 0x7f8c680c9eb0 2024-05-20T22:08:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c4ffff640 1 -- 172.21.15.55:0/2065115452 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6002f450 con 0x7f8c680c9eb0 2024-05-20T22:08:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c6db37640 1 -- 172.21.15.55:0/2065115452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c68154ed0 con 0x7f8c680c9eb0 2024-05-20T22:08:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c4ffff640 1 -- 172.21.15.55:0/2065115452 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60009de0 con 0x7f8c680c9eb0 2024-05-20T22:08:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.762+0000 7f8c6db37640 1 -- 172.21.15.55:0/2065115452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c680ca2f0 con 0x7f8c680c9eb0 2024-05-20T22:08:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.763+0000 7f8c6db37640 1 -- 172.21.15.55:0/2065115452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c680c94e0 con 0x7f8c680c9eb0 2024-05-20T22:08:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.764+0000 7f8c4ffff640 1 -- 172.21.15.55:0/2065115452 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c60036070 con 0x7f8c680c9eb0 2024-05-20T22:08:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.764+0000 7f8c4ffff640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8c5406be30 0x7f8c5406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.764+0000 7f8c4ffff640 1 -- 172.21.15.55:0/2065115452 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8c600abce0 con 0x7f8c680c9eb0 2024-05-20T22:08:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.764+0000 7f8c677fe640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8c5406be30 0x7f8c5406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:36.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.765+0000 7f8c677fe640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8c5406be30 0x7f8c5406e340 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8c58007fd0 tx=0x7f8c58009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:36.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.769+0000 7f8c4ffff640 1 -- 172.21.15.55:0/2065115452 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8c60047030 con 0x7f8c680c9eb0 2024-05-20T22:08:36.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.954+0000 7f8c6db37640 1 -- 172.21.15.55:0/2065115452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8c68069090 con 0x7f8c680c9eb0 2024-05-20T22:08:36.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.954+0000 7f8c4ffff640 1 -- 172.21.15.55:0/2065115452 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f8c6007a320 con 0x7f8c680c9eb0 2024-05-20T22:08:36.955 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.956+0000 7f8c6db37640 1 -- 172.21.15.55:0/2065115452 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8c5406be30 msgr2=0x7f8c5406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.956+0000 7f8c6db37640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8c5406be30 0x7f8c5406e340 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8c58007fd0 tx=0x7f8c58009040 comp rx=0 tx=0).stop 2024-05-20T22:08:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.956+0000 7f8c6db37640 1 -- 172.21.15.55:0/2065115452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c680c9eb0 msgr2=0x7f8c680c89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.956+0000 7f8c6db37640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c680c9eb0 0x7f8c680c89c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000e3e0 tx=0x7f8c60009750 comp rx=0 tx=0).stop 2024-05-20T22:08:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.956+0000 7f8c6db37640 1 -- 172.21.15.55:0/2065115452 shutdown_connections 2024-05-20T22:08:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.956+0000 7f8c6db37640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8c5406be30 0x7f8c5406e340 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.956+0000 7f8c6db37640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c680c9eb0 0x7f8c680c89c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.956+0000 7f8c6db37640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c681f5130 0x7f8c680c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.956+0000 7f8c6db37640 1 --2- 172.21.15.55:0/2065115452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c50004d40 0x7f8c681559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.956+0000 7f8c6db37640 1 -- 172.21.15.55:0/2065115452 >> 172.21.15.55:0/2065115452 conn(0x7f8c680b6de0 msgr2=0x7f8c6806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.957+0000 7f8c6db37640 1 -- 172.21.15.55:0/2065115452 shutdown_connections 2024-05-20T22:08:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:36.957+0000 7f8c6db37640 1 -- 172.21.15.55:0/2065115452 wait complete. 2024-05-20T22:08:36.969 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:37.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.085+0000 7f16b5eb8640 1 Processor -- start 2024-05-20T22:08:37.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.085+0000 7f16b5eb8640 1 -- start start 2024-05-20T22:08:37.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16b5eb8640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f16b00b4080 0x7f16b015f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16b5eb8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00d3290 0x7f16b00d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16b5eb8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16b006b500 con 0x7f16b00b35e0 2024-05-20T22:08:37.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16b5eb8640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f16b0163960 con 0x7f16b00b4080 2024-05-20T22:08:37.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16b5eb8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f16b00b3e10 con 0x7f16b00d3290 2024-05-20T22:08:37.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16af7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f16b00b4080 0x7f16b015f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16af7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f16b00b4080 0x7f16b015f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38674/0 (socket says 172.21.15.55:38674) 2024-05-20T22:08:37.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16af7fe640 1 -- 172.21.15.55:0/2785452324 learned_addr learned my addr 172.21.15.55:0/2785452324 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:37.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16affff640 1 --2- 172.21.15.55:0/2785452324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00d3290 0x7f16b00d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16af7fe640 1 -- 172.21.15.55:0/2785452324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00d3290 msgr2=0x7f16b00d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:37.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16af7fe640 1 --2- 172.21.15.55:0/2785452324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00d3290 0x7f16b00d36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16af7fe640 1 -- 172.21.15.55:0/2785452324 >> v1:172.21.15.138:6789/0 conn(0x7f16b00b35e0 legacy=0x7f16b015f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:37.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16af7fe640 1 -- 172.21.15.55:0/2785452324 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b00b3b30 con 0x7f16b00b4080 2024-05-20T22:08:37.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.087+0000 7f16affff640 1 --2- 172.21.15.55:0/2785452324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00d3290 0x7f16b00d36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:37.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.088+0000 7f16af7fe640 1 --2- 172.21.15.55:0/2785452324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00b4080 0x7f16b015f2e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009870 tx=0x7f16a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc628eeee3d9402 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:37.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.088+0000 7f16ae7fc640 1 -- 172.21.15.55:0/2785452324 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0037070 con 0x7f16b00b4080 2024-05-20T22:08:37.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.088+0000 7f16ae7fc640 1 -- 172.21.15.55:0/2785452324 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a0032440 con 0x7f16b00b4080 2024-05-20T22:08:37.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.088+0000 7f16ae7fc640 1 -- 172.21.15.55:0/2785452324 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0032810 con 0x7f16b00b4080 2024-05-20T22:08:37.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.089+0000 7f16b5eb8640 1 -- 172.21.15.55:0/2785452324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00b4080 msgr2=0x7f16b015f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.089+0000 7f16b5eb8640 1 --2- 172.21.15.55:0/2785452324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00b4080 0x7f16b015f2e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009870 tx=0x7f16a002a210 comp rx=0 tx=0).stop 2024-05-20T22:08:37.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.089+0000 7f16b5eb8640 1 -- 172.21.15.55:0/2785452324 shutdown_connections 2024-05-20T22:08:37.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.089+0000 7f16b5eb8640 1 --2- 172.21.15.55:0/2785452324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00d3290 0x7f16b00d36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.089+0000 7f16b5eb8640 1 --2- 172.21.15.55:0/2785452324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00b4080 0x7f16b015f2e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.089+0000 7f16b5eb8640 1 -- 172.21.15.55:0/2785452324 >> 172.21.15.55:0/2785452324 conn(0x7f16b00d51b0 msgr2=0x7f16b00b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:37.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.089+0000 7f16b5eb8640 1 -- 172.21.15.55:0/2785452324 shutdown_connections 2024-05-20T22:08:37.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.089+0000 7f16b5eb8640 1 -- 172.21.15.55:0/2785452324 wait complete. 2024-05-20T22:08:37.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.091+0000 7f16b5eb8640 1 Processor -- start 2024-05-20T22:08:37.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.091+0000 7f16b5eb8640 1 -- start start 2024-05-20T22:08:37.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16b5eb8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00b35e0 0x7f16b00cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16b5eb8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b00b4080 0x7f16b0156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16b5eb8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00d3290 0x7f16b00c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16b5eb8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01635f0 con 0x7f16b00d3290 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16b5eb8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01604f0 con 0x7f16b00b4080 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16b5eb8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f16b015ff30 con 0x7f16b00b35e0 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16affff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00d3290 0x7f16b00c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16affff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00d3290 0x7f16b00c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38684/0 (socket says 172.21.15.55:38684) 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16affff640 1 -- 172.21.15.55:0/860806485 learned_addr learned my addr 172.21.15.55:0/860806485 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16af7fe640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00b35e0 0x7f16b00cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.092+0000 7f16aeffd640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b00b4080 0x7f16b0156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f16affff640 1 -- 172.21.15.55:0/860806485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00b35e0 msgr2=0x7f16b00cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f16affff640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00b35e0 0x7f16b00cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f16affff640 1 -- 172.21.15.55:0/860806485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b00b4080 msgr2=0x7f16b0156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f16affff640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b00b4080 0x7f16b0156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f16affff640 1 -- 172.21.15.55:0/860806485 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b00b8200 con 0x7f16b00d3290 2024-05-20T22:08:37.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f16aeffd640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b00b4080 0x7f16b0156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:37.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f16affff640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00d3290 0x7f16b00c8250 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f16a800cd80 tx=0x7f16a800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:37.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f1697fff640 1 -- 172.21.15.55:0/860806485 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800f680 con 0x7f16b00d3290 2024-05-20T22:08:37.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f1697fff640 1 -- 172.21.15.55:0/860806485 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a800f840 con 0x7f16b00d3290 2024-05-20T22:08:37.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f16b5eb8640 1 -- 172.21.15.55:0/860806485 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b00b84e0 con 0x7f16b00d3290 2024-05-20T22:08:37.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f1697fff640 1 -- 172.21.15.55:0/860806485 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800fc30 con 0x7f16b00d3290 2024-05-20T22:08:37.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.093+0000 7f16b5eb8640 1 -- 172.21.15.55:0/860806485 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b00b8aa0 con 0x7f16b00d3290 2024-05-20T22:08:37.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.094+0000 7f16b5eb8640 1 -- 172.21.15.55:0/860806485 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b0069090 con 0x7f16b00d3290 2024-05-20T22:08:37.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.095+0000 7f1697fff640 1 -- 172.21.15.55:0/860806485 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f16a80213f0 con 0x7f16b00d3290 2024-05-20T22:08:37.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.095+0000 7f1697fff640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f169c06be60 0x7f169c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.095+0000 7f1697fff640 1 -- 172.21.15.55:0/860806485 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f16a808bfa0 con 0x7f16b00d3290 2024-05-20T22:08:37.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.095+0000 7f16af7fe640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f169c06be60 0x7f169c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.096+0000 7f16af7fe640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f169c06be60 0x7f169c06e370 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16a0034000 tx=0x7f16a002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:37.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.100+0000 7f1697fff640 1 -- 172.21.15.55:0/860806485 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f16a8014030 con 0x7f16b00d3290 2024-05-20T22:08:37.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.282+0000 7f16b5eb8640 1 -- 172.21.15.55:0/860806485 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f16b00d2ec0 con 0x7f16b00d3290 2024-05-20T22:08:37.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.283+0000 7f1697fff640 1 -- 172.21.15.55:0/860806485 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f16a805a4c0 con 0x7f16b00d3290 2024-05-20T22:08:37.284 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.284+0000 7f16b5eb8640 1 -- 172.21.15.55:0/860806485 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f169c06be60 msgr2=0x7f169c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.284+0000 7f16b5eb8640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f169c06be60 0x7f169c06e370 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16a0034000 tx=0x7f16a002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.284+0000 7f16b5eb8640 1 -- 172.21.15.55:0/860806485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00d3290 msgr2=0x7f16b00c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.284+0000 7f16b5eb8640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00d3290 0x7f16b00c8250 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f16a800cd80 tx=0x7f16a800a650 comp rx=0 tx=0).stop 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.285+0000 7f16b5eb8640 1 -- 172.21.15.55:0/860806485 shutdown_connections 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.285+0000 7f16b5eb8640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f169c06be60 0x7f169c06e370 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.285+0000 7f16b5eb8640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16b00d3290 0x7f16b00c8250 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.285+0000 7f16b5eb8640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b00b4080 0x7f16b0156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.285+0000 7f16b5eb8640 1 --2- 172.21.15.55:0/860806485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16b00b35e0 0x7f16b00cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.285+0000 7f16b5eb8640 1 -- 172.21.15.55:0/860806485 >> 172.21.15.55:0/860806485 conn(0x7f16b00d51b0 msgr2=0x7f16b00d4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.285+0000 7f16b5eb8640 1 -- 172.21.15.55:0/860806485 shutdown_connections 2024-05-20T22:08:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.285+0000 7f16b5eb8640 1 -- 172.21.15.55:0/860806485 wait complete. 2024-05-20T22:08:37.297 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:37.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.411+0000 7fc2b635e640 1 Processor -- start 2024-05-20T22:08:37.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.411+0000 7fc2b635e640 1 -- start start 2024-05-20T22:08:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.412+0000 7fc2b635e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2b00b3b30 0x7fc2b00b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2b635e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc2b006c790 0x7fc2b00cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2b635e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2b00d3dd0 con 0x7fc2b015f100 2024-05-20T22:08:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2b635e640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fc2b00d40b0 con 0x7fc2b00b3b30 2024-05-20T22:08:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2b635e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc2b00d4390 con 0x7fc2b006c790 2024-05-20T22:08:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2affff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2b00b3b30 0x7fc2b00b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2affff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2b00b3b30 0x7fc2b00b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38700/0 (socket says 172.21.15.55:38700) 2024-05-20T22:08:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2affff640 1 -- 172.21.15.55:0/3709930134 learned_addr learned my addr 172.21.15.55:0/3709930134 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2b48fb640 1 --2- 172.21.15.55:0/3709930134 >> v2:172.21.15.138:3300/0 conn(0x7fc2b006c790 0x7fc2b00cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2affff640 1 -- 172.21.15.55:0/3709930134 >> v2:172.21.15.138:3300/0 conn(0x7fc2b006c790 msgr2=0x7fc2b00cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:37.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2affff640 1 --2- 172.21.15.55:0/3709930134 >> v2:172.21.15.138:3300/0 conn(0x7fc2b006c790 0x7fc2b00cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2affff640 1 -- 172.21.15.55:0/3709930134 >> v1:172.21.15.138:6789/0 conn(0x7fc2b015f100 legacy=0x7fc2b006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.413+0000 7fc2affff640 1 -- 172.21.15.55:0/3709930134 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b006b5a0 con 0x7fc2b00b3b30 2024-05-20T22:08:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.414+0000 7fc2affff640 1 --2- 172.21.15.55:0/3709930134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b00b3b30 0x7fc2b00b7fe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00d12f0 tx=0x7fc2a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d665940ff43c840 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.414+0000 7fc2aeffd640 1 -- 172.21.15.55:0/3709930134 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0037070 con 0x7fc2b00b3b30 2024-05-20T22:08:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.414+0000 7fc2aeffd640 1 -- 172.21.15.55:0/3709930134 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a0032440 con 0x7fc2b00b3b30 2024-05-20T22:08:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.414+0000 7fc2aeffd640 1 -- 172.21.15.55:0/3709930134 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0032810 con 0x7fc2b00b3b30 2024-05-20T22:08:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.415+0000 7fc2b635e640 1 -- 172.21.15.55:0/3709930134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b00b3b30 msgr2=0x7fc2b00b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.415+0000 7fc2b635e640 1 --2- 172.21.15.55:0/3709930134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b00b3b30 0x7fc2b00b7fe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00d12f0 tx=0x7fc2a002a270 comp rx=0 tx=0).stop 2024-05-20T22:08:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.415+0000 7fc2b635e640 1 -- 172.21.15.55:0/3709930134 shutdown_connections 2024-05-20T22:08:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.415+0000 7fc2b635e640 1 --2- 172.21.15.55:0/3709930134 >> v2:172.21.15.138:3300/0 conn(0x7fc2b006c790 0x7fc2b00cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.415+0000 7fc2b635e640 1 --2- 172.21.15.55:0/3709930134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b00b3b30 0x7fc2b00b7fe0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.415+0000 7fc2b635e640 1 -- 172.21.15.55:0/3709930134 >> 172.21.15.55:0/3709930134 conn(0x7fc2b00d51d0 msgr2=0x7fc2b00b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:37.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.415+0000 7fc2b635e640 1 -- 172.21.15.55:0/3709930134 shutdown_connections 2024-05-20T22:08:37.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.415+0000 7fc2b635e640 1 -- 172.21.15.55:0/3709930134 wait complete. 2024-05-20T22:08:37.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.417+0000 7fc2b635e640 1 Processor -- start 2024-05-20T22:08:37.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.417+0000 7fc2b635e640 1 -- start start 2024-05-20T22:08:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2b635e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2b006c790 0x7fc2b01ea540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2b635e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2b00b3b30 0x7fc2b014d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2b635e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b015f100 0x7fc2b0155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2b635e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b006cba0 con 0x7fc2b015f100 2024-05-20T22:08:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2b635e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b006bb60 con 0x7fc2b006c790 2024-05-20T22:08:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2b635e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b006cd20 con 0x7fc2b00b3b30 2024-05-20T22:08:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2b48fb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b015f100 0x7fc2b0155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2b48fb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b015f100 0x7fc2b0155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38704/0 (socket says 172.21.15.55:38704) 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2b48fb640 1 -- 172.21.15.55:0/4199010917 learned_addr learned my addr 172.21.15.55:0/4199010917 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2affff640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2b006c790 0x7fc2b01ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.418+0000 7fc2af7fe640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2b00b3b30 0x7fc2b014d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2b48fb640 1 -- 172.21.15.55:0/4199010917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2b00b3b30 msgr2=0x7fc2b014d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2b48fb640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2b00b3b30 0x7fc2b014d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2b48fb640 1 -- 172.21.15.55:0/4199010917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2b006c790 msgr2=0x7fc2b01ea540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2b48fb640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2b006c790 0x7fc2b01ea540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2b48fb640 1 -- 172.21.15.55:0/4199010917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b014dcf0 con 0x7fc2b015f100 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2b48fb640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b015f100 0x7fc2b0155880 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009d30 tx=0x7fc2a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2acff9640 1 -- 172.21.15.55:0/4199010917 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8018020 con 0x7fc2b015f100 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2acff9640 1 -- 172.21.15.55:0/4199010917 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a8014480 con 0x7fc2b015f100 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2b635e640 1 -- 172.21.15.55:0/4199010917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b01e94a0 con 0x7fc2b015f100 2024-05-20T22:08:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2acff9640 1 -- 172.21.15.55:0/4199010917 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8014870 con 0x7fc2b015f100 2024-05-20T22:08:37.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.419+0000 7fc2b635e640 1 -- 172.21.15.55:0/4199010917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b01e9a60 con 0x7fc2b015f100 2024-05-20T22:08:37.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.420+0000 7fc2b635e640 1 -- 172.21.15.55:0/4199010917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b0069090 con 0x7fc2b015f100 2024-05-20T22:08:37.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.421+0000 7fc2acff9640 1 -- 172.21.15.55:0/4199010917 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2a8005d10 con 0x7fc2b015f100 2024-05-20T22:08:37.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.421+0000 7fc2acff9640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc29c06be30 0x7fc29c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.421+0000 7fc2acff9640 1 -- 172.21.15.55:0/4199010917 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc2a808be20 con 0x7fc2b015f100 2024-05-20T22:08:37.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.421+0000 7fc2affff640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc29c06be30 0x7fc29c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.422+0000 7fc2affff640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc29c06be30 0x7fc29c06e340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0034000 tx=0x7fc2a002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:37.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.426+0000 7fc2acff9640 1 -- 172.21.15.55:0/4199010917 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc2a8022030 con 0x7fc2b015f100 2024-05-20T22:08:37.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.611+0000 7fc2b635e640 1 -- 172.21.15.55:0/4199010917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2b01e9230 con 0x7fc2b015f100 2024-05-20T22:08:37.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.612+0000 7fc2acff9640 1 -- 172.21.15.55:0/4199010917 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7fc2a8010040 con 0x7fc2b015f100 2024-05-20T22:08:37.613 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:37.613 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:37.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 -- 172.21.15.55:0/4199010917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc29c06be30 msgr2=0x7fc29c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc29c06be30 0x7fc29c06e340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0034000 tx=0x7fc2a002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:37.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 -- 172.21.15.55:0/4199010917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b015f100 msgr2=0x7fc2b0155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b015f100 0x7fc2b0155880 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009d30 tx=0x7fc2a800c490 comp rx=0 tx=0).stop 2024-05-20T22:08:37.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 -- 172.21.15.55:0/4199010917 shutdown_connections 2024-05-20T22:08:37.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc29c06be30 0x7fc29c06e340 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b015f100 0x7fc2b0155880 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2b00b3b30 0x7fc2b014d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 --2- 172.21.15.55:0/4199010917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2b006c790 0x7fc2b01ea540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 -- 172.21.15.55:0/4199010917 >> 172.21.15.55:0/4199010917 conn(0x7fc2b00d51d0 msgr2=0x7fc2b00b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:37.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 -- 172.21.15.55:0/4199010917 shutdown_connections 2024-05-20T22:08:37.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.614+0000 7fc2b635e640 1 -- 172.21.15.55:0/4199010917 wait complete. 2024-05-20T22:08:37.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:37.630 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:37.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.750+0000 7f6b76509640 1 Processor -- start 2024-05-20T22:08:37.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.750+0000 7f6b76509640 1 -- start start 2024-05-20T22:08:37.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.751+0000 7f6b76509640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6b700b29e0 0x7f6b7015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.751+0000 7f6b76509640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b700cf7f0 0x7f6b700d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.751+0000 7f6b76509640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7008c690 con 0x7f6b7008bcb0 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.751+0000 7f6b76509640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f6b700b2df0 con 0x7f6b700b29e0 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.751+0000 7f6b76509640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7008c970 con 0x7f6b700cf7f0 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.751+0000 7f6b6ffff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6b700b29e0 0x7f6b7015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.751+0000 7f6b67fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6b7008bcb0 0x7f6b700cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56844/0 (socket says 172.21.15.55:56844) 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.751+0000 7f6b67fff640 1 -- 172.21.15.55:0/2637550247 learned_addr learned my addr 172.21.15.55:0/2637550247 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.752+0000 7f6b74aa6640 1 --2- 172.21.15.55:0/2637550247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b700cf7f0 0x7f6b700d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.752+0000 7f6b6ffff640 1 -- 172.21.15.55:0/2637550247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b700cf7f0 msgr2=0x7f6b700d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.752+0000 7f6b6ffff640 1 --2- 172.21.15.55:0/2637550247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b700cf7f0 0x7f6b700d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.752+0000 7f6b6ffff640 1 -- 172.21.15.55:0/2637550247 >> v1:172.21.15.55:6789/0 conn(0x7f6b7008bcb0 legacy=0x7f6b700cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.752+0000 7f6b6f7fe640 1 -- 172.21.15.55:0/2637550247 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3914097704 0 0) 0x7f6b7008c690 con 0x7f6b7008bcb0 2024-05-20T22:08:37.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.752+0000 7f6b6ffff640 1 -- 172.21.15.55:0/2637550247 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b700d0de0 con 0x7f6b700b29e0 2024-05-20T22:08:37.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.753+0000 7f6b6ffff640 1 --2- 172.21.15.55:0/2637550247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b700b29e0 0x7f6b7015f0e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6b600097a0 tx=0x7f6b6002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700b15938f40724f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:37.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.753+0000 7f6b6f7fe640 1 -- 172.21.15.55:0/2637550247 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60037070 con 0x7f6b700b29e0 2024-05-20T22:08:37.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.753+0000 7f6b6f7fe640 1 -- 172.21.15.55:0/2637550247 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b60032440 con 0x7f6b700b29e0 2024-05-20T22:08:37.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.753+0000 7f6b6f7fe640 1 -- 172.21.15.55:0/2637550247 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60032810 con 0x7f6b700b29e0 2024-05-20T22:08:37.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.753+0000 7f6b76509640 1 -- 172.21.15.55:0/2637550247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b700b29e0 msgr2=0x7f6b7015f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.753+0000 7f6b76509640 1 --2- 172.21.15.55:0/2637550247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b700b29e0 0x7f6b7015f0e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6b600097a0 tx=0x7f6b6002a180 comp rx=0 tx=0).stop 2024-05-20T22:08:37.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.754+0000 7f6b76509640 1 -- 172.21.15.55:0/2637550247 shutdown_connections 2024-05-20T22:08:37.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.754+0000 7f6b76509640 1 --2- 172.21.15.55:0/2637550247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b700cf7f0 0x7f6b700d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.754+0000 7f6b76509640 1 --2- 172.21.15.55:0/2637550247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b700b29e0 0x7f6b7015f0e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.754+0000 7f6b76509640 1 -- 172.21.15.55:0/2637550247 >> 172.21.15.55:0/2637550247 conn(0x7f6b700d2860 msgr2=0x7f6b700d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:37.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.754+0000 7f6b76509640 1 -- 172.21.15.55:0/2637550247 shutdown_connections 2024-05-20T22:08:37.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.754+0000 7f6b76509640 1 -- 172.21.15.55:0/2637550247 wait complete. 2024-05-20T22:08:37.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.755+0000 7f6b76509640 1 Processor -- start 2024-05-20T22:08:37.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.755+0000 7f6b76509640 1 -- start start 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b76509640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7008bcb0 0x7f6b701528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b76509640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b700b29e0 0x7f6b70071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b76509640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b70152f20 0x7f6b700af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b76509640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7008cc50 con 0x7f6b7008bcb0 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b76509640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7015f620 con 0x7f6b700b29e0 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b76509640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6b700b80e0 con 0x7f6b70152f20 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b6ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7008bcb0 0x7f6b701528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b6ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7008bcb0 0x7f6b701528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38732/0 (socket says 172.21.15.55:38732) 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b6ffff640 1 -- 172.21.15.55:0/1791582019 learned_addr learned my addr 172.21.15.55:0/1791582019 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b74aa6640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b70152f20 0x7f6b700af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b67fff640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b700b29e0 0x7f6b70071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b6ffff640 1 -- 172.21.15.55:0/1791582019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b70152f20 msgr2=0x7f6b700af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b6ffff640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b70152f20 0x7f6b700af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b6ffff640 1 -- 172.21.15.55:0/1791582019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b700b29e0 msgr2=0x7f6b70071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b6ffff640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b700b29e0 0x7f6b70071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b6ffff640 1 -- 172.21.15.55:0/1791582019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b70072010 con 0x7f6b7008bcb0 2024-05-20T22:08:37.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b6ffff640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7008bcb0 0x7f6b701528b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6b6002a770 tx=0x7f6b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:37.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.757+0000 7f6b6d7fa640 1 -- 172.21.15.55:0/1791582019 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60037040 con 0x7f6b7008bcb0 2024-05-20T22:08:37.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.758+0000 7f6b6d7fa640 1 -- 172.21.15.55:0/1791582019 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b60030770 con 0x7f6b7008bcb0 2024-05-20T22:08:37.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.758+0000 7f6b6d7fa640 1 -- 172.21.15.55:0/1791582019 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60030b40 con 0x7f6b7008bcb0 2024-05-20T22:08:37.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.758+0000 7f6b76509640 1 -- 172.21.15.55:0/1791582019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b70070a30 con 0x7f6b7008bcb0 2024-05-20T22:08:37.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.758+0000 7f6b76509640 1 -- 172.21.15.55:0/1791582019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b70070ff0 con 0x7f6b7008bcb0 2024-05-20T22:08:37.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.759+0000 7f6b76509640 1 -- 172.21.15.55:0/1791582019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b70068750 con 0x7f6b7008bcb0 2024-05-20T22:08:37.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.759+0000 7f6b6d7fa640 1 -- 172.21.15.55:0/1791582019 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b60044020 con 0x7f6b7008bcb0 2024-05-20T22:08:37.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.760+0000 7f6b6d7fa640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b5006be60 0x7f6b5006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:37.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.760+0000 7f6b6d7fa640 1 -- 172.21.15.55:0/1791582019 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6b600ab580 con 0x7f6b7008bcb0 2024-05-20T22:08:37.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.760+0000 7f6b67fff640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b5006be60 0x7f6b5006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:37.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.760+0000 7f6b67fff640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b5006be60 0x7f6b5006e370 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b58000f60 tx=0x7f6b58007c00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:37.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.765+0000 7f6b6d7fa640 1 -- 172.21.15.55:0/1791582019 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6b70068750 con 0x7f6b7008bcb0 2024-05-20T22:08:37.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.948+0000 7f6b76509640 1 -- 172.21.15.55:0/1791582019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b70069090 con 0x7f6b7008bcb0 2024-05-20T22:08:37.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.948+0000 7f6b6d7fa640 1 -- 172.21.15.55:0/1791582019 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f6b6004b020 con 0x7f6b7008bcb0 2024-05-20T22:08:37.949 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:37.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 -- 172.21.15.55:0/1791582019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b5006be60 msgr2=0x7f6b5006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b5006be60 0x7f6b5006e370 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b58000f60 tx=0x7f6b58007c00 comp rx=0 tx=0).stop 2024-05-20T22:08:37.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 -- 172.21.15.55:0/1791582019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7008bcb0 msgr2=0x7f6b701528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:37.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7008bcb0 0x7f6b701528b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6b6002a770 tx=0x7f6b600046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 -- 172.21.15.55:0/1791582019 shutdown_connections 2024-05-20T22:08:37.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b5006be60 0x7f6b5006e370 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b70152f20 0x7f6b700af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b700b29e0 0x7f6b70071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 --2- 172.21.15.55:0/1791582019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b7008bcb0 0x7f6b701528b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:37.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 -- 172.21.15.55:0/1791582019 >> 172.21.15.55:0/1791582019 conn(0x7f6b700d2860 msgr2=0x7f6b70162d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:37.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 -- 172.21.15.55:0/1791582019 shutdown_connections 2024-05-20T22:08:37.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:37.950+0000 7f6b76509640 1 -- 172.21.15.55:0/1791582019 wait complete. 2024-05-20T22:08:37.962 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:38.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.079+0000 7fb74af45640 1 Processor -- start 2024-05-20T22:08:38.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.079+0000 7fb74af45640 1 -- start start 2024-05-20T22:08:38.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb74af45640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7440d32a0 0x7fb74415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb74af45640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb7440cf690 0x7fb7440cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb74af45640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7440d3780 con 0x7fb7440cffe0 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb74af45640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fb74406c510 con 0x7fb7440cf690 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb74af45640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7440d3a60 con 0x7fb7440d32a0 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb743fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb7440cf690 0x7fb7440cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb743fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb7440cf690 0x7fb7440cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38742/0 (socket says 172.21.15.55:38742) 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb743fff640 1 -- 172.21.15.55:0/3082039387 learned_addr learned my addr 172.21.15.55:0/3082039387 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb748ce1640 1 --2- 172.21.15.55:0/3082039387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7440d32a0 0x7fb74415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb743fff640 1 -- 172.21.15.55:0/3082039387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7440d32a0 msgr2=0x7fb74415f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb743fff640 1 --2- 172.21.15.55:0/3082039387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7440d32a0 0x7fb74415f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb743fff640 1 -- 172.21.15.55:0/3082039387 >> v1:172.21.15.138:6789/0 conn(0x7fb7440cffe0 legacy=0x7fb744162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.081+0000 7fb743fff640 1 -- 172.21.15.55:0/3082039387 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7440d3d40 con 0x7fb7440cf690 2024-05-20T22:08:38.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.082+0000 7fb743fff640 1 --2- 172.21.15.55:0/3082039387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440cf690 0x7fb7440cfaa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb730009870 tx=0x7fb73002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eb9a7fdc2be92d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:38.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.082+0000 7fb7437fe640 1 -- 172.21.15.55:0/3082039387 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730037070 con 0x7fb7440cf690 2024-05-20T22:08:38.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.082+0000 7fb7437fe640 1 -- 172.21.15.55:0/3082039387 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb730032440 con 0x7fb7440cf690 2024-05-20T22:08:38.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.082+0000 7fb7437fe640 1 -- 172.21.15.55:0/3082039387 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730032810 con 0x7fb7440cf690 2024-05-20T22:08:38.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.083+0000 7fb74af45640 1 -- 172.21.15.55:0/3082039387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440cf690 msgr2=0x7fb7440cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.083+0000 7fb74af45640 1 --2- 172.21.15.55:0/3082039387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440cf690 0x7fb7440cfaa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb730009870 tx=0x7fb73002a270 comp rx=0 tx=0).stop 2024-05-20T22:08:38.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.083+0000 7fb74af45640 1 -- 172.21.15.55:0/3082039387 shutdown_connections 2024-05-20T22:08:38.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.083+0000 7fb74af45640 1 --2- 172.21.15.55:0/3082039387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440cf690 0x7fb7440cfaa0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.083+0000 7fb74af45640 1 --2- 172.21.15.55:0/3082039387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7440d32a0 0x7fb74415f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.083+0000 7fb74af45640 1 -- 172.21.15.55:0/3082039387 >> 172.21.15.55:0/3082039387 conn(0x7fb7440b6de0 msgr2=0x7fb7440d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:38.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.083+0000 7fb74af45640 1 -- 172.21.15.55:0/3082039387 shutdown_connections 2024-05-20T22:08:38.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.083+0000 7fb74af45640 1 -- 172.21.15.55:0/3082039387 wait complete. 2024-05-20T22:08:38.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.085+0000 7fb74af45640 1 Processor -- start 2024-05-20T22:08:38.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.085+0000 7fb74af45640 1 -- start start 2024-05-20T22:08:38.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.086+0000 7fb74af45640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7440cf690 0x7fb7440cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.086+0000 7fb74af45640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7440cffe0 0x7fb7440ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.086+0000 7fb74af45640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440d32a0 0x7fb744159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.086+0000 7fb74af45640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb7440d03f0 con 0x7fb7440d32a0 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.086+0000 7fb74af45640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7440d4300 con 0x7fb7440cf690 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.086+0000 7fb74af45640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7440d4820 con 0x7fb7440cffe0 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.086+0000 7fb7494e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440d32a0 0x7fb744159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.086+0000 7fb7494e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440d32a0 0x7fb744159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38744/0 (socket says 172.21.15.55:38744) 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.086+0000 7fb7494e2640 1 -- 172.21.15.55:0/1643043475 learned_addr learned my addr 172.21.15.55:0/1643043475 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb743fff640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7440cffe0 0x7fb7440ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb748ce1640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7440cf690 0x7fb7440cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb7494e2640 1 -- 172.21.15.55:0/1643043475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7440cffe0 msgr2=0x7fb7440ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb7494e2640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7440cffe0 0x7fb7440ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb7494e2640 1 -- 172.21.15.55:0/1643043475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7440cf690 msgr2=0x7fb7440cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb7494e2640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7440cf690 0x7fb7440cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb7494e2640 1 -- 172.21.15.55:0/1643043475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7440ce980 con 0x7fb7440d32a0 2024-05-20T22:08:38.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb748ce1640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7440cf690 0x7fb7440cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:38.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb7494e2640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440d32a0 0x7fb744159d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00c730 tx=0x7fb73c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:38.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb7417fa640 1 -- 172.21.15.55:0/1643043475 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c003b70 con 0x7fb7440d32a0 2024-05-20T22:08:38.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb7417fa640 1 -- 172.21.15.55:0/1643043475 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73c003d10 con 0x7fb7440d32a0 2024-05-20T22:08:38.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb74af45640 1 -- 172.21.15.55:0/1643043475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74415a280 con 0x7fb7440d32a0 2024-05-20T22:08:38.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb7417fa640 1 -- 172.21.15.55:0/1643043475 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c005310 con 0x7fb7440d32a0 2024-05-20T22:08:38.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.087+0000 7fb74af45640 1 -- 172.21.15.55:0/1643043475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7440ccb20 con 0x7fb7440d32a0 2024-05-20T22:08:38.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.088+0000 7fb74af45640 1 -- 172.21.15.55:0/1643043475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb744069090 con 0x7fb7440d32a0 2024-05-20T22:08:38.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.089+0000 7fb7417fa640 1 -- 172.21.15.55:0/1643043475 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb73c0054b0 con 0x7fb7440d32a0 2024-05-20T22:08:38.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.089+0000 7fb7417fa640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb72806be60 0x7fb72806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.089+0000 7fb7417fa640 1 -- 172.21.15.55:0/1643043475 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb73c08b150 con 0x7fb7440d32a0 2024-05-20T22:08:38.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.089+0000 7fb748ce1640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb72806be60 0x7fb72806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.090+0000 7fb748ce1640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb72806be60 0x7fb72806e370 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb738009a20 tx=0x7fb738008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.094+0000 7fb7417fa640 1 -- 172.21.15.55:0/1643043475 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb7440ce980 con 0x7fb7440d32a0 2024-05-20T22:08:38.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.277+0000 7fb74af45640 1 -- 172.21.15.55:0/1643043475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7440d58e0 con 0x7fb7440d32a0 2024-05-20T22:08:38.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.277+0000 7fb7417fa640 1 -- 172.21.15.55:0/1643043475 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fb73c0595d0 con 0x7fb7440d32a0 2024-05-20T22:08:38.279 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:38.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.279+0000 7fb74af45640 1 -- 172.21.15.55:0/1643043475 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb72806be60 msgr2=0x7fb72806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.279+0000 7fb74af45640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb72806be60 0x7fb72806e370 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb738009a20 tx=0x7fb738008040 comp rx=0 tx=0).stop 2024-05-20T22:08:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.279+0000 7fb74af45640 1 -- 172.21.15.55:0/1643043475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440d32a0 msgr2=0x7fb744159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.279+0000 7fb74af45640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440d32a0 0x7fb744159d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00c730 tx=0x7fb73c00cbf0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.279+0000 7fb74af45640 1 -- 172.21.15.55:0/1643043475 shutdown_connections 2024-05-20T22:08:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.279+0000 7fb74af45640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb72806be60 0x7fb72806e370 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.279+0000 7fb74af45640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7440d32a0 0x7fb744159d40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.279+0000 7fb74af45640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7440cffe0 0x7fb7440ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.279+0000 7fb74af45640 1 --2- 172.21.15.55:0/1643043475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7440cf690 0x7fb7440cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.279+0000 7fb74af45640 1 -- 172.21.15.55:0/1643043475 >> 172.21.15.55:0/1643043475 conn(0x7fb7440b6de0 msgr2=0x7fb74415cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.280+0000 7fb74af45640 1 -- 172.21.15.55:0/1643043475 shutdown_connections 2024-05-20T22:08:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.280+0000 7fb74af45640 1 -- 172.21.15.55:0/1643043475 wait complete. 2024-05-20T22:08:38.292 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:38.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.408+0000 7fc13c6f9640 1 Processor -- start 2024-05-20T22:08:38.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.408+0000 7fc13c6f9640 1 -- start start 2024-05-20T22:08:38.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc13c6f9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc13408bcb0 0x7fc1340cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc13c6f9640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc13408c690 con 0x7fc13408c0c0 2024-05-20T22:08:38.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc13c6f9640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc13408c970 con 0x7fc1340b29e0 2024-05-20T22:08:38.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc13c6f9640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc1340b2df0 con 0x7fc13408bcb0 2024-05-20T22:08:38.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc13ac96640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc13408c0c0 0x7fc1340d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56854/0 (socket says 172.21.15.55:56854) 2024-05-20T22:08:38.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc13ac96640 1 -- 172.21.15.55:0/1214008567 learned_addr learned my addr 172.21.15.55:0/1214008567 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:38.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc139c94640 1 --2- 172.21.15.55:0/1214008567 >> v2:172.21.15.138:3300/0 conn(0x7fc13408bcb0 0x7fc1340cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc139493640 1 -- 172.21.15.55:0/1214008567 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692364010 0 0) 0x7fc13408c690 con 0x7fc13408c0c0 2024-05-20T22:08:38.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc139493640 1 -- 172.21.15.55:0/1214008567 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc120003520 con 0x7fc13408c0c0 2024-05-20T22:08:38.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc139c94640 1 -- 172.21.15.55:0/1214008567 >> v1:172.21.15.138:6789/0 conn(0x7fc1340b29e0 legacy=0x7fc13415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc139c94640 1 -- 172.21.15.55:0/1214008567 >> v1:172.21.15.55:6789/0 conn(0x7fc13408c0c0 legacy=0x7fc1340d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.410+0000 7fc139c94640 1 -- 172.21.15.55:0/1214008567 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1340d0de0 con 0x7fc13408bcb0 2024-05-20T22:08:38.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.411+0000 7fc139c94640 1 --2- 172.21.15.55:0/1214008567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc13408bcb0 0x7fc1340cf0e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1240097a0 tx=0x7fc12402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a68d49d271a0c661 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:38.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.411+0000 7fc139493640 1 -- 172.21.15.55:0/1214008567 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124037070 con 0x7fc13408bcb0 2024-05-20T22:08:38.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.411+0000 7fc139493640 1 -- 172.21.15.55:0/1214008567 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc124032440 con 0x7fc13408bcb0 2024-05-20T22:08:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.411+0000 7fc139493640 1 -- 172.21.15.55:0/1214008567 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1240327f0 con 0x7fc13408bcb0 2024-05-20T22:08:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.412+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1214008567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc13408bcb0 msgr2=0x7fc1340cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.412+0000 7fc13c6f9640 1 --2- 172.21.15.55:0/1214008567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc13408bcb0 0x7fc1340cf0e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1240097a0 tx=0x7fc12402a230 comp rx=0 tx=0).stop 2024-05-20T22:08:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.412+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1214008567 shutdown_connections 2024-05-20T22:08:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.412+0000 7fc13c6f9640 1 --2- 172.21.15.55:0/1214008567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc13408bcb0 0x7fc1340cf0e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.412+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1214008567 >> 172.21.15.55:0/1214008567 conn(0x7fc1340d2860 msgr2=0x7fc1340d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.412+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1214008567 shutdown_connections 2024-05-20T22:08:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.412+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1214008567 wait complete. 2024-05-20T22:08:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.414+0000 7fc13c6f9640 1 Processor -- start 2024-05-20T22:08:38.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.414+0000 7fc13c6f9640 1 -- start start 2024-05-20T22:08:38.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.415+0000 7fc13c6f9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc13408bcb0 0x7fc13414e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.415+0000 7fc13c6f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc13408c0c0 0x7fc1340b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.415+0000 7fc13c6f9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1340b29e0 0x7fc1340b7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.415+0000 7fc13c6f9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc1340b80e0 con 0x7fc13408bcb0 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.415+0000 7fc13c6f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc13408cc50 con 0x7fc13408c0c0 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.415+0000 7fc13c6f9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc1340b8260 con 0x7fc1340b29e0 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.415+0000 7fc13a495640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc13408bcb0 0x7fc13414e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13a495640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc13408bcb0 0x7fc13414e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38750/0 (socket says 172.21.15.55:38750) 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13a495640 1 -- 172.21.15.55:0/1272805823 learned_addr learned my addr 172.21.15.55:0/1272805823 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc139c94640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc13408c0c0 0x7fc1340b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13ac96640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1340b29e0 0x7fc1340b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13a495640 1 -- 172.21.15.55:0/1272805823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1340b29e0 msgr2=0x7fc1340b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13a495640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1340b29e0 0x7fc1340b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13a495640 1 -- 172.21.15.55:0/1272805823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc13408c0c0 msgr2=0x7fc1340b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13a495640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc13408c0c0 0x7fc1340b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13a495640 1 -- 172.21.15.55:0/1272805823 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1340b5900 con 0x7fc13408bcb0 2024-05-20T22:08:38.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc139c94640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc13408c0c0 0x7fc1340b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:38.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13a495640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc13408bcb0 0x7fc13414e990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c440 tx=0x7fc13000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:38.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc12affd640 1 -- 172.21.15.55:0/1272805823 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1300052d0 con 0x7fc13408bcb0 2024-05-20T22:08:38.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc12affd640 1 -- 172.21.15.55:0/1272805823 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc130005470 con 0x7fc13408bcb0 2024-05-20T22:08:38.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1272805823 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1340b5be0 con 0x7fc13408bcb0 2024-05-20T22:08:38.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc12affd640 1 -- 172.21.15.55:0/1272805823 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130007260 con 0x7fc13408bcb0 2024-05-20T22:08:38.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.416+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1272805823 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1340b61a0 con 0x7fc13408bcb0 2024-05-20T22:08:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.417+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1272805823 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc134069090 con 0x7fc13408bcb0 2024-05-20T22:08:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.418+0000 7fc12affd640 1 -- 172.21.15.55:0/1272805823 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1300074c0 con 0x7fc13408bcb0 2024-05-20T22:08:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.418+0000 7fc12affd640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc11806be30 0x7fc11806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.418+0000 7fc12affd640 1 -- 172.21.15.55:0/1272805823 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc13008a6a0 con 0x7fc13408bcb0 2024-05-20T22:08:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.418+0000 7fc139c94640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc11806be30 0x7fc11806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.419+0000 7fc139c94640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc11806be30 0x7fc11806e340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc124034000 tx=0x7fc12402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.423+0000 7fc12affd640 1 -- 172.21.15.55:0/1272805823 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc130058c50 con 0x7fc13408bcb0 2024-05-20T22:08:38.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.606+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1272805823 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc1340b27c0 con 0x7fc13408bcb0 2024-05-20T22:08:38.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.607+0000 7fc12affd640 1 -- 172.21.15.55:0/1272805823 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc130058550 con 0x7fc13408bcb0 2024-05-20T22:08:38.608 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:38.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1272805823 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc11806be30 msgr2=0x7fc11806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc11806be30 0x7fc11806e340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc124034000 tx=0x7fc12402e000 comp rx=0 tx=0).stop 2024-05-20T22:08:38.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1272805823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc13408bcb0 msgr2=0x7fc13414e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc13408bcb0 0x7fc13414e990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c440 tx=0x7fc13000c900 comp rx=0 tx=0).stop 2024-05-20T22:08:38.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1272805823 shutdown_connections 2024-05-20T22:08:38.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc11806be30 0x7fc11806e340 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1340b29e0 0x7fc1340b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc13408c0c0 0x7fc1340b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 --2- 172.21.15.55:0/1272805823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc13408bcb0 0x7fc13414e990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1272805823 >> 172.21.15.55:0/1272805823 conn(0x7fc1340d2860 msgr2=0x7fc13406d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:38.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1272805823 shutdown_connections 2024-05-20T22:08:38.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.609+0000 7fc13c6f9640 1 -- 172.21.15.55:0/1272805823 wait complete. 2024-05-20T22:08:38.621 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:38.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.738+0000 7f32d9a5a640 1 Processor -- start 2024-05-20T22:08:38.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.738+0000 7f32d9a5a640 1 -- start start 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.739+0000 7f32d9a5a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f32d40b40e0 0x7f32d40cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.739+0000 7f32d9a5a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d415f1c0 0x7f32d40cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.739+0000 7f32d9a5a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d40d0400 con 0x7f32d40b3640 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.739+0000 7f32d9a5a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f32d40d3b40 con 0x7f32d40b40e0 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.739+0000 7f32d9a5a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f32d40d01b0 con 0x7f32d415f1c0 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.739+0000 7f32d2ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f32d40b40e0 0x7f32d40cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.739+0000 7f32d37fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f32d40b3640 0x7f32d4162bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60400/0 (socket says 172.21.15.55:60400) 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.739+0000 7f32d37fe640 1 -- 172.21.15.55:0/3367064845 learned_addr learned my addr 172.21.15.55:0/3367064845 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.740+0000 7f32d27fc640 1 --2- 172.21.15.55:0/3367064845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d415f1c0 0x7f32d40cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.740+0000 7f32d2ffd640 1 -- 172.21.15.55:0/3367064845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d415f1c0 msgr2=0x7f32d40cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.740+0000 7f32d2ffd640 1 --2- 172.21.15.55:0/3367064845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d415f1c0 0x7f32d40cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.740+0000 7f32d2ffd640 1 -- 172.21.15.55:0/3367064845 >> v1:172.21.15.55:6789/0 conn(0x7f32d40b3640 legacy=0x7f32d4162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.740+0000 7f32d2ffd640 1 -- 172.21.15.55:0/3367064845 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d40d3d90 con 0x7f32d40b40e0 2024-05-20T22:08:38.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.740+0000 7f32d27fc640 1 --2- 172.21.15.55:0/3367064845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d415f1c0 0x7f32d40cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:38.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.740+0000 7f32d2ffd640 1 --2- 172.21.15.55:0/3367064845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d40b40e0 0x7f32d40cf730 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f32d40d3cc0 tx=0x7f32c402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4271488f2d1c6c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:38.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d1ffb640 1 -- 172.21.15.55:0/3367064845 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4007740 con 0x7f32d40b40e0 2024-05-20T22:08:38.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d1ffb640 1 -- 172.21.15.55:0/3367064845 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c40078e0 con 0x7f32d40b40e0 2024-05-20T22:08:38.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d1ffb640 1 -- 172.21.15.55:0/3367064845 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4007cb0 con 0x7f32d40b40e0 2024-05-20T22:08:38.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d9a5a640 1 -- 172.21.15.55:0/3367064845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d40b40e0 msgr2=0x7f32d40cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d9a5a640 1 --2- 172.21.15.55:0/3367064845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d40b40e0 0x7f32d40cf730 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f32d40d3cc0 tx=0x7f32c402a5b0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d9a5a640 1 -- 172.21.15.55:0/3367064845 shutdown_connections 2024-05-20T22:08:38.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d9a5a640 1 --2- 172.21.15.55:0/3367064845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d415f1c0 0x7f32d40cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d9a5a640 1 --2- 172.21.15.55:0/3367064845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d40b40e0 0x7f32d40cf730 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d9a5a640 1 -- 172.21.15.55:0/3367064845 >> 172.21.15.55:0/3367064845 conn(0x7f32d40b65e0 msgr2=0x7f32d40b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:38.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d9a5a640 1 -- 172.21.15.55:0/3367064845 shutdown_connections 2024-05-20T22:08:38.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.741+0000 7f32d9a5a640 1 -- 172.21.15.55:0/3367064845 wait complete. 2024-05-20T22:08:38.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.743+0000 7f32d9a5a640 1 Processor -- start 2024-05-20T22:08:38.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.743+0000 7f32d9a5a640 1 -- start start 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d9a5a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d40b3640 0x7f32d41e9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d9a5a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d40b40e0 0x7f32d414bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d9a5a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d415f1c0 0x7f32d4149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d9a5a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f32d40b3d10 con 0x7f32d415f1c0 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d9a5a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32d40b3b90 con 0x7f32d40b40e0 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d9a5a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f32d406c510 con 0x7f32d40b3640 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d415f1c0 0x7f32d4149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d415f1c0 0x7f32d4149c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49416/0 (socket says 172.21.15.55:49416) 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d37fe640 1 -- 172.21.15.55:0/1602315007 learned_addr learned my addr 172.21.15.55:0/1602315007 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d2ffd640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d40b3640 0x7f32d41e9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d27fc640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d40b40e0 0x7f32d414bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d37fe640 1 -- 172.21.15.55:0/1602315007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d40b3640 msgr2=0x7f32d41e9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d37fe640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d40b3640 0x7f32d41e9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d37fe640 1 -- 172.21.15.55:0/1602315007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d40b40e0 msgr2=0x7f32d414bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d37fe640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d40b40e0 0x7f32d414bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d37fe640 1 -- 172.21.15.55:0/1602315007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d41e88f0 con 0x7f32d415f1c0 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d27fc640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d40b40e0 0x7f32d414bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d37fe640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d415f1c0 0x7f32d4149c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00c340 tx=0x7f32cc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32cbfff640 1 -- 172.21.15.55:0/1602315007 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc017070 con 0x7f32d415f1c0 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32cbfff640 1 -- 172.21.15.55:0/1602315007 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32cc005ce0 con 0x7f32d415f1c0 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32d9a5a640 1 -- 172.21.15.55:0/1602315007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d41e8bd0 con 0x7f32d415f1c0 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.745+0000 7f32cbfff640 1 -- 172.21.15.55:0/1602315007 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0072d0 con 0x7f32d415f1c0 2024-05-20T22:08:38.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.746+0000 7f32d9a5a640 1 -- 172.21.15.55:0/1602315007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d41e9190 con 0x7f32d415f1c0 2024-05-20T22:08:38.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.747+0000 7f32d9a5a640 1 -- 172.21.15.55:0/1602315007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d406c930 con 0x7f32d415f1c0 2024-05-20T22:08:38.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.747+0000 7f32cbfff640 1 -- 172.21.15.55:0/1602315007 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f32cc01a070 con 0x7f32d415f1c0 2024-05-20T22:08:38.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.748+0000 7f32cbfff640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f32b406be30 0x7f32b406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:38.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.748+0000 7f32cbfff640 1 -- 172.21.15.55:0/1602315007 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f32cc08a450 con 0x7f32d415f1c0 2024-05-20T22:08:38.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.748+0000 7f32d2ffd640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f32b406be30 0x7f32b406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:38.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.748+0000 7f32d2ffd640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f32b406be30 0x7f32b406e340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f32c4034000 tx=0x7f32c4033040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:38.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.752+0000 7f32cbfff640 1 -- 172.21.15.55:0/1602315007 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f32d406c930 con 0x7f32d415f1c0 2024-05-20T22:08:38.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.935+0000 7f32d9a5a640 1 -- 172.21.15.55:0/1602315007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f32d4069090 con 0x7f32d415f1c0 2024-05-20T22:08:38.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.936+0000 7f32cbfff640 1 -- 172.21.15.55:0/1602315007 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f32cc0583b0 con 0x7f32d415f1c0 2024-05-20T22:08:38.936 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:38.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 -- 172.21.15.55:0/1602315007 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f32b406be30 msgr2=0x7f32b406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f32b406be30 0x7f32b406e340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f32c4034000 tx=0x7f32c4033040 comp rx=0 tx=0).stop 2024-05-20T22:08:38.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 -- 172.21.15.55:0/1602315007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d415f1c0 msgr2=0x7f32d4149c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:38.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d415f1c0 0x7f32d4149c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00c340 tx=0x7f32cc00c800 comp rx=0 tx=0).stop 2024-05-20T22:08:38.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 -- 172.21.15.55:0/1602315007 shutdown_connections 2024-05-20T22:08:38.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f32b406be30 0x7f32b406e340 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32d415f1c0 0x7f32d4149c70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d40b40e0 0x7f32d414bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 --2- 172.21.15.55:0/1602315007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32d40b3640 0x7f32d41e9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:38.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 -- 172.21.15.55:0/1602315007 >> 172.21.15.55:0/1602315007 conn(0x7f32d40b65e0 msgr2=0x7f32d40b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:38.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 -- 172.21.15.55:0/1602315007 shutdown_connections 2024-05-20T22:08:38.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:38.938+0000 7f32d9a5a640 1 -- 172.21.15.55:0/1602315007 wait complete. 2024-05-20T22:08:38.998 INFO:tasks.cephfs_test_runner:Starting test: test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) The maximum number of openfiles omap objects keys are now equal to 2024-05-20T22:08:38.998 INFO:tasks.cephfs_test_runner:test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) 2024-05-20T22:08:38.999 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:39.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.114+0000 7f1e9edce640 1 Processor -- start 2024-05-20T22:08:39.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.114+0000 7f1e9edce640 1 -- start start 2024-05-20T22:08:39.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.115+0000 7f1e9edce640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1e980b7fe0 0x7f1e98169f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.115+0000 7f1e9edce640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e980b83f0 con 0x7f1e980b3b30 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.115+0000 7f1e9edce640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e980d3dd0 con 0x7f1e9815f100 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.115+0000 7f1e9edce640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f1e9806cea0 con 0x7f1e980b7fe0 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.115+0000 7f1e9cb6a640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1e980b3b30 0x7f1e9806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60406/0 (socket says 172.21.15.55:60406) 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.115+0000 7f1e9cb6a640 1 -- 172.21.15.55:0/870500861 learned_addr learned my addr 172.21.15.55:0/870500861 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.115+0000 7f1e9d36b640 1 --2- 172.21.15.55:0/870500861 >> v2:172.21.15.55:3300/0 conn(0x7f1e980b7fe0 0x7f1e98169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.115+0000 7f1e977fe640 1 -- 172.21.15.55:0/870500861 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838925511 0 0) 0x7f1e980b83f0 con 0x7f1e980b3b30 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.116+0000 7f1e977fe640 1 -- 172.21.15.55:0/870500861 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e84003520 con 0x7f1e980b3b30 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.116+0000 7f1e9d36b640 1 -- 172.21.15.55:0/870500861 >> v1:172.21.15.138:6789/0 conn(0x7f1e9815f100 legacy=0x7f1e9806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.116+0000 7f1e9d36b640 1 -- 172.21.15.55:0/870500861 >> v1:172.21.15.55:6789/0 conn(0x7f1e980b3b30 legacy=0x7f1e9806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.116+0000 7f1e9d36b640 1 -- 172.21.15.55:0/870500861 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e980d40b0 con 0x7f1e980b7fe0 2024-05-20T22:08:39.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.116+0000 7f1e9d36b640 1 --2- 172.21.15.55:0/870500861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 0x7f1e98169f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009d90 tx=0x7f1e9002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95853a0665367e4b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.116+0000 7f1e977fe640 1 -- 172.21.15.55:0/870500861 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90007740 con 0x7f1e980b7fe0 2024-05-20T22:08:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.116+0000 7f1e977fe640 1 -- 172.21.15.55:0/870500861 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e900078e0 con 0x7f1e980b7fe0 2024-05-20T22:08:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.117+0000 7f1e977fe640 1 -- 172.21.15.55:0/870500861 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90007c90 con 0x7f1e980b7fe0 2024-05-20T22:08:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.117+0000 7f1e9edce640 1 -- 172.21.15.55:0/870500861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 msgr2=0x7f1e98169f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.117+0000 7f1e9edce640 1 --2- 172.21.15.55:0/870500861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 0x7f1e98169f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009d90 tx=0x7f1e9002a6f0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.117+0000 7f1e9edce640 1 -- 172.21.15.55:0/870500861 shutdown_connections 2024-05-20T22:08:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.117+0000 7f1e9edce640 1 --2- 172.21.15.55:0/870500861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 0x7f1e98169f00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.117+0000 7f1e9edce640 1 -- 172.21.15.55:0/870500861 >> 172.21.15.55:0/870500861 conn(0x7f1e980d51d0 msgr2=0x7f1e980b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.117+0000 7f1e9edce640 1 -- 172.21.15.55:0/870500861 shutdown_connections 2024-05-20T22:08:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.117+0000 7f1e9edce640 1 -- 172.21.15.55:0/870500861 wait complete. 2024-05-20T22:08:39.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.119+0000 7f1e9edce640 1 Processor -- start 2024-05-20T22:08:39.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.119+0000 7f1e9edce640 1 -- start start 2024-05-20T22:08:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.120+0000 7f1e9edce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e980b3b30 0x7f1e98159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.120+0000 7f1e9edce640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 0x7f1e980b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.120+0000 7f1e9edce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9815f100 0x7f1e980ba250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.120+0000 7f1e9edce640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9806b880 con 0x7f1e980b7fe0 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.120+0000 7f1e9edce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9806b5a0 con 0x7f1e980b3b30 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.120+0000 7f1e9edce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9806bb60 con 0x7f1e9815f100 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e97fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 0x7f1e980b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e97fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 0x7f1e980b9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49436/0 (socket says 172.21.15.55:49436) 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e97fff640 1 -- 172.21.15.55:0/2127953601 learned_addr learned my addr 172.21.15.55:0/2127953601 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e9cb6a640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e980b3b30 0x7f1e98159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e9d36b640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9815f100 0x7f1e980ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e97fff640 1 -- 172.21.15.55:0/2127953601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9815f100 msgr2=0x7f1e980ba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e97fff640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9815f100 0x7f1e980ba250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e97fff640 1 -- 172.21.15.55:0/2127953601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e980b3b30 msgr2=0x7f1e98159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e97fff640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e980b3b30 0x7f1e98159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e97fff640 1 -- 172.21.15.55:0/2127953601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e981f9e60 con 0x7f1e980b7fe0 2024-05-20T22:08:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e97fff640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 0x7f1e980b9d10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c3c0 tx=0x7f1e8000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e957fa640 1 -- 172.21.15.55:0/2127953601 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800052d0 con 0x7f1e980b7fe0 2024-05-20T22:08:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e957fa640 1 -- 172.21.15.55:0/2127953601 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e80005490 con 0x7f1e980b7fe0 2024-05-20T22:08:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e9edce640 1 -- 172.21.15.55:0/2127953601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e981fa140 con 0x7f1e980b7fe0 2024-05-20T22:08:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e957fa640 1 -- 172.21.15.55:0/2127953601 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80007260 con 0x7f1e980b7fe0 2024-05-20T22:08:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.121+0000 7f1e9edce640 1 -- 172.21.15.55:0/2127953601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e981fa700 con 0x7f1e980b7fe0 2024-05-20T22:08:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.122+0000 7f1e9edce640 1 -- 172.21.15.55:0/2127953601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e98069090 con 0x7f1e980b7fe0 2024-05-20T22:08:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.123+0000 7f1e957fa640 1 -- 172.21.15.55:0/2127953601 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e80007400 con 0x7f1e980b7fe0 2024-05-20T22:08:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.123+0000 7f1e957fa640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7806be60 0x7f1e7806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.124+0000 7f1e957fa640 1 -- 172.21.15.55:0/2127953601 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1e8008a810 con 0x7f1e980b7fe0 2024-05-20T22:08:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.124+0000 7f1e9cb6a640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7806be60 0x7f1e7806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.124+0000 7f1e9cb6a640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7806be60 0x7f1e7806e370 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1e88005880 tx=0x7f1e88009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.128+0000 7f1e957fa640 1 -- 172.21.15.55:0/2127953601 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1e80058d30 con 0x7f1e980b7fe0 2024-05-20T22:08:39.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.314+0000 7f1e9edce640 1 -- 172.21.15.55:0/2127953601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e98161dd0 con 0x7f1e980b7fe0 2024-05-20T22:08:39.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.315+0000 7f1e957fa640 1 -- 172.21.15.55:0/2127953601 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7f1e80058630 con 0x7f1e980b7fe0 2024-05-20T22:08:39.316 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:39.317 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 -- 172.21.15.55:0/2127953601 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7806be60 msgr2=0x7f1e7806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7806be60 0x7f1e7806e370 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1e88005880 tx=0x7f1e88009040 comp rx=0 tx=0).stop 2024-05-20T22:08:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 -- 172.21.15.55:0/2127953601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 msgr2=0x7f1e980b9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 0x7f1e980b9d10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c3c0 tx=0x7f1e8000c7c0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 -- 172.21.15.55:0/2127953601 shutdown_connections 2024-05-20T22:08:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7806be60 0x7f1e7806e370 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9815f100 0x7f1e980ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e980b7fe0 0x7f1e980b9d10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 --2- 172.21.15.55:0/2127953601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e980b3b30 0x7f1e98159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 -- 172.21.15.55:0/2127953601 >> 172.21.15.55:0/2127953601 conn(0x7f1e980d51d0 msgr2=0x7f1e9806c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 -- 172.21.15.55:0/2127953601 shutdown_connections 2024-05-20T22:08:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.317+0000 7f1e9edce640 1 -- 172.21.15.55:0/2127953601 wait complete. 2024-05-20T22:08:39.330 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:39.330 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:39.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.447+0000 7fd3f5a06640 1 Processor -- start 2024-05-20T22:08:39.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.447+0000 7fd3f5a06640 1 -- start start 2024-05-20T22:08:39.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3f5a06640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3f00b4080 0x7fd3f015f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3f5a06640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd3f00b35e0 0x7fd3f015f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3f5a06640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3f015fd60 0x7fd3f00cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3f5a06640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fd3f0163960 con 0x7fd3f00b35e0 2024-05-20T22:08:39.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3f5a06640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd3f006b500 con 0x7fd3f015fd60 2024-05-20T22:08:39.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3f5a06640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f00b3e10 con 0x7fd3f00b4080 2024-05-20T22:08:39.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3ee7fc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd3f00b35e0 0x7fd3f015f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3ee7fc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd3f00b35e0 0x7fd3f015f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49442/0 (socket says 172.21.15.55:49442) 2024-05-20T22:08:39.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3ee7fc640 1 -- 172.21.15.55:0/3720191570 learned_addr learned my addr 172.21.15.55:0/3720191570 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:39.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3eeffd640 1 --2- 172.21.15.55:0/3720191570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3f00b4080 0x7fd3f015f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3ef7fe640 1 --2- 172.21.15.55:0/3720191570 >> v2:172.21.15.138:3300/0 conn(0x7fd3f015fd60 0x7fd3f00cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3ee7fc640 1 -- 172.21.15.55:0/3720191570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3f00b4080 msgr2=0x7fd3f015f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:39.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3ee7fc640 1 --2- 172.21.15.55:0/3720191570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3f00b4080 0x7fd3f015f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3ee7fc640 1 -- 172.21.15.55:0/3720191570 >> v2:172.21.15.138:3300/0 conn(0x7fd3f015fd60 msgr2=0x7fd3f00cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:39.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3ee7fc640 1 --2- 172.21.15.55:0/3720191570 >> v2:172.21.15.138:3300/0 conn(0x7fd3f015fd60 0x7fd3f00cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.449+0000 7fd3ee7fc640 1 -- 172.21.15.55:0/3720191570 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f00b3b30 con 0x7fd3f00b35e0 2024-05-20T22:08:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.450+0000 7fd3ee7fc640 1 --2- 172.21.15.55:0/3720191570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f00b35e0 0x7fd3f015f820 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00d2e60 tx=0x7fd3d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bc7b3147896ae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.450+0000 7fd3edffb640 1 -- 172.21.15.55:0/3720191570 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8037070 con 0x7fd3f00b35e0 2024-05-20T22:08:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.450+0000 7fd3edffb640 1 -- 172.21.15.55:0/3720191570 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d8032440 con 0x7fd3f00b35e0 2024-05-20T22:08:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.450+0000 7fd3edffb640 1 -- 172.21.15.55:0/3720191570 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8032810 con 0x7fd3f00b35e0 2024-05-20T22:08:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.451+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3720191570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f00b35e0 msgr2=0x7fd3f015f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.451+0000 7fd3f5a06640 1 --2- 172.21.15.55:0/3720191570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f00b35e0 0x7fd3f015f820 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00d2e60 tx=0x7fd3d802a1e0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.451+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3720191570 shutdown_connections 2024-05-20T22:08:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.451+0000 7fd3f5a06640 1 --2- 172.21.15.55:0/3720191570 >> v2:172.21.15.138:3300/0 conn(0x7fd3f015fd60 0x7fd3f00cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.451+0000 7fd3f5a06640 1 --2- 172.21.15.55:0/3720191570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f00b35e0 0x7fd3f015f820 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.451+0000 7fd3f5a06640 1 --2- 172.21.15.55:0/3720191570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3f00b4080 0x7fd3f015f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.451+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3720191570 >> 172.21.15.55:0/3720191570 conn(0x7fd3f00d51b0 msgr2=0x7fd3f00b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:39.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.451+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3720191570 shutdown_connections 2024-05-20T22:08:39.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.451+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3720191570 wait complete. 2024-05-20T22:08:39.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.453+0000 7fd3f5a06640 1 Processor -- start 2024-05-20T22:08:39.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.453+0000 7fd3f5a06640 1 -- start start 2024-05-20T22:08:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.454+0000 7fd3f5a06640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3f00b35e0 0x7fd3f00cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.454+0000 7fd3f5a06640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3f00b4080 0x7fd3f01551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.454+0000 7fd3f5a06640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f015fd60 0x7fd3f00cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.454+0000 7fd3f5a06640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f01635f0 con 0x7fd3f015fd60 2024-05-20T22:08:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.454+0000 7fd3f5a06640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f01604f0 con 0x7fd3f00b35e0 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.454+0000 7fd3f5a06640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f00cfb60 con 0x7fd3f00b4080 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3ef7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f015fd60 0x7fd3f00cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3ef7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f015fd60 0x7fd3f00cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49452/0 (socket says 172.21.15.55:49452) 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3ef7fe640 1 -- 172.21.15.55:0/3409901782 learned_addr learned my addr 172.21.15.55:0/3409901782 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3ee7fc640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3f00b4080 0x7fd3f01551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3eeffd640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3f00b35e0 0x7fd3f00cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3ef7fe640 1 -- 172.21.15.55:0/3409901782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3f00b4080 msgr2=0x7fd3f01551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3ef7fe640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3f00b4080 0x7fd3f01551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3ef7fe640 1 -- 172.21.15.55:0/3409901782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3f00b35e0 msgr2=0x7fd3f00cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3ef7fe640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3f00b35e0 0x7fd3f00cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3ef7fe640 1 -- 172.21.15.55:0/3409901782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f00cc600 con 0x7fd3f015fd60 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3eeffd640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3f00b35e0 0x7fd3f00cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3ef7fe640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f015fd60 0x7fd3f00cdc60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0007d80 tx=0x7fd3e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:39.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3e7fff640 1 -- 172.21.15.55:0/3409901782 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0014420 con 0x7fd3f015fd60 2024-05-20T22:08:39.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3e7fff640 1 -- 172.21.15.55:0/3409901782 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e0019070 con 0x7fd3f015fd60 2024-05-20T22:08:39.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3409901782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f00b97f0 con 0x7fd3f015fd60 2024-05-20T22:08:39.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3e7fff640 1 -- 172.21.15.55:0/3409901782 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0014800 con 0x7fd3f015fd60 2024-05-20T22:08:39.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.455+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3409901782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f00b9db0 con 0x7fd3f015fd60 2024-05-20T22:08:39.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.456+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3409901782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f00b9540 con 0x7fd3f015fd60 2024-05-20T22:08:39.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.457+0000 7fd3e7fff640 1 -- 172.21.15.55:0/3409901782 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3e00149a0 con 0x7fd3f015fd60 2024-05-20T22:08:39.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.457+0000 7fd3e7fff640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd3d006be30 0x7fd3d006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.458+0000 7fd3e7fff640 1 -- 172.21.15.55:0/3409901782 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd3e008bad0 con 0x7fd3f015fd60 2024-05-20T22:08:39.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.458+0000 7fd3eeffd640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd3d006be30 0x7fd3d006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.458+0000 7fd3eeffd640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd3d006be30 0x7fd3d006e340 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8006fd0 tx=0x7fd3e8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:39.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.462+0000 7fd3e7fff640 1 -- 172.21.15.55:0/3409901782 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd3e0059ff0 con 0x7fd3f015fd60 2024-05-20T22:08:39.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.644+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3409901782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3f0069090 con 0x7fd3f015fd60 2024-05-20T22:08:39.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.644+0000 7fd3e7fff640 1 -- 172.21.15.55:0/3409901782 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd3e00598f0 con 0x7fd3f015fd60 2024-05-20T22:08:39.646 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:39.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3409901782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd3d006be30 msgr2=0x7fd3d006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd3d006be30 0x7fd3d006e340 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8006fd0 tx=0x7fd3e8008040 comp rx=0 tx=0).stop 2024-05-20T22:08:39.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3409901782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f015fd60 msgr2=0x7fd3f00cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f015fd60 0x7fd3f00cdc60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0007d80 tx=0x7fd3e000e4b0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3409901782 shutdown_connections 2024-05-20T22:08:39.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd3d006be30 0x7fd3d006e340 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3f015fd60 0x7fd3f00cdc60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3f00b4080 0x7fd3f01551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 --2- 172.21.15.55:0/3409901782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3f00b35e0 0x7fd3f00cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3409901782 >> 172.21.15.55:0/3409901782 conn(0x7fd3f00d51b0 msgr2=0x7fd3f00677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:39.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3409901782 shutdown_connections 2024-05-20T22:08:39.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.646+0000 7fd3f5a06640 1 -- 172.21.15.55:0/3409901782 wait complete. 2024-05-20T22:08:39.659 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:39.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.775+0000 7fb43b0ca640 1 Processor -- start 2024-05-20T22:08:39.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.775+0000 7fb43b0ca640 1 -- start start 2024-05-20T22:08:39.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb43b0ca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 0x7fb43415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb43b0ca640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4340d3780 con 0x7fb4340cf690 2024-05-20T22:08:39.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb43b0ca640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4340d3a60 con 0x7fb4340d01b0 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb43b0ca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4340d3d40 con 0x7fb4340d32a0 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb433fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb4340cf690 0x7fb4340cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60416/0 (socket says 172.21.15.55:60416) 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb433fff640 1 -- 172.21.15.55:0/3558223568 learned_addr learned my addr 172.21.15.55:0/3558223568 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb438e66640 1 --2- 172.21.15.55:0/3558223568 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 0x7fb43415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb4337fe640 1 -- 172.21.15.55:0/3558223568 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1204201141 0 0) 0x7fb4340d3780 con 0x7fb4340cf690 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb4337fe640 1 -- 172.21.15.55:0/3558223568 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb420003520 con 0x7fb4340cf690 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb4337fe640 1 -- 172.21.15.55:0/3558223568 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3223715508 0 0) 0x7fb4340d3a60 con 0x7fb4340d01b0 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.777+0000 7fb4337fe640 1 -- 172.21.15.55:0/3558223568 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4340d3780 con 0x7fb4340d01b0 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.778+0000 7fb438e66640 1 -- 172.21.15.55:0/3558223568 >> v1:172.21.15.138:6789/0 conn(0x7fb4340d01b0 legacy=0x7fb43406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.778+0000 7fb438e66640 1 -- 172.21.15.55:0/3558223568 >> v1:172.21.15.55:6789/0 conn(0x7fb4340cf690 legacy=0x7fb4340cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.778+0000 7fb438e66640 1 -- 172.21.15.55:0/3558223568 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4340d4020 con 0x7fb4340d32a0 2024-05-20T22:08:39.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.778+0000 7fb438e66640 1 --2- 172.21.15.55:0/3558223568 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 0x7fb43415f150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb4240097d0 tx=0x7fb42402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79ea52922643d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:39.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.779+0000 7fb4337fe640 1 -- 172.21.15.55:0/3558223568 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424037070 con 0x7fb4340d32a0 2024-05-20T22:08:39.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.779+0000 7fb4337fe640 1 -- 172.21.15.55:0/3558223568 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb424032440 con 0x7fb4340d32a0 2024-05-20T22:08:39.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.779+0000 7fb4337fe640 1 -- 172.21.15.55:0/3558223568 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4240327f0 con 0x7fb4340d32a0 2024-05-20T22:08:39.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.779+0000 7fb43b0ca640 1 -- 172.21.15.55:0/3558223568 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 msgr2=0x7fb43415f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.779+0000 7fb43b0ca640 1 --2- 172.21.15.55:0/3558223568 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 0x7fb43415f150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb4240097d0 tx=0x7fb42402a210 comp rx=0 tx=0).stop 2024-05-20T22:08:39.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.779+0000 7fb43b0ca640 1 -- 172.21.15.55:0/3558223568 shutdown_connections 2024-05-20T22:08:39.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.779+0000 7fb43b0ca640 1 --2- 172.21.15.55:0/3558223568 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 0x7fb43415f150 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.779+0000 7fb43b0ca640 1 -- 172.21.15.55:0/3558223568 >> 172.21.15.55:0/3558223568 conn(0x7fb4340b6de0 msgr2=0x7fb4340d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:39.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.779+0000 7fb43b0ca640 1 -- 172.21.15.55:0/3558223568 shutdown_connections 2024-05-20T22:08:39.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.779+0000 7fb43b0ca640 1 -- 172.21.15.55:0/3558223568 wait complete. 2024-05-20T22:08:39.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.781+0000 7fb43b0ca640 1 Processor -- start 2024-05-20T22:08:39.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.781+0000 7fb43b0ca640 1 -- start start 2024-05-20T22:08:39.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb43b0ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4340cf690 0x7fb43415f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb43b0ca640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4340d01b0 0x7fb43414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb43b0ca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 0x7fb4341e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb43b0ca640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340d4820 con 0x7fb4340d01b0 2024-05-20T22:08:39.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb43b0ca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340b4350 con 0x7fb4340cf690 2024-05-20T22:08:39.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb43b0ca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4341eb8d0 con 0x7fb4340d32a0 2024-05-20T22:08:39.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb433fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4340d01b0 0x7fb43414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb433fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4340d01b0 0x7fb43414f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49458/0 (socket says 172.21.15.55:49458) 2024-05-20T22:08:39.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb433fff640 1 -- 172.21.15.55:0/1064632521 learned_addr learned my addr 172.21.15.55:0/1064632521 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:39.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb438e66640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4340cf690 0x7fb43415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb439667640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 0x7fb4341e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb433fff640 1 -- 172.21.15.55:0/1064632521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 msgr2=0x7fb4341e96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb433fff640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 0x7fb4341e96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb433fff640 1 -- 172.21.15.55:0/1064632521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4340cf690 msgr2=0x7fb43415f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb433fff640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4340cf690 0x7fb43415f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb433fff640 1 -- 172.21.15.55:0/1064632521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4340c9d00 con 0x7fb4340d01b0 2024-05-20T22:08:39.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb433fff640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4340d01b0 0x7fb43414f800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00c440 tx=0x7fb41c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:39.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb4317fa640 1 -- 172.21.15.55:0/1064632521 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c017070 con 0x7fb4340d01b0 2024-05-20T22:08:39.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb4317fa640 1 -- 172.21.15.55:0/1064632521 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb41c005ce0 con 0x7fb4340d01b0 2024-05-20T22:08:39.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb43b0ca640 1 -- 172.21.15.55:0/1064632521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4340c9fe0 con 0x7fb4340d01b0 2024-05-20T22:08:39.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.783+0000 7fb4317fa640 1 -- 172.21.15.55:0/1064632521 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c007270 con 0x7fb4340d01b0 2024-05-20T22:08:39.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.784+0000 7fb43b0ca640 1 -- 172.21.15.55:0/1064632521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4340ca5a0 con 0x7fb4340d01b0 2024-05-20T22:08:39.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.785+0000 7fb43b0ca640 1 -- 172.21.15.55:0/1064632521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb434069090 con 0x7fb4340d01b0 2024-05-20T22:08:39.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.785+0000 7fb4317fa640 1 -- 172.21.15.55:0/1064632521 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb41c01a070 con 0x7fb4340d01b0 2024-05-20T22:08:39.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.786+0000 7fb4317fa640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 0x7fb41406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:39.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.786+0000 7fb438e66640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 0x7fb41406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:39.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.786+0000 7fb4317fa640 1 -- 172.21.15.55:0/1064632521 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb41c08a490 con 0x7fb4340d01b0 2024-05-20T22:08:39.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.787+0000 7fb438e66640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 0x7fb41406e340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb424034000 tx=0x7fb42402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:39.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.791+0000 7fb4317fa640 1 -- 172.21.15.55:0/1064632521 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb41c0583f0 con 0x7fb4340d01b0 2024-05-20T22:08:39.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.972+0000 7fb43b0ca640 1 -- 172.21.15.55:0/1064632521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4340cec80 con 0x7fb4340d01b0 2024-05-20T22:08:39.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.973+0000 7fb4317fa640 1 -- 172.21.15.55:0/1064632521 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fb41c05c6f0 con 0x7fb4340d01b0 2024-05-20T22:08:39.974 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 -- 172.21.15.55:0/1064632521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 msgr2=0x7fb41406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 0x7fb41406e340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb424034000 tx=0x7fb42402e000 comp rx=0 tx=0).stop 2024-05-20T22:08:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 -- 172.21.15.55:0/1064632521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4340d01b0 msgr2=0x7fb43414f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4340d01b0 0x7fb43414f800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00c440 tx=0x7fb41c00c900 comp rx=0 tx=0).stop 2024-05-20T22:08:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 -- 172.21.15.55:0/1064632521 shutdown_connections 2024-05-20T22:08:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 0x7fb41406e340 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4340d32a0 0x7fb4341e96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4340d01b0 0x7fb43414f800 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 --2- 172.21.15.55:0/1064632521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4340cf690 0x7fb43415f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:39.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 -- 172.21.15.55:0/1064632521 >> 172.21.15.55:0/1064632521 conn(0x7fb4340b6de0 msgr2=0x7fb43415e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:39.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 -- 172.21.15.55:0/1064632521 shutdown_connections 2024-05-20T22:08:39.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:39.975+0000 7fb43b0ca640 1 -- 172.21.15.55:0/1064632521 wait complete. 2024-05-20T22:08:39.987 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:40.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.103+0000 7f95f541b640 1 Processor -- start 2024-05-20T22:08:40.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.103+0000 7f95f541b640 1 -- start start 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.105+0000 7f95f541b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00d32a0 0x7f95f015f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.105+0000 7f95f541b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95f00d3780 con 0x7f95f00cf690 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.105+0000 7f95f541b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95f00d3a60 con 0x7f95f00d01b0 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.105+0000 7f95f541b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f95f00d3d40 con 0x7f95f00d32a0 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.105+0000 7f95ee7fc640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f95f00cf690 0x7f95f00cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60430/0 (socket says 172.21.15.55:60430) 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.105+0000 7f95ee7fc640 1 -- 172.21.15.55:0/3994987643 learned_addr learned my addr 172.21.15.55:0/3994987643 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.105+0000 7f95eeffd640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00d32a0 0x7f95f015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.105+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1524597841 0 0) 0x7f95f00d3780 con 0x7f95f00cf690 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.105+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95d8003520 con 0x7f95f00cf690 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 243518037 0 0) 0x7f95d8003520 con 0x7f95f00cf690 2024-05-20T22:08:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00d32a0 msgr2=0x7f95f015f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00d32a0 0x7f95f015f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 >> v1:172.21.15.138:6789/0 conn(0x7f95f00d01b0 legacy=0x7f95f006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f00d4020 con 0x7f95f00cf690 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f95e4001670 con 0x7f95f00cf690 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 >> v1:172.21.15.55:6789/0 conn(0x7f95f00cf690 legacy=0x7f95f00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95d80039e0 0x7f95d8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95d8004800 0x7f95d8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00d32a0 0x7f95d8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f95f00d3d40 con 0x7f95d80039e0 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95d800baf0 con 0x7f95d8004800 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f95d800bcf0 con 0x7f95f00d32a0 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95eeffd640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95d80039e0 0x7f95d8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95ee7fc640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95d8004800 0x7f95d8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95ef7fe640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00d32a0 0x7f95d8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95eeffd640 1 -- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00d32a0 msgr2=0x7f95d8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95eeffd640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00d32a0 0x7f95d8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95eeffd640 1 -- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95d8004800 msgr2=0x7f95d8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95eeffd640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95d8004800 0x7f95d8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95eeffd640 1 -- 172.21.15.55:0/3994987643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d8003520 con 0x7f95d80039e0 2024-05-20T22:08:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.106+0000 7f95ef7fe640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95f00d32a0 0x7f95d8005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.107+0000 7f95eeffd640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95d80039e0 0x7f95d8003e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009f90 tx=0x7f95d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed04a44f636ff2bd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.107+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0009c10 con 0x7f95d80039e0 2024-05-20T22:08:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.107+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e0036070 con 0x7f95d80039e0 2024-05-20T22:08:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.107+0000 7f95edffb640 1 -- 172.21.15.55:0/3994987643 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0031850 con 0x7f95d80039e0 2024-05-20T22:08:40.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.108+0000 7f95f541b640 1 -- 172.21.15.55:0/3994987643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95d80039e0 msgr2=0x7f95d8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.108+0000 7f95f541b640 1 --2- 172.21.15.55:0/3994987643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95d80039e0 0x7f95d8003e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009f90 tx=0x7f95d8005830 comp rx=0 tx=0).stop 2024-05-20T22:08:40.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.108+0000 7f95ef7fe640 1 -- 172.21.15.55:0/3994987643 reap_dead start 2024-05-20T22:08:40.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.108+0000 7f95f541b640 1 -- 172.21.15.55:0/3994987643 shutdown_connections 2024-05-20T22:08:40.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.108+0000 7f95f541b640 1 -- 172.21.15.55:0/3994987643 >> 172.21.15.55:0/3994987643 conn(0x7f95f00b6de0 msgr2=0x7f95f00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:40.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.108+0000 7f95f541b640 1 -- 172.21.15.55:0/3994987643 shutdown_connections 2024-05-20T22:08:40.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.108+0000 7f95f541b640 1 -- 172.21.15.55:0/3994987643 wait complete. 2024-05-20T22:08:40.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.110+0000 7f95f541b640 1 Processor -- start 2024-05-20T22:08:40.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.110+0000 7f95f541b640 1 -- start start 2024-05-20T22:08:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95f541b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95d80039e0 0x7f95f01f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95f541b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f01f4cd0 0x7f95f00b9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95f541b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f00b9cc0 0x7f95f01f8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95f541b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f95f00d4820 con 0x7f95f01f4cd0 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95f541b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95f00b4350 con 0x7f95f00b9cc0 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95f541b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f95f00cd2d0 con 0x7f95d80039e0 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f01f4cd0 0x7f95f00b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f01f4cd0 0x7f95f00b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49484/0 (socket says 172.21.15.55:49484) 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95ee7fc640 1 -- 172.21.15.55:0/2382308424 learned_addr learned my addr 172.21.15.55:0/2382308424 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95eeffd640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95d80039e0 0x7f95f01f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95ef7fe640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f00b9cc0 0x7f95f01f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95ee7fc640 1 -- 172.21.15.55:0/2382308424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95d80039e0 msgr2=0x7f95f01f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.111+0000 7f95ee7fc640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95d80039e0 0x7f95f01f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.112+0000 7f95ee7fc640 1 -- 172.21.15.55:0/2382308424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f00b9cc0 msgr2=0x7f95f01f8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.112+0000 7f95ee7fc640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f00b9cc0 0x7f95f01f8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.112+0000 7f95ee7fc640 1 -- 172.21.15.55:0/2382308424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f00ba0d0 con 0x7f95f01f4cd0 2024-05-20T22:08:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.112+0000 7f95eeffd640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95d80039e0 0x7f95f01f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.112+0000 7f95ee7fc640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f01f4cd0 0x7f95f00b9780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f95e4007310 tx=0x7f95e40077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.112+0000 7f95d77fe640 1 -- 172.21.15.55:0/2382308424 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e400f960 con 0x7f95f01f4cd0 2024-05-20T22:08:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.112+0000 7f95d77fe640 1 -- 172.21.15.55:0/2382308424 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e4018070 con 0x7f95f01f4cd0 2024-05-20T22:08:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.112+0000 7f95f541b640 1 -- 172.21.15.55:0/2382308424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f01e9b20 con 0x7f95f01f4cd0 2024-05-20T22:08:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.112+0000 7f95d77fe640 1 -- 172.21.15.55:0/2382308424 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e400fd60 con 0x7f95f01f4cd0 2024-05-20T22:08:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.112+0000 7f95f541b640 1 -- 172.21.15.55:0/2382308424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f01ea0c0 con 0x7f95f01f4cd0 2024-05-20T22:08:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.113+0000 7f95f541b640 1 -- 172.21.15.55:0/2382308424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f0069090 con 0x7f95f01f4cd0 2024-05-20T22:08:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.114+0000 7f95d77fe640 1 -- 172.21.15.55:0/2382308424 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95e401d460 con 0x7f95f01f4cd0 2024-05-20T22:08:40.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.114+0000 7f95d77fe640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95dc06be30 0x7f95dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.114+0000 7f95eeffd640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95dc06be30 0x7f95dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.114+0000 7f95d77fe640 1 -- 172.21.15.55:0/2382308424 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f95e408b3c0 con 0x7f95f01f4cd0 2024-05-20T22:08:40.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.115+0000 7f95eeffd640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95dc06be30 0x7f95dc06e340 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95e0035000 tx=0x7f95e002f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:40.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.119+0000 7f95d77fe640 1 -- 172.21.15.55:0/2382308424 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f95f00ba0d0 con 0x7f95f01f4cd0 2024-05-20T22:08:40.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.303+0000 7f95f541b640 1 -- 172.21.15.55:0/2382308424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f95f00cbce0 con 0x7f95f01f4cd0 2024-05-20T22:08:40.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.303+0000 7f95d77fe640 1 -- 172.21.15.55:0/2382308424 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f95e40598e0 con 0x7f95f01f4cd0 2024-05-20T22:08:40.304 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.305+0000 7f95f541b640 1 -- 172.21.15.55:0/2382308424 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95dc06be30 msgr2=0x7f95dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.305+0000 7f95f541b640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95dc06be30 0x7f95dc06e340 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95e0035000 tx=0x7f95e002f000 comp rx=0 tx=0).stop 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.305+0000 7f95f541b640 1 -- 172.21.15.55:0/2382308424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f01f4cd0 msgr2=0x7f95f00b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.305+0000 7f95f541b640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f01f4cd0 0x7f95f00b9780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f95e4007310 tx=0x7f95e40077d0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.305+0000 7f95f541b640 1 -- 172.21.15.55:0/2382308424 shutdown_connections 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.305+0000 7f95f541b640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f95dc06be30 0x7f95dc06e340 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.305+0000 7f95f541b640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95f00b9cc0 0x7f95f01f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.305+0000 7f95f541b640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95f01f4cd0 0x7f95f00b9780 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.305+0000 7f95f541b640 1 --2- 172.21.15.55:0/2382308424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95d80039e0 0x7f95f01f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.306+0000 7f95f541b640 1 -- 172.21.15.55:0/2382308424 >> 172.21.15.55:0/2382308424 conn(0x7f95f00b6de0 msgr2=0x7f95f006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.306+0000 7f95f541b640 1 -- 172.21.15.55:0/2382308424 shutdown_connections 2024-05-20T22:08:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.306+0000 7f95f541b640 1 -- 172.21.15.55:0/2382308424 wait complete. 2024-05-20T22:08:40.318 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:40.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.435+0000 7f5cc25bc640 1 Processor -- start 2024-05-20T22:08:40.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.435+0000 7f5cc25bc640 1 -- start start 2024-05-20T22:08:40.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.436+0000 7f5cc25bc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5cbc0d32a0 0x7f5cbc15f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.436+0000 7f5cc25bc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc0d01b0 0x7f5cbc06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.436+0000 7f5cc25bc640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cbc0d3780 con 0x7f5cbc0cf690 2024-05-20T22:08:40.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.436+0000 7f5cc25bc640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f5cbc0d3a60 con 0x7f5cbc0d32a0 2024-05-20T22:08:40.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.436+0000 7f5cc25bc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5cbc0d3d40 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.437+0000 7f5cbbfff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5cbc0d32a0 0x7f5cbc15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.437+0000 7f5cbb7fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5cbc0cf690 0x7f5cbc0cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60446/0 (socket says 172.21.15.55:60446) 2024-05-20T22:08:40.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.437+0000 7f5cbb7fe640 1 -- 172.21.15.55:0/3218804581 learned_addr learned my addr 172.21.15.55:0/3218804581 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:40.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.437+0000 7f5cc0b59640 1 --2- 172.21.15.55:0/3218804581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc0d01b0 0x7f5cbc06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.437+0000 7f5cbbfff640 1 -- 172.21.15.55:0/3218804581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc0d01b0 msgr2=0x7f5cbc06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:40.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.437+0000 7f5cbbfff640 1 --2- 172.21.15.55:0/3218804581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc0d01b0 0x7f5cbc06b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.437+0000 7f5cbbfff640 1 -- 172.21.15.55:0/3218804581 >> v1:172.21.15.55:6789/0 conn(0x7f5cbc0cf690 legacy=0x7f5cbc0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.437+0000 7f5cbaffd640 1 -- 172.21.15.55:0/3218804581 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2929621227 0 0) 0x7f5cbc0d3780 con 0x7f5cbc0cf690 2024-05-20T22:08:40.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.437+0000 7f5cbbfff640 1 -- 172.21.15.55:0/3218804581 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cbc0d4020 con 0x7f5cbc0d32a0 2024-05-20T22:08:40.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.438+0000 7f5cbbfff640 1 --2- 172.21.15.55:0/3218804581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d32a0 0x7f5cbc15f150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0097a0 tx=0x7f5cac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c19da0dfa40a27 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:40.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.438+0000 7f5cbaffd640 1 -- 172.21.15.55:0/3218804581 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac037070 con 0x7f5cbc0d32a0 2024-05-20T22:08:40.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.438+0000 7f5cbaffd640 1 -- 172.21.15.55:0/3218804581 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac032440 con 0x7f5cbc0d32a0 2024-05-20T22:08:40.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.438+0000 7f5cbaffd640 1 -- 172.21.15.55:0/3218804581 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac032810 con 0x7f5cbc0d32a0 2024-05-20T22:08:40.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.439+0000 7f5cc25bc640 1 -- 172.21.15.55:0/3218804581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d32a0 msgr2=0x7f5cbc15f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.439+0000 7f5cc25bc640 1 --2- 172.21.15.55:0/3218804581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d32a0 0x7f5cbc15f150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0097a0 tx=0x7f5cac02a180 comp rx=0 tx=0).stop 2024-05-20T22:08:40.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.439+0000 7f5cc25bc640 1 -- 172.21.15.55:0/3218804581 shutdown_connections 2024-05-20T22:08:40.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.439+0000 7f5cc25bc640 1 --2- 172.21.15.55:0/3218804581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc0d01b0 0x7f5cbc06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.439+0000 7f5cc25bc640 1 --2- 172.21.15.55:0/3218804581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d32a0 0x7f5cbc15f150 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.439+0000 7f5cc25bc640 1 -- 172.21.15.55:0/3218804581 >> 172.21.15.55:0/3218804581 conn(0x7f5cbc0b6de0 msgr2=0x7f5cbc0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:40.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.439+0000 7f5cc25bc640 1 -- 172.21.15.55:0/3218804581 shutdown_connections 2024-05-20T22:08:40.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.439+0000 7f5cc25bc640 1 -- 172.21.15.55:0/3218804581 wait complete. 2024-05-20T22:08:40.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.440+0000 7f5cc25bc640 1 Processor -- start 2024-05-20T22:08:40.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.441+0000 7f5cc25bc640 1 -- start start 2024-05-20T22:08:40.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cc25bc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc0cf690 0x7f5cbc0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cc25bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d01b0 0x7f5cbc150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cc25bc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc0d32a0 0x7f5cbc0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cc25bc640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc0d4820 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cc25bc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc0b4350 con 0x7f5cbc0cf690 2024-05-20T22:08:40.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cc25bc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5cbc0c9e80 con 0x7f5cbc0d32a0 2024-05-20T22:08:40.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d01b0 0x7f5cbc150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d01b0 0x7f5cbc150160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49512/0 (socket says 172.21.15.55:49512) 2024-05-20T22:08:40.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cbb7fe640 1 -- 172.21.15.55:0/2668329962 learned_addr learned my addr 172.21.15.55:0/2668329962 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cbbfff640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc0cf690 0x7f5cbc0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cc0b59640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc0d32a0 0x7f5cbc0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cbb7fe640 1 -- 172.21.15.55:0/2668329962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc0d32a0 msgr2=0x7f5cbc0cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cbb7fe640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc0d32a0 0x7f5cbc0cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cbb7fe640 1 -- 172.21.15.55:0/2668329962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc0cf690 msgr2=0x7f5cbc0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cbb7fe640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc0cf690 0x7f5cbc0cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.442+0000 7f5cbb7fe640 1 -- 172.21.15.55:0/2668329962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cbc0cc1f0 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.443+0000 7f5cbb7fe640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d01b0 0x7f5cbc150160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000c430 tx=0x7f5cb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.443+0000 7f5cb8ff9640 1 -- 172.21.15.55:0/2668329962 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00074e0 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.443+0000 7f5cb8ff9640 1 -- 172.21.15.55:0/2668329962 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb00047e0 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.443+0000 7f5cc25bc640 1 -- 172.21.15.55:0/2668329962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cbc0cc4d0 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.443+0000 7f5cb8ff9640 1 -- 172.21.15.55:0/2668329962 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0004bd0 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.443+0000 7f5cc25bc640 1 -- 172.21.15.55:0/2668329962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cbc0cca90 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.444+0000 7f5cc25bc640 1 -- 172.21.15.55:0/2668329962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cbc069090 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.444+0000 7f5cb8ff9640 1 -- 172.21.15.55:0/2668329962 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5cb0011070 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.445+0000 7f5cb8ff9640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ca806be30 0x7f5ca806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.445+0000 7f5cb8ff9640 1 -- 172.21.15.55:0/2668329962 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5cb008ae60 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.445+0000 7f5cbbfff640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ca806be30 0x7f5ca806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.445+0000 7f5cbbfff640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ca806be30 0x7f5ca806e340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5cac034000 tx=0x7f5cac02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:40.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.449+0000 7f5cb8ff9640 1 -- 172.21.15.55:0/2668329962 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5cb00594a0 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.631+0000 7f5cc25bc640 1 -- 172.21.15.55:0/2668329962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5cbc0cbf80 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.631+0000 7f5cb8ff9640 1 -- 172.21.15.55:0/2668329962 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f5cb0058da0 con 0x7f5cbc0d01b0 2024-05-20T22:08:40.632 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:40.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 -- 172.21.15.55:0/2668329962 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ca806be30 msgr2=0x7f5ca806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ca806be30 0x7f5ca806e340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5cac034000 tx=0x7f5cac02e000 comp rx=0 tx=0).stop 2024-05-20T22:08:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 -- 172.21.15.55:0/2668329962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d01b0 msgr2=0x7f5cbc150160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d01b0 0x7f5cbc150160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000c430 tx=0x7f5cb000c8f0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 -- 172.21.15.55:0/2668329962 shutdown_connections 2024-05-20T22:08:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ca806be30 0x7f5ca806e340 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc0d32a0 0x7f5cbc0cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0d01b0 0x7f5cbc150160 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 --2- 172.21.15.55:0/2668329962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc0cf690 0x7f5cbc0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 -- 172.21.15.55:0/2668329962 >> 172.21.15.55:0/2668329962 conn(0x7f5cbc0b6de0 msgr2=0x7f5cbc15ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 -- 172.21.15.55:0/2668329962 shutdown_connections 2024-05-20T22:08:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.633+0000 7f5cc25bc640 1 -- 172.21.15.55:0/2668329962 wait complete. 2024-05-20T22:08:40.646 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj' 2024-05-20T22:08:40.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.761+0000 7f77052de640 1 Processor -- start 2024-05-20T22:08:40.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.761+0000 7f77052de640 1 -- start start 2024-05-20T22:08:40.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.762+0000 7f77052de640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77000b3b30 0x7f77000b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f77052de640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77000d3dd0 con 0x7f770015f100 2024-05-20T22:08:40.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f77052de640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77000d40b0 con 0x7f770006c790 2024-05-20T22:08:40.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f77052de640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f77000d4390 con 0x7f77000b3b30 2024-05-20T22:08:40.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fe7fc640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f770015f100 0x7f770006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60448/0 (socket says 172.21.15.55:60448) 2024-05-20T22:08:40.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fe7fc640 1 -- 172.21.15.55:0/1320852126 learned_addr learned my addr 172.21.15.55:0/1320852126 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:40.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76feffd640 1 --2- 172.21.15.55:0/1320852126 >> v2:172.21.15.138:3300/0 conn(0x7f77000b3b30 0x7f77000b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3984684528 0 0) 0x7f77000d3dd0 con 0x7f770015f100 2024-05-20T22:08:40.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76e8003520 con 0x7f770015f100 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4269665139 0 0) 0x7f76e8003520 con 0x7f770015f100 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 >> v2:172.21.15.138:3300/0 conn(0x7f77000b3b30 msgr2=0x7f77000b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 --2- 172.21.15.55:0/1320852126 >> v2:172.21.15.138:3300/0 conn(0x7f77000b3b30 0x7f77000b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 >> v1:172.21.15.138:6789/0 conn(0x7f770006c790 legacy=0x7f77000cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770006b5a0 con 0x7f770015f100 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f76f4001670 con 0x7f770015f100 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 >> v1:172.21.15.55:6789/0 conn(0x7f770015f100 legacy=0x7f770006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76e80039b0 0x7f76e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e8004800 0x7f76e8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76e80051c0 0x7f76e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f77000d4390 con 0x7f76e8004800 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76e800c160 con 0x7f76e80051c0 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.763+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f76e800c360 con 0x7f76e80039b0 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.764+0000 7f76fe7fc640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e8004800 0x7f76e8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.764+0000 7f76feffd640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76e80039b0 0x7f76e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.764+0000 7f76ff7fe640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76e80051c0 0x7f76e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.764+0000 7f76fe7fc640 1 -- 172.21.15.55:0/1320852126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76e80039b0 msgr2=0x7f76e8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.764+0000 7f76fe7fc640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76e80039b0 0x7f76e8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.764+0000 7f76fe7fc640 1 -- 172.21.15.55:0/1320852126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76e80051c0 msgr2=0x7f76e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.764+0000 7f76fe7fc640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76e80051c0 0x7f76e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.764+0000 7f76ff7fe640 1 -- 172.21.15.55:0/1320852126 reap_dead start 2024-05-20T22:08:40.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.764+0000 7f76fe7fc640 1 -- 172.21.15.55:0/1320852126 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e8003520 con 0x7f76e8004800 2024-05-20T22:08:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.765+0000 7f76fe7fc640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e8004800 0x7f76e8004c10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f77000c8380 tx=0x7f76e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e820bef8db341d92 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.765+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4007d10 con 0x7f76e8004800 2024-05-20T22:08:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.765+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f4005ad0 con 0x7f76e8004800 2024-05-20T22:08:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.765+0000 7f76fdffb640 1 -- 172.21.15.55:0/1320852126 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4007070 con 0x7f76e8004800 2024-05-20T22:08:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.765+0000 7f77052de640 1 -- 172.21.15.55:0/1320852126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e8004800 msgr2=0x7f76e8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.765+0000 7f77052de640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e8004800 0x7f76e8004c10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f77000c8380 tx=0x7f76e8005830 comp rx=0 tx=0).stop 2024-05-20T22:08:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.766+0000 7f77052de640 1 -- 172.21.15.55:0/1320852126 shutdown_connections 2024-05-20T22:08:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.766+0000 7f77052de640 1 --2- 172.21.15.55:0/1320852126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76e8004800 0x7f76e8004c10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.766+0000 7f77052de640 1 -- 172.21.15.55:0/1320852126 >> 172.21.15.55:0/1320852126 conn(0x7f77000d51d0 msgr2=0x7f77000b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:40.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.766+0000 7f77052de640 1 -- 172.21.15.55:0/1320852126 shutdown_connections 2024-05-20T22:08:40.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.766+0000 7f77052de640 1 -- 172.21.15.55:0/1320852126 wait complete. 2024-05-20T22:08:40.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.768+0000 7f77052de640 1 Processor -- start 2024-05-20T22:08:40.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.768+0000 7f77052de640 1 -- start start 2024-05-20T22:08:40.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f77052de640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76e8004800 0x7f77001f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f77052de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7700160f70 0x7f77001574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f77052de640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7700161380 0x7f7700157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f77052de640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f770006cba0 con 0x7f7700161380 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f77052de640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f770006bb60 con 0x7f7700160f70 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f77052de640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f770006cd20 con 0x7f76e8004800 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f76ff7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7700161380 0x7f7700157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f76ff7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7700161380 0x7f7700157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49528/0 (socket says 172.21.15.55:49528) 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f76ff7fe640 1 -- 172.21.15.55:0/3039452662 learned_addr learned my addr 172.21.15.55:0/3039452662 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f76feffd640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76e8004800 0x7f77001f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f76fe7fc640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7700160f70 0x7f77001574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f76ff7fe640 1 -- 172.21.15.55:0/3039452662 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76e8004800 msgr2=0x7f77001f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f76ff7fe640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76e8004800 0x7f77001f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f76ff7fe640 1 -- 172.21.15.55:0/3039452662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7700160f70 msgr2=0x7f77001574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f76ff7fe640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7700160f70 0x7f77001574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.769+0000 7f76ff7fe640 1 -- 172.21.15.55:0/3039452662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7700161b80 con 0x7f7700161380 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.770+0000 7f76ff7fe640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7700161380 0x7f7700157a00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f77000d09a0 tx=0x7f76f800f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.770+0000 7f76e77fe640 1 -- 172.21.15.55:0/3039452662 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f801a070 con 0x7f7700161380 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.770+0000 7f76e77fe640 1 -- 172.21.15.55:0/3039452662 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f8005340 con 0x7f7700161380 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.770+0000 7f77052de640 1 -- 172.21.15.55:0/3039452662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7700161e60 con 0x7f7700161380 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.770+0000 7f76e77fe640 1 -- 172.21.15.55:0/3039452662 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8007250 con 0x7f7700161380 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.770+0000 7f77052de640 1 -- 172.21.15.55:0/3039452662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7700154930 con 0x7f7700161380 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.771+0000 7f77052de640 1 -- 172.21.15.55:0/3039452662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77001618f0 con 0x7f7700161380 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.772+0000 7f76e77fe640 1 -- 172.21.15.55:0/3039452662 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76f801d070 con 0x7f7700161380 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.772+0000 7f76e77fe640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76ec06be30 0x7f76ec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.772+0000 7f76e77fe640 1 -- 172.21.15.55:0/3039452662 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f76f808a400 con 0x7f7700161380 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.772+0000 7f76feffd640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76ec06be30 0x7f76ec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:40.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.772+0000 7f76feffd640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76ec06be30 0x7f76ec06e340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f770006d0c0 tx=0x7f76f000b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:40.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:40.777+0000 7f76e77fe640 1 -- 172.21.15.55:0/3039452662 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f76f80589b0 con 0x7f7700161380 2024-05-20T22:08:41.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.031+0000 7f77052de640 1 -- 172.21.15.55:0/3039452662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]} v 0) -- 0x7f7700069090 con 0x7f7700161380 2024-05-20T22:08:41.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.945+0000 7f76e77fe640 1 -- 172.21.15.55:0/3039452662 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]}]=0 v68) ==== 150+0+0 (secure 0 0 0) 0x7f76f80582b0 con 0x7f7700161380 2024-05-20T22:08:41.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 -- 172.21.15.55:0/3039452662 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76ec06be30 msgr2=0x7f76ec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:41.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76ec06be30 0x7f76ec06e340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f770006d0c0 tx=0x7f76f000b040 comp rx=0 tx=0).stop 2024-05-20T22:08:41.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 -- 172.21.15.55:0/3039452662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7700161380 msgr2=0x7f7700157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:41.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7700161380 0x7f7700157a00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f77000d09a0 tx=0x7f76f800f900 comp rx=0 tx=0).stop 2024-05-20T22:08:41.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 -- 172.21.15.55:0/3039452662 shutdown_connections 2024-05-20T22:08:41.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76ec06be30 0x7f76ec06e340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:41.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7700161380 0x7f7700157a00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:41.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7700160f70 0x7f77001574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:41.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 --2- 172.21.15.55:0/3039452662 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76e8004800 0x7f77001f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:41.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 -- 172.21.15.55:0/3039452662 >> 172.21.15.55:0/3039452662 conn(0x7f77000d51d0 msgr2=0x7f7700162b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:41.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 -- 172.21.15.55:0/3039452662 shutdown_connections 2024-05-20T22:08:41.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:41.948+0000 7f77052de640 1 -- 172.21.15.55:0/3039452662 wait complete. 2024-05-20T22:08:41.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-20T22:08:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.088+0000 7fe6606ca640 1 Processor -- start 2024-05-20T22:08:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.088+0000 7fe6606ca640 1 -- start start 2024-05-20T22:08:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe6606ca640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe65815f150 0x7fe6580cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe6606ca640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe65806b500 0x7fe65806b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe6606ca640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6580d3780 con 0x7fe6580d32a0 2024-05-20T22:08:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe6606ca640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fe6580d3a60 con 0x7fe65815f150 2024-05-20T22:08:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe6606ca640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe6580d3d40 con 0x7fe65806b500 2024-05-20T22:08:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe65dc65640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe65815f150 0x7fe6580cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe65dc65640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe65815f150 0x7fe6580cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49532/0 (socket says 172.21.15.55:49532) 2024-05-20T22:08:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe65dc65640 1 -- 172.21.15.55:0/3743359716 learned_addr learned my addr 172.21.15.55:0/3743359716 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe65ec67640 1 --2- 172.21.15.55:0/3743359716 >> v2:172.21.15.138:3300/0 conn(0x7fe65806b500 0x7fe65806b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe65dc65640 1 -- 172.21.15.55:0/3743359716 >> v2:172.21.15.138:3300/0 conn(0x7fe65806b500 msgr2=0x7fe65806b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe65dc65640 1 --2- 172.21.15.55:0/3743359716 >> v2:172.21.15.138:3300/0 conn(0x7fe65806b500 0x7fe65806b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe65dc65640 1 -- 172.21.15.55:0/3743359716 >> v1:172.21.15.138:6789/0 conn(0x7fe6580d32a0 legacy=0x7fe6580cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe65dc65640 1 -- 172.21.15.55:0/3743359716 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6580d4020 con 0x7fe65815f150 2024-05-20T22:08:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.090+0000 7fe65d464640 1 -- 172.21.15.55:0/3743359716 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414008723 0 0) 0x7fe6580d3780 con 0x7fe6580d32a0 2024-05-20T22:08:42.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.091+0000 7fe65dc65640 1 --2- 172.21.15.55:0/3743359716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 0x7fe6580cfda0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe658163d70 tx=0x7fe64802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565348a0923e7f0a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:42.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.091+0000 7fe65d464640 1 -- 172.21.15.55:0/3743359716 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648037070 con 0x7fe65815f150 2024-05-20T22:08:42.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.091+0000 7fe65d464640 1 -- 172.21.15.55:0/3743359716 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe648032440 con 0x7fe65815f150 2024-05-20T22:08:42.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.091+0000 7fe65d464640 1 -- 172.21.15.55:0/3743359716 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648032810 con 0x7fe65815f150 2024-05-20T22:08:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.092+0000 7fe6606ca640 1 -- 172.21.15.55:0/3743359716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 msgr2=0x7fe6580cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.092+0000 7fe6606ca640 1 --2- 172.21.15.55:0/3743359716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 0x7fe6580cfda0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe658163d70 tx=0x7fe64802a270 comp rx=0 tx=0).stop 2024-05-20T22:08:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.092+0000 7fe6606ca640 1 -- 172.21.15.55:0/3743359716 shutdown_connections 2024-05-20T22:08:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.092+0000 7fe6606ca640 1 --2- 172.21.15.55:0/3743359716 >> v2:172.21.15.138:3300/0 conn(0x7fe65806b500 0x7fe65806b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.092+0000 7fe6606ca640 1 --2- 172.21.15.55:0/3743359716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 0x7fe6580cfda0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.092+0000 7fe6606ca640 1 -- 172.21.15.55:0/3743359716 >> 172.21.15.55:0/3743359716 conn(0x7fe6580b6de0 msgr2=0x7fe6580d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.092+0000 7fe6606ca640 1 -- 172.21.15.55:0/3743359716 shutdown_connections 2024-05-20T22:08:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.092+0000 7fe6606ca640 1 -- 172.21.15.55:0/3743359716 wait complete. 2024-05-20T22:08:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.094+0000 7fe6606ca640 1 Processor -- start 2024-05-20T22:08:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.094+0000 7fe6606ca640 1 -- start start 2024-05-20T22:08:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe6606ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65806b500 0x7fe658154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe6606ca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6580d32a0 0x7fe65815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe6606ca640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 0x7fe6580b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe6606ca640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe6580d4820 con 0x7fe65815f150 2024-05-20T22:08:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe6606ca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe65815f560 con 0x7fe65806b500 2024-05-20T22:08:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe6606ca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6580cce70 con 0x7fe6580d32a0 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe65ec67640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 0x7fe6580b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe65ec67640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 0x7fe6580b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49548/0 (socket says 172.21.15.55:49548) 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe65ec67640 1 -- 172.21.15.55:0/71451328 learned_addr learned my addr 172.21.15.55:0/71451328 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe65dc65640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6580d32a0 0x7fe65815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe65e466640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65806b500 0x7fe658154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe65ec67640 1 -- 172.21.15.55:0/71451328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6580d32a0 msgr2=0x7fe65815b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe65ec67640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6580d32a0 0x7fe65815b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe65ec67640 1 -- 172.21.15.55:0/71451328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65806b500 msgr2=0x7fe658154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe65ec67640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65806b500 0x7fe658154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.095+0000 7fe65ec67640 1 -- 172.21.15.55:0/71451328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6581f8950 con 0x7fe65815f150 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.096+0000 7fe65ec67640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 0x7fe6580b52e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe65000efc0 tx=0x7fe65000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.096+0000 7fe64effd640 1 -- 172.21.15.55:0/71451328 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650018020 con 0x7fe65815f150 2024-05-20T22:08:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.096+0000 7fe64effd640 1 -- 172.21.15.55:0/71451328 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe650014480 con 0x7fe65815f150 2024-05-20T22:08:42.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.096+0000 7fe6606ca640 1 -- 172.21.15.55:0/71451328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6581f8c30 con 0x7fe65815f150 2024-05-20T22:08:42.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.096+0000 7fe64effd640 1 -- 172.21.15.55:0/71451328 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650009e60 con 0x7fe65815f150 2024-05-20T22:08:42.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.096+0000 7fe6606ca640 1 -- 172.21.15.55:0/71451328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6581f91f0 con 0x7fe65815f150 2024-05-20T22:08:42.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.097+0000 7fe6606ca640 1 -- 172.21.15.55:0/71451328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe658069090 con 0x7fe65815f150 2024-05-20T22:08:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.097+0000 7fe64effd640 1 -- 172.21.15.55:0/71451328 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe650005d10 con 0x7fe65815f150 2024-05-20T22:08:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.098+0000 7fe64effd640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe63c06be30 0x7fe63c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.098+0000 7fe64effd640 1 -- 172.21.15.55:0/71451328 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe65008bf50 con 0x7fe65815f150 2024-05-20T22:08:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.098+0000 7fe65e466640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe63c06be30 0x7fe63c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.098+0000 7fe65e466640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe63c06be30 0x7fe63c06e340 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe654007fd0 tx=0x7fe654009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.102+0000 7fe64effd640 1 -- 172.21.15.55:0/71451328 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe650022030 con 0x7fe65815f150 2024-05-20T22:08:42.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.286+0000 7fe6606ca640 1 -- 172.21.15.55:0/71451328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe6580d2e50 con 0x7fe65815f150 2024-05-20T22:08:42.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.290+0000 7fe64effd640 1 -- 172.21.15.55:0/71451328 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fe650010040 con 0x7fe65815f150 2024-05-20T22:08:42.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 -- 172.21.15.55:0/71451328 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe63c06be30 msgr2=0x7fe63c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe63c06be30 0x7fe63c06e340 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe654007fd0 tx=0x7fe654009040 comp rx=0 tx=0).stop 2024-05-20T22:08:42.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 -- 172.21.15.55:0/71451328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 msgr2=0x7fe6580b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 0x7fe6580b52e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe65000efc0 tx=0x7fe65000c490 comp rx=0 tx=0).stop 2024-05-20T22:08:42.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 -- 172.21.15.55:0/71451328 shutdown_connections 2024-05-20T22:08:42.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe63c06be30 0x7fe63c06e340 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe65815f150 0x7fe6580b52e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6580d32a0 0x7fe65815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 --2- 172.21.15.55:0/71451328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65806b500 0x7fe658154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 -- 172.21.15.55:0/71451328 >> 172.21.15.55:0/71451328 conn(0x7fe6580b6de0 msgr2=0x7fe65815e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:42.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 -- 172.21.15.55:0/71451328 shutdown_connections 2024-05-20T22:08:42.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.292+0000 7fe6606ca640 1 -- 172.21.15.55:0/71451328 wait complete. 2024-05-20T22:08:42.305 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T22:08:42.305 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T22:08:42.346 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T22:08:42.391 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T22:08:42.392 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:08:42.392 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:42.409 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:08:42.409 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:42.465 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:42.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.618+0000 7f705d938640 1 Processor -- start 2024-05-20T22:08:42.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.618+0000 7f705d938640 1 -- start start 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f705d938640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70580b7fe0 0x7f7058169f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f705d938640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70580b83f0 con 0x7f705815f100 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f705d938640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70580d3dd0 con 0x7f70580b3b30 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f705d938640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f705806cea0 con 0x7f70580b7fe0 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f70567fc640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f705815f100 0x7f705806c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60462/0 (socket says 172.21.15.55:60462) 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f70567fc640 1 -- 172.21.15.55:0/3519533317 learned_addr learned my addr 172.21.15.55:0/3519533317 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f70577fe640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70580b7fe0 0x7f7058169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164351046 0 0) 0x7f70580b83f0 con 0x7f705815f100 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7044003520 con 0x7f705815f100 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 327995412 0 0) 0x7f70580d3dd0 con 0x7f70580b3b30 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.620+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70580b83f0 con 0x7f70580b3b30 2024-05-20T22:08:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3850788843 0 0) 0x7f7044003520 con 0x7f705815f100 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70580b7fe0 msgr2=0x7f7058169f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70580b7fe0 0x7f7058169f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 >> v1:172.21.15.138:6789/0 conn(0x7f70580b3b30 legacy=0x7f705806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70580d40b0 con 0x7f705815f100 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f7040001690 con 0x7f705815f100 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 >> v1:172.21.15.55:6789/0 conn(0x7f705815f100 legacy=0x7f705806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7044003b70 0x7f7044003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7044004800 0x7f7044004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70580b7fe0 0x7f7044005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f705806cea0 con 0x7f7044003b70 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f704400bad0 con 0x7f7044004800 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f704400bcd0 con 0x7f70580b7fe0 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7056ffd640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7044003b70 0x7f7044003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f70577fe640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70580b7fe0 0x7f7044005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f70567fc640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7044004800 0x7f7044004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7056ffd640 1 -- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70580b7fe0 msgr2=0x7f7044005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7056ffd640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70580b7fe0 0x7f7044005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7056ffd640 1 -- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7044004800 msgr2=0x7f7044004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7056ffd640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7044004800 0x7f7044004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.621+0000 7f7056ffd640 1 -- 172.21.15.55:0/3519533317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7044003520 con 0x7f7044003b70 2024-05-20T22:08:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.622+0000 7f7056ffd640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7044003b70 0x7f7044003fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7048001fa0 tx=0x7f7044005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef5415a53578b40 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.622+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048001bf0 con 0x7f7044003b70 2024-05-20T22:08:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.622+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7048003b70 con 0x7f7044003b70 2024-05-20T22:08:42.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.622+0000 7f7055ffb640 1 -- 172.21.15.55:0/3519533317 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70480050d0 con 0x7f7044003b70 2024-05-20T22:08:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.623+0000 7f705d938640 1 -- 172.21.15.55:0/3519533317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7044003b70 msgr2=0x7f7044003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.623+0000 7f705d938640 1 --2- 172.21.15.55:0/3519533317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7044003b70 0x7f7044003fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7048001fa0 tx=0x7f7044005830 comp rx=0 tx=0).stop 2024-05-20T22:08:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.623+0000 7f70577fe640 1 -- 172.21.15.55:0/3519533317 reap_dead start 2024-05-20T22:08:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.623+0000 7f705d938640 1 -- 172.21.15.55:0/3519533317 shutdown_connections 2024-05-20T22:08:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.623+0000 7f705d938640 1 -- 172.21.15.55:0/3519533317 >> 172.21.15.55:0/3519533317 conn(0x7f70580d51d0 msgr2=0x7f70580b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.623+0000 7f705d938640 1 -- 172.21.15.55:0/3519533317 shutdown_connections 2024-05-20T22:08:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.623+0000 7f705d938640 1 -- 172.21.15.55:0/3519533317 wait complete. 2024-05-20T22:08:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.625+0000 7f705d938640 1 Processor -- start 2024-05-20T22:08:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.625+0000 7f705d938640 1 -- start start 2024-05-20T22:08:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.626+0000 7f705d938640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7044003b70 0x7f705815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.626+0000 7f705d938640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7058067490 0x7f7058159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.626+0000 7f705d938640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705815a200 0x7f7058154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.626+0000 7f705d938640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f705806b880 con 0x7f705815a200 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.626+0000 7f705d938640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f705806b5a0 con 0x7f7058067490 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.626+0000 7f705d938640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f705806bb60 con 0x7f7044003b70 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.626+0000 7f70577fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705815a200 0x7f7058154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.626+0000 7f70577fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705815a200 0x7f7058154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49566/0 (socket says 172.21.15.55:49566) 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.626+0000 7f70577fe640 1 -- 172.21.15.55:0/1982334510 learned_addr learned my addr 172.21.15.55:0/1982334510 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f7056ffd640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7044003b70 0x7f705815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f70567fc640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7058067490 0x7f7058159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f70577fe640 1 -- 172.21.15.55:0/1982334510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7044003b70 msgr2=0x7f705815c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f70577fe640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7044003b70 0x7f705815c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f70577fe640 1 -- 172.21.15.55:0/1982334510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7058067490 msgr2=0x7f7058159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f70577fe640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7058067490 0x7f7058159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f70577fe640 1 -- 172.21.15.55:0/1982334510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7058158080 con 0x7f705815a200 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f70567fc640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7058067490 0x7f7058159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f70577fe640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705815a200 0x7f7058154a10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7044004780 tx=0x7f7050011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f704ffff640 1 -- 172.21.15.55:0/1982334510 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050004240 con 0x7f705815a200 2024-05-20T22:08:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f704ffff640 1 -- 172.21.15.55:0/1982334510 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f705001b070 con 0x7f705815a200 2024-05-20T22:08:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f705d938640 1 -- 172.21.15.55:0/1982334510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7058158320 con 0x7f705815a200 2024-05-20T22:08:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f704ffff640 1 -- 172.21.15.55:0/1982334510 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705000f7a0 con 0x7f705815a200 2024-05-20T22:08:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.627+0000 7f705d938640 1 -- 172.21.15.55:0/1982334510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70581544f0 con 0x7f705815a200 2024-05-20T22:08:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.628+0000 7f705d938640 1 -- 172.21.15.55:0/1982334510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7058069090 con 0x7f705815a200 2024-05-20T22:08:42.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.629+0000 7f704ffff640 1 -- 172.21.15.55:0/1982334510 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7050012040 con 0x7f705815a200 2024-05-20T22:08:42.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.629+0000 7f704ffff640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f703806be30 0x7f703806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.629+0000 7f704ffff640 1 -- 172.21.15.55:0/1982334510 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f705008b0f0 con 0x7f705815a200 2024-05-20T22:08:42.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.629+0000 7f7056ffd640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f703806be30 0x7f703806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.630+0000 7f7056ffd640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f703806be30 0x7f703806e340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7048032000 tx=0x7f704802f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:42.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.634+0000 7f704ffff640 1 -- 172.21.15.55:0/1982334510 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f70500596a0 con 0x7f705815a200 2024-05-20T22:08:42.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.819+0000 7f705d938640 1 -- 172.21.15.55:0/1982334510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70580d0500 con 0x7f705815a200 2024-05-20T22:08:42.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.821+0000 7f704ffff640 1 -- 172.21.15.55:0/1982334510 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7f7050058fa0 con 0x7f705815a200 2024-05-20T22:08:42.822 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:42.822 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:42.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 -- 172.21.15.55:0/1982334510 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f703806be30 msgr2=0x7f703806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f703806be30 0x7f703806e340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7048032000 tx=0x7f704802f000 comp rx=0 tx=0).stop 2024-05-20T22:08:42.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 -- 172.21.15.55:0/1982334510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705815a200 msgr2=0x7f7058154a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705815a200 0x7f7058154a10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7044004780 tx=0x7f7050011cf0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 -- 172.21.15.55:0/1982334510 shutdown_connections 2024-05-20T22:08:42.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f703806be30 0x7f703806e340 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705815a200 0x7f7058154a10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7058067490 0x7f7058159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 --2- 172.21.15.55:0/1982334510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7044003b70 0x7f705815c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 -- 172.21.15.55:0/1982334510 >> 172.21.15.55:0/1982334510 conn(0x7f70580d51d0 msgr2=0x7f70581615c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:42.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 -- 172.21.15.55:0/1982334510 shutdown_connections 2024-05-20T22:08:42.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.823+0000 7f705d938640 1 -- 172.21.15.55:0/1982334510 wait complete. 2024-05-20T22:08:42.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:42.836 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:42.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.953+0000 7ff6ff77e640 1 Processor -- start 2024-05-20T22:08:42.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.953+0000 7ff6ff77e640 1 -- start start 2024-05-20T22:08:42.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6ff77e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff6f80b35e0 0x7ff6f815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6ff77e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80d3290 0x7ff6f80d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6ff77e640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6f806b500 con 0x7ff6f80b4080 2024-05-20T22:08:42.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6ff77e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff6f8163960 con 0x7ff6f80b35e0 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6ff77e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f80b3e10 con 0x7ff6f80d3290 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6fd51a640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff6f80b4080 0x7ff6f815f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60470/0 (socket says 172.21.15.55:60470) 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6fd51a640 1 -- 172.21.15.55:0/3819138259 learned_addr learned my addr 172.21.15.55:0/3819138259 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6fdd1b640 1 --2- 172.21.15.55:0/3819138259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80d3290 0x7ff6f80d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6fcd19640 1 --2- 172.21.15.55:0/3819138259 >> v2:172.21.15.138:3300/0 conn(0x7ff6f80b35e0 0x7ff6f815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6effff640 1 -- 172.21.15.55:0/3819138259 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 547533081 0 0) 0x7ff6f4001110 con 0x7ff6f80b4080 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6effff640 1 -- 172.21.15.55:0/3819138259 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6e4003520 con 0x7ff6f80b4080 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6fdd1b640 1 -- 172.21.15.55:0/3819138259 >> v2:172.21.15.138:3300/0 conn(0x7ff6f80b35e0 msgr2=0x7ff6f815f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6fdd1b640 1 --2- 172.21.15.55:0/3819138259 >> v2:172.21.15.138:3300/0 conn(0x7ff6f80b35e0 0x7ff6f815f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6effff640 1 -- 172.21.15.55:0/3819138259 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2326702050 0 0) 0x7ff6e4003520 con 0x7ff6f80b4080 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6fdd1b640 1 -- 172.21.15.55:0/3819138259 >> v1:172.21.15.55:6789/0 conn(0x7ff6f80b4080 legacy=0x7ff6f815f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.955+0000 7ff6fdd1b640 1 -- 172.21.15.55:0/3819138259 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f80b3b30 con 0x7ff6f80d3290 2024-05-20T22:08:42.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.956+0000 7ff6fdd1b640 1 --2- 172.21.15.55:0/3819138259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80d3290 0x7ff6f80d36a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009d90 tx=0x7ff6f002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00062b192a12420 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.956+0000 7ff6effff640 1 -- 172.21.15.55:0/3819138259 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4001840 con 0x7ff6f80d3290 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.956+0000 7ff6effff640 1 -- 172.21.15.55:0/3819138259 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f0007540 con 0x7ff6f80d3290 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.956+0000 7ff6effff640 1 -- 172.21.15.55:0/3819138259 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00078f0 con 0x7ff6f80d3290 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.957+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3819138259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80d3290 msgr2=0x7ff6f80d36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.957+0000 7ff6ff77e640 1 --2- 172.21.15.55:0/3819138259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80d3290 0x7ff6f80d36a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009d90 tx=0x7ff6f002a5f0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.957+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3819138259 shutdown_connections 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.957+0000 7ff6ff77e640 1 --2- 172.21.15.55:0/3819138259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80d3290 0x7ff6f80d36a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.957+0000 7ff6ff77e640 1 --2- 172.21.15.55:0/3819138259 >> v2:172.21.15.138:3300/0 conn(0x7ff6f80b35e0 0x7ff6f815f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.957+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3819138259 >> 172.21.15.55:0/3819138259 conn(0x7ff6f80d51b0 msgr2=0x7ff6f80b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.957+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3819138259 shutdown_connections 2024-05-20T22:08:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.957+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3819138259 wait complete. 2024-05-20T22:08:42.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.959+0000 7ff6ff77e640 1 Processor -- start 2024-05-20T22:08:42.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.959+0000 7ff6ff77e640 1 -- start start 2024-05-20T22:08:42.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.960+0000 7ff6ff77e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80b35e0 0x7ff6f8158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.960+0000 7ff6ff77e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f80b4080 0x7ff6f80cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.960+0000 7ff6ff77e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f80d3290 0x7ff6f80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.960+0000 7ff6ff77e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f81635f0 con 0x7ff6f80d3290 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.960+0000 7ff6ff77e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f81604f0 con 0x7ff6f80b4080 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.960+0000 7ff6ff77e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f815ff30 con 0x7ff6f80b35e0 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.960+0000 7ff6fdd1b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f80d3290 0x7ff6f80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6fdd1b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f80d3290 0x7ff6f80ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49582/0 (socket says 172.21.15.55:49582) 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6fdd1b640 1 -- 172.21.15.55:0/3612308052 learned_addr learned my addr 172.21.15.55:0/3612308052 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6fd51a640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80b35e0 0x7ff6f8158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6fcd19640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f80b4080 0x7ff6f80cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6fdd1b640 1 -- 172.21.15.55:0/3612308052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80b35e0 msgr2=0x7ff6f8158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6fdd1b640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80b35e0 0x7ff6f8158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6fdd1b640 1 -- 172.21.15.55:0/3612308052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f80b4080 msgr2=0x7ff6f80cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6fdd1b640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f80b4080 0x7ff6f80cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6fdd1b640 1 -- 172.21.15.55:0/3612308052 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f80ccaa0 con 0x7ff6f80d3290 2024-05-20T22:08:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6fdd1b640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f80d3290 0x7ff6f80ce420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6f002abe0 tx=0x7ff6f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:42.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6edffb640 1 -- 172.21.15.55:0/3612308052 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f003f070 con 0x7ff6f80d3290 2024-05-20T22:08:42.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6edffb640 1 -- 172.21.15.55:0/3612308052 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f0031c40 con 0x7ff6f80d3290 2024-05-20T22:08:42.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3612308052 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f80ccd80 con 0x7ff6f80d3290 2024-05-20T22:08:42.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.961+0000 7ff6edffb640 1 -- 172.21.15.55:0/3612308052 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0009450 con 0x7ff6f80d3290 2024-05-20T22:08:42.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.962+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3612308052 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f80cd340 con 0x7ff6f80d3290 2024-05-20T22:08:42.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.962+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3612308052 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f8069090 con 0x7ff6f80d3290 2024-05-20T22:08:42.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.963+0000 7ff6edffb640 1 -- 172.21.15.55:0/3612308052 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6f0043410 con 0x7ff6f80d3290 2024-05-20T22:08:42.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.964+0000 7ff6edffb640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff6dc06be30 0x7ff6dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:42.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.964+0000 7ff6edffb640 1 -- 172.21.15.55:0/3612308052 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff6f00ac590 con 0x7ff6f80d3290 2024-05-20T22:08:42.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.964+0000 7ff6fd51a640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff6dc06be30 0x7ff6dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:42.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.964+0000 7ff6fd51a640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff6dc06be30 0x7ff6dc06e340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4005880 tx=0x7ff6f4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:42.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:42.968+0000 7ff6edffb640 1 -- 172.21.15.55:0/3612308052 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff6f007ab20 con 0x7ff6f80d3290 2024-05-20T22:08:43.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.154+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3612308052 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff6f80ba9e0 con 0x7ff6f80d3290 2024-05-20T22:08:43.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.155+0000 7ff6edffb640 1 -- 172.21.15.55:0/3612308052 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7ff6f007a420 con 0x7ff6f80d3290 2024-05-20T22:08:43.156 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.157+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3612308052 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff6dc06be30 msgr2=0x7ff6dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.157+0000 7ff6ff77e640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff6dc06be30 0x7ff6dc06e340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4005880 tx=0x7ff6f4009040 comp rx=0 tx=0).stop 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.157+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3612308052 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f80d3290 msgr2=0x7ff6f80ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.157+0000 7ff6ff77e640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f80d3290 0x7ff6f80ce420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6f002abe0 tx=0x7ff6f0004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.157+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3612308052 shutdown_connections 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.157+0000 7ff6ff77e640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff6dc06be30 0x7ff6dc06e340 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.157+0000 7ff6ff77e640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6f80d3290 0x7ff6f80ce420 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.157+0000 7ff6ff77e640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f80b4080 0x7ff6f80cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.157+0000 7ff6ff77e640 1 --2- 172.21.15.55:0/3612308052 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff6f80b35e0 0x7ff6f8158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.157+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3612308052 >> 172.21.15.55:0/3612308052 conn(0x7ff6f80d51b0 msgr2=0x7ff6f80ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.158+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3612308052 shutdown_connections 2024-05-20T22:08:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.158+0000 7ff6ff77e640 1 -- 172.21.15.55:0/3612308052 wait complete. 2024-05-20T22:08:43.170 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:43.260 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:08:43.263 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:08:43.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.288+0000 7fdfd64ab640 1 Processor -- start 2024-05-20T22:08:43.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.288+0000 7fdfd64ab640 1 -- start start 2024-05-20T22:08:43.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.289+0000 7fdfd64ab640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdfd009ac90 0x7fdfd00d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.289+0000 7fdfd64ab640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd00b5750 0x7fdfd00b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.289+0000 7fdfd64ab640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd00b7f00 con 0x7fdfd009b0a0 2024-05-20T22:08:43.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.289+0000 7fdfd64ab640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd00ce6d0 con 0x7fdfd009ac90 2024-05-20T22:08:43.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.289+0000 7fdfd64ab640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd00ce080 con 0x7fdfd00b5750 2024-05-20T22:08:43.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.290+0000 7fdfcffff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdfd009ac90 0x7fdfd00d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.290+0000 7fdfcf7fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdfd009b0a0 0x7fdfd00d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60474/0 (socket says 172.21.15.55:60474) 2024-05-20T22:08:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.290+0000 7fdfcf7fe640 1 -- 172.21.15.55:0/3506527128 learned_addr learned my addr 172.21.15.55:0/3506527128 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.290+0000 7fdfd4a48640 1 --2- 172.21.15.55:0/3506527128 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd00b5750 0x7fdfd00b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.290+0000 7fdfcffff640 1 -- 172.21.15.55:0/3506527128 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd00b5750 msgr2=0x7fdfd00b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.290+0000 7fdfcffff640 1 --2- 172.21.15.55:0/3506527128 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd00b5750 0x7fdfd00b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.290+0000 7fdfcffff640 1 -- 172.21.15.55:0/3506527128 >> v1:172.21.15.55:6789/0 conn(0x7fdfd009b0a0 legacy=0x7fdfd00d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.290+0000 7fdfcffff640 1 -- 172.21.15.55:0/3506527128 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd00ce360 con 0x7fdfd009ac90 2024-05-20T22:08:43.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.291+0000 7fdfcffff640 1 --2- 172.21.15.55:0/3506527128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 0x7fdfd00d0870 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00ce850 tx=0x7fdfc002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662722990e05fc41 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:43.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.292+0000 7fdfceffd640 1 -- 172.21.15.55:0/3506527128 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0037070 con 0x7fdfd009ac90 2024-05-20T22:08:43.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.292+0000 7fdfceffd640 1 -- 172.21.15.55:0/3506527128 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc002ae50 con 0x7fdfd009ac90 2024-05-20T22:08:43.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.292+0000 7fdfceffd640 1 -- 172.21.15.55:0/3506527128 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0032660 con 0x7fdfd009ac90 2024-05-20T22:08:43.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.292+0000 7fdfd64ab640 1 -- 172.21.15.55:0/3506527128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 msgr2=0x7fdfd00d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.292+0000 7fdfd64ab640 1 --2- 172.21.15.55:0/3506527128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 0x7fdfd00d0870 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00ce850 tx=0x7fdfc002a100 comp rx=0 tx=0).stop 2024-05-20T22:08:43.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.293+0000 7fdfd64ab640 1 -- 172.21.15.55:0/3506527128 shutdown_connections 2024-05-20T22:08:43.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.293+0000 7fdfd64ab640 1 --2- 172.21.15.55:0/3506527128 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd00b5750 0x7fdfd00b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.293+0000 7fdfd64ab640 1 --2- 172.21.15.55:0/3506527128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 0x7fdfd00d0870 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.293+0000 7fdfd64ab640 1 -- 172.21.15.55:0/3506527128 >> 172.21.15.55:0/3506527128 conn(0x7fdfd00b43f0 msgr2=0x7fdfd00b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:43.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.293+0000 7fdfd64ab640 1 -- 172.21.15.55:0/3506527128 shutdown_connections 2024-05-20T22:08:43.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.293+0000 7fdfd64ab640 1 -- 172.21.15.55:0/3506527128 wait complete. 2024-05-20T22:08:43.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.294+0000 7fdfd64ab640 1 Processor -- start 2024-05-20T22:08:43.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.295+0000 7fdfd64ab640 1 -- start start 2024-05-20T22:08:43.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfd64ab640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 0x7fdfd01578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfd64ab640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd009b0a0 0x7fdfd015e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfd64ab640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd00b5750 0x7fdfd009d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfd64ab640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd009bb70 con 0x7fdfd009ac90 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfd64ab640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd00b81c0 con 0x7fdfd00b5750 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfd64ab640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd009bcf0 con 0x7fdfd009b0a0 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfcffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 0x7fdfd01578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfcffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 0x7fdfd01578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49600/0 (socket says 172.21.15.55:49600) 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfcffff640 1 -- 172.21.15.55:0/56706406 learned_addr learned my addr 172.21.15.55:0/56706406 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfcf7fe640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd009b0a0 0x7fdfd015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfd4a48640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd00b5750 0x7fdfd009d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfcffff640 1 -- 172.21.15.55:0/56706406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd009b0a0 msgr2=0x7fdfd015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfcffff640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd009b0a0 0x7fdfd015e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfcffff640 1 -- 172.21.15.55:0/56706406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd00b5750 msgr2=0x7fdfd009d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfcffff640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd00b5750 0x7fdfd009d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.296+0000 7fdfcffff640 1 -- 172.21.15.55:0/56706406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd009ef40 con 0x7fdfd009ac90 2024-05-20T22:08:43.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.297+0000 7fdfcffff640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 0x7fdfd01578f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009890 tx=0x7fdfc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:43.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.297+0000 7fdfccff9640 1 -- 172.21.15.55:0/56706406 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0041020 con 0x7fdfd009ac90 2024-05-20T22:08:43.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.297+0000 7fdfccff9640 1 -- 172.21.15.55:0/56706406 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc003c670 con 0x7fdfd009ac90 2024-05-20T22:08:43.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.297+0000 7fdfd64ab640 1 -- 172.21.15.55:0/56706406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd009f220 con 0x7fdfd009ac90 2024-05-20T22:08:43.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.297+0000 7fdfccff9640 1 -- 172.21.15.55:0/56706406 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0037040 con 0x7fdfd009ac90 2024-05-20T22:08:43.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.297+0000 7fdfd64ab640 1 -- 172.21.15.55:0/56706406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd0157e30 con 0x7fdfd009ac90 2024-05-20T22:08:43.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.298+0000 7fdfd64ab640 1 -- 172.21.15.55:0/56706406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd0069090 con 0x7fdfd009ac90 2024-05-20T22:08:43.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.298+0000 7fdfccff9640 1 -- 172.21.15.55:0/56706406 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfc0044020 con 0x7fdfd009ac90 2024-05-20T22:08:43.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.299+0000 7fdfccff9640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdfb006be30 0x7fdfb006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.299+0000 7fdfccff9640 1 -- 172.21.15.55:0/56706406 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdfc00ab970 con 0x7fdfd009ac90 2024-05-20T22:08:43.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.299+0000 7fdfcf7fe640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdfb006be30 0x7fdfb006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.299+0000 7fdfcf7fe640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdfb006be30 0x7fdfb006e340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdfd015e2c0 tx=0x7fdfb8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:43.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.304+0000 7fdfccff9640 1 -- 172.21.15.55:0/56706406 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdfc004b030 con 0x7fdfd009ac90 2024-05-20T22:08:43.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.487+0000 7fdfd64ab640 1 -- 172.21.15.55:0/56706406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdfd009b720 con 0x7fdfd009ac90 2024-05-20T22:08:43.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.487+0000 7fdfccff9640 1 -- 172.21.15.55:0/56706406 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fdfc0079f20 con 0x7fdfd009ac90 2024-05-20T22:08:43.488 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:43.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 -- 172.21.15.55:0/56706406 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdfb006be30 msgr2=0x7fdfb006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdfb006be30 0x7fdfb006e340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdfd015e2c0 tx=0x7fdfb8009040 comp rx=0 tx=0).stop 2024-05-20T22:08:43.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 -- 172.21.15.55:0/56706406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 msgr2=0x7fdfd01578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 0x7fdfd01578f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009890 tx=0x7fdfc00046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 -- 172.21.15.55:0/56706406 shutdown_connections 2024-05-20T22:08:43.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdfb006be30 0x7fdfb006e340 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfd00b5750 0x7fdfd009d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfd009b0a0 0x7fdfd015e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 --2- 172.21.15.55:0/56706406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfd009ac90 0x7fdfd01578f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 -- 172.21.15.55:0/56706406 >> 172.21.15.55:0/56706406 conn(0x7fdfd00b43f0 msgr2=0x7fdfd015c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:43.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 -- 172.21.15.55:0/56706406 shutdown_connections 2024-05-20T22:08:43.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.489+0000 7fdfd64ab640 1 -- 172.21.15.55:0/56706406 wait complete. 2024-05-20T22:08:43.501 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:43.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.618+0000 7fa2b626c640 1 Processor -- start 2024-05-20T22:08:43.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.618+0000 7fa2b626c640 1 -- start start 2024-05-20T22:08:43.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.619+0000 7fa2b626c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa2b00d32a0 0x7fa2b015f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.619+0000 7fa2b626c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2b00cf690 0x7fa2b00cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.619+0000 7fa2b626c640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b00d3780 con 0x7fa2b00cffe0 2024-05-20T22:08:43.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.619+0000 7fa2b626c640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b006c510 con 0x7fa2b00d32a0 2024-05-20T22:08:43.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.619+0000 7fa2b626c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b00d3a60 con 0x7fa2b00cf690 2024-05-20T22:08:43.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.620+0000 7fa2affff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa2b00d32a0 0x7fa2b015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.620+0000 7fa2b4809640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa2b00cffe0 0x7fa2b0162cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60488/0 (socket says 172.21.15.55:60488) 2024-05-20T22:08:43.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.620+0000 7fa2b4809640 1 -- 172.21.15.55:0/2375059024 learned_addr learned my addr 172.21.15.55:0/2375059024 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:43.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.620+0000 7fa2af7fe640 1 --2- 172.21.15.55:0/2375059024 >> v2:172.21.15.138:3300/0 conn(0x7fa2b00cf690 0x7fa2b00cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.620+0000 7fa2affff640 1 -- 172.21.15.55:0/2375059024 >> v2:172.21.15.138:3300/0 conn(0x7fa2b00cf690 msgr2=0x7fa2b00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:43.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.620+0000 7fa2affff640 1 --2- 172.21.15.55:0/2375059024 >> v2:172.21.15.138:3300/0 conn(0x7fa2b00cf690 0x7fa2b00cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.620+0000 7fa2affff640 1 -- 172.21.15.55:0/2375059024 >> v1:172.21.15.55:6789/0 conn(0x7fa2b00cffe0 legacy=0x7fa2b0162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.620+0000 7fa2affff640 1 -- 172.21.15.55:0/2375059024 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b00d3d40 con 0x7fa2b00d32a0 2024-05-20T22:08:43.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.620+0000 7fa2aeffd640 1 -- 172.21.15.55:0/2375059024 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 108151877 0 0) 0x7fa2b00d3780 con 0x7fa2b00cffe0 2024-05-20T22:08:43.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.621+0000 7fa2affff640 1 --2- 172.21.15.55:0/2375059024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00d32a0 0x7fa2b015f150 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa2b008e150 tx=0x7fa2a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389afa452adea5f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:43.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.621+0000 7fa2aeffd640 1 -- 172.21.15.55:0/2375059024 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0037070 con 0x7fa2b00d32a0 2024-05-20T22:08:43.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.621+0000 7fa2aeffd640 1 -- 172.21.15.55:0/2375059024 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a0032440 con 0x7fa2b00d32a0 2024-05-20T22:08:43.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.621+0000 7fa2aeffd640 1 -- 172.21.15.55:0/2375059024 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0032810 con 0x7fa2b00d32a0 2024-05-20T22:08:43.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.621+0000 7fa2b626c640 1 -- 172.21.15.55:0/2375059024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00d32a0 msgr2=0x7fa2b015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.621+0000 7fa2b626c640 1 --2- 172.21.15.55:0/2375059024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00d32a0 0x7fa2b015f150 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa2b008e150 tx=0x7fa2a002a1a0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.622+0000 7fa2b626c640 1 -- 172.21.15.55:0/2375059024 shutdown_connections 2024-05-20T22:08:43.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.622+0000 7fa2b626c640 1 --2- 172.21.15.55:0/2375059024 >> v2:172.21.15.138:3300/0 conn(0x7fa2b00cf690 0x7fa2b00cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.622+0000 7fa2b626c640 1 --2- 172.21.15.55:0/2375059024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00d32a0 0x7fa2b015f150 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.622+0000 7fa2b626c640 1 -- 172.21.15.55:0/2375059024 >> 172.21.15.55:0/2375059024 conn(0x7fa2b00b6de0 msgr2=0x7fa2b00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:43.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.622+0000 7fa2b626c640 1 -- 172.21.15.55:0/2375059024 shutdown_connections 2024-05-20T22:08:43.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.622+0000 7fa2b626c640 1 -- 172.21.15.55:0/2375059024 wait complete. 2024-05-20T22:08:43.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.623+0000 7fa2b626c640 1 Processor -- start 2024-05-20T22:08:43.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.624+0000 7fa2b626c640 1 -- start start 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2b626c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2b00cf690 0x7fa2b0156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2b626c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00cffe0 0x7fa2b00cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2b626c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b00d32a0 0x7fa2b00cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2b626c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b00d03f0 con 0x7fa2b00cffe0 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2b626c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b00d4300 con 0x7fa2b00d32a0 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2b626c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b00d4820 con 0x7fa2b00cf690 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2af7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00cffe0 0x7fa2b00cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2af7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00cffe0 0x7fa2b00cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49610/0 (socket says 172.21.15.55:49610) 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2af7fe640 1 -- 172.21.15.55:0/4270409314 learned_addr learned my addr 172.21.15.55:0/4270409314 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2affff640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2b00cf690 0x7fa2b0156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2b4809640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b00d32a0 0x7fa2b00cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2af7fe640 1 -- 172.21.15.55:0/4270409314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2b00cf690 msgr2=0x7fa2b0156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2af7fe640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2b00cf690 0x7fa2b0156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2af7fe640 1 -- 172.21.15.55:0/4270409314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b00d32a0 msgr2=0x7fa2b00cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2af7fe640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b00d32a0 0x7fa2b00cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2af7fe640 1 -- 172.21.15.55:0/4270409314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b01f96f0 con 0x7fa2b00cffe0 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.625+0000 7fa2b4809640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b00d32a0 0x7fa2b00cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.626+0000 7fa2af7fe640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00cffe0 0x7fa2b00cba30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400d620 tx=0x7fa2a400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.626+0000 7fa2acff9640 1 -- 172.21.15.55:0/4270409314 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400b620 con 0x7fa2b00cffe0 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.626+0000 7fa2acff9640 1 -- 172.21.15.55:0/4270409314 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a4019070 con 0x7fa2b00cffe0 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.626+0000 7fa2b626c640 1 -- 172.21.15.55:0/4270409314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b00c9fa0 con 0x7fa2b00cffe0 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.626+0000 7fa2acff9640 1 -- 172.21.15.55:0/4270409314 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400ba00 con 0x7fa2b00cffe0 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.626+0000 7fa2b626c640 1 -- 172.21.15.55:0/4270409314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b00ca560 con 0x7fa2b00cffe0 2024-05-20T22:08:43.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.627+0000 7fa2b626c640 1 -- 172.21.15.55:0/4270409314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b00c9d90 con 0x7fa2b00cffe0 2024-05-20T22:08:43.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.627+0000 7fa2acff9640 1 -- 172.21.15.55:0/4270409314 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2a400bba0 con 0x7fa2b00cffe0 2024-05-20T22:08:43.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.628+0000 7fa2acff9640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa29c06be30 0x7fa29c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.628+0000 7fa2acff9640 1 -- 172.21.15.55:0/4270409314 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa2a408b660 con 0x7fa2b00cffe0 2024-05-20T22:08:43.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.628+0000 7fa2affff640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa29c06be30 0x7fa29c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.629+0000 7fa2affff640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa29c06be30 0x7fa29c06e340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0034000 tx=0x7fa2a002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:43.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.633+0000 7fa2acff9640 1 -- 172.21.15.55:0/4270409314 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa2a4059b80 con 0x7fa2b00cffe0 2024-05-20T22:08:43.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.814+0000 7fa2b626c640 1 -- 172.21.15.55:0/4270409314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa2b01f9480 con 0x7fa2b00cffe0 2024-05-20T22:08:43.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.815+0000 7fa2acff9640 1 -- 172.21.15.55:0/4270409314 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fa2a4059480 con 0x7fa2b00cffe0 2024-05-20T22:08:43.816 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:43.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 -- 172.21.15.55:0/4270409314 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa29c06be30 msgr2=0x7fa29c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa29c06be30 0x7fa29c06e340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0034000 tx=0x7fa2a002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:43.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 -- 172.21.15.55:0/4270409314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00cffe0 msgr2=0x7fa2b00cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00cffe0 0x7fa2b00cba30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400d620 tx=0x7fa2a400dae0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 -- 172.21.15.55:0/4270409314 shutdown_connections 2024-05-20T22:08:43.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa29c06be30 0x7fa29c06e340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b00d32a0 0x7fa2b00cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2b00cffe0 0x7fa2b00cba30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 --2- 172.21.15.55:0/4270409314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2b00cf690 0x7fa2b0156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 -- 172.21.15.55:0/4270409314 >> 172.21.15.55:0/4270409314 conn(0x7fa2b00b6de0 msgr2=0x7fa2b015ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:43.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 -- 172.21.15.55:0/4270409314 shutdown_connections 2024-05-20T22:08:43.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.817+0000 7fa2b626c640 1 -- 172.21.15.55:0/4270409314 wait complete. 2024-05-20T22:08:43.829 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:43.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.946+0000 7fb6d412c640 1 Processor -- start 2024-05-20T22:08:43.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.947+0000 7fb6d412c640 1 -- start start 2024-05-20T22:08:43.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.948+0000 7fb6d412c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0b1ef0 0x7fb6cc0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.948+0000 7fb6d412c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb6cc0d1540 0x7fb6cc0b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.948+0000 7fb6d412c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6cc0b40d0 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.948+0000 7fb6d412c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb6cc0b6160 con 0x7fb6cc0d1540 2024-05-20T22:08:43.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.948+0000 7fb6d412c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb6cc0b5b10 con 0x7fb6cc0b1ef0 2024-05-20T22:08:43.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.948+0000 7fb6d16c7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0b1ef0 0x7fb6cc0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.948+0000 7fb6d26c9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb6cc0d1540 0x7fb6cc0b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.948+0000 7fb6d1ec8640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb6cc0b7d90 0x7fb6cc0b3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:44296/0 (socket says 172.21.15.55:44296) 2024-05-20T22:08:43.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.948+0000 7fb6d1ec8640 1 -- 172.21.15.55:0/789896976 learned_addr learned my addr 172.21.15.55:0/789896976 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:43.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.949+0000 7fb6d16c7640 1 -- 172.21.15.55:0/789896976 >> v2:172.21.15.138:3300/0 conn(0x7fb6cc0d1540 msgr2=0x7fb6cc0b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:43.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.949+0000 7fb6d16c7640 1 --2- 172.21.15.55:0/789896976 >> v2:172.21.15.138:3300/0 conn(0x7fb6cc0d1540 0x7fb6cc0b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.949+0000 7fb6d16c7640 1 -- 172.21.15.55:0/789896976 >> v1:172.21.15.138:6789/0 conn(0x7fb6cc0b7d90 legacy=0x7fb6cc0b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.949+0000 7fb6d16c7640 1 -- 172.21.15.55:0/789896976 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6cc0b5df0 con 0x7fb6cc0b1ef0 2024-05-20T22:08:43.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.949+0000 7fb6d16c7640 1 --2- 172.21.15.55:0/789896976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0b1ef0 0x7fb6cc0d1000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0097a0 tx=0x7fb6bc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4bed9f5e8b7b04b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:43.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.952+0000 7fb6d0ec6640 1 -- 172.21.15.55:0/789896976 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc037070 con 0x7fb6cc0b1ef0 2024-05-20T22:08:43.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.952+0000 7fb6d0ec6640 1 -- 172.21.15.55:0/789896976 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6bc032440 con 0x7fb6cc0b1ef0 2024-05-20T22:08:43.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.952+0000 7fb6d0ec6640 1 -- 172.21.15.55:0/789896976 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc032810 con 0x7fb6cc0b1ef0 2024-05-20T22:08:43.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.953+0000 7fb6d412c640 1 -- 172.21.15.55:0/789896976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0b1ef0 msgr2=0x7fb6cc0d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.953+0000 7fb6d412c640 1 --2- 172.21.15.55:0/789896976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0b1ef0 0x7fb6cc0d1000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0097a0 tx=0x7fb6bc02a180 comp rx=0 tx=0).stop 2024-05-20T22:08:43.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.953+0000 7fb6d412c640 1 -- 172.21.15.55:0/789896976 shutdown_connections 2024-05-20T22:08:43.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.953+0000 7fb6d412c640 1 --2- 172.21.15.55:0/789896976 >> v2:172.21.15.138:3300/0 conn(0x7fb6cc0d1540 0x7fb6cc0b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.953+0000 7fb6d412c640 1 --2- 172.21.15.55:0/789896976 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0b1ef0 0x7fb6cc0d1000 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.953+0000 7fb6d412c640 1 -- 172.21.15.55:0/789896976 >> 172.21.15.55:0/789896976 conn(0x7fb6cc09be10 msgr2=0x7fb6cc0d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:43.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.953+0000 7fb6d412c640 1 -- 172.21.15.55:0/789896976 shutdown_connections 2024-05-20T22:08:43.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.953+0000 7fb6d412c640 1 -- 172.21.15.55:0/789896976 wait complete. 2024-05-20T22:08:43.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.955+0000 7fb6d412c640 1 Processor -- start 2024-05-20T22:08:43.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.955+0000 7fb6d412c640 1 -- start start 2024-05-20T22:08:43.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.956+0000 7fb6d412c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6cc0b1ef0 0x7fb6cc15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.956+0000 7fb6d412c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7d90 0x7fb6cc0cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.956+0000 7fb6d412c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0d1540 0x7fb6cc06dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.956+0000 7fb6d412c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc0b6cb0 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.956+0000 7fb6d412c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc0b6b30 con 0x7fb6cc0b1ef0 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.956+0000 7fb6d412c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb6cc0b6e30 con 0x7fb6cc0d1540 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.956+0000 7fb6d16c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7d90 0x7fb6cc0cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.956+0000 7fb6d16c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7d90 0x7fb6cc0cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49624/0 (socket says 172.21.15.55:49624) 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.956+0000 7fb6d16c7640 1 -- 172.21.15.55:0/3211152080 learned_addr learned my addr 172.21.15.55:0/3211152080 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6d26c9640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0d1540 0x7fb6cc06dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6d1ec8640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6cc0b1ef0 0x7fb6cc15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6d16c7640 1 -- 172.21.15.55:0/3211152080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0d1540 msgr2=0x7fb6cc06dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6d16c7640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0d1540 0x7fb6cc06dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6d16c7640 1 -- 172.21.15.55:0/3211152080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6cc0b1ef0 msgr2=0x7fb6cc15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6d16c7640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6cc0b1ef0 0x7fb6cc15af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6d16c7640 1 -- 172.21.15.55:0/3211152080 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6cc0cbea0 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6d16c7640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7d90 0x7fb6cc0cb440 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0098c0 tx=0x7fb6bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:43.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6c27fc640 1 -- 172.21.15.55:0/3211152080 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc037040 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6c27fc640 1 -- 172.21.15.55:0/3211152080 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6bc030500 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6d412c640 1 -- 172.21.15.55:0/3211152080 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6cc0cc180 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6c27fc640 1 -- 172.21.15.55:0/3211152080 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0308d0 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.957+0000 7fb6d412c640 1 -- 172.21.15.55:0/3211152080 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6cc15c130 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.959+0000 7fb6d412c640 1 -- 172.21.15.55:0/3211152080 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6cc069090 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.959+0000 7fb6c27fc640 1 -- 172.21.15.55:0/3211152080 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6bc044020 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.959+0000 7fb6c27fc640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6b006be30 0x7fb6b006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:43.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.960+0000 7fb6c27fc640 1 -- 172.21.15.55:0/3211152080 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb6bc0ab440 con 0x7fb6cc0b7d90 2024-05-20T22:08:43.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.960+0000 7fb6d1ec8640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6b006be30 0x7fb6b006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:43.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.960+0000 7fb6d1ec8640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6b006be30 0x7fb6b006e340 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80012e0 tx=0x7fb6c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:43.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:43.964+0000 7fb6c27fc640 1 -- 172.21.15.55:0/3211152080 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb6bc0798e0 con 0x7fb6cc0b7d90 2024-05-20T22:08:44.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.147+0000 7fb6d412c640 1 -- 172.21.15.55:0/3211152080 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb6cc0b2680 con 0x7fb6cc0b7d90 2024-05-20T22:08:44.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.147+0000 7fb6c27fc640 1 -- 172.21.15.55:0/3211152080 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb6bc0791e0 con 0x7fb6cc0b7d90 2024-05-20T22:08:44.148 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:44.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 -- 172.21.15.55:0/3211152080 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6b006be30 msgr2=0x7fb6b006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6b006be30 0x7fb6b006e340 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80012e0 tx=0x7fb6c8009040 comp rx=0 tx=0).stop 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 -- 172.21.15.55:0/3211152080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7d90 msgr2=0x7fb6cc0cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7d90 0x7fb6cc0cb440 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0098c0 tx=0x7fb6bc0046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 -- 172.21.15.55:0/3211152080 shutdown_connections 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6b006be30 0x7fb6b006e340 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6cc0d1540 0x7fb6cc06dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6cc0b7d90 0x7fb6cc0cb440 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 --2- 172.21.15.55:0/3211152080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6cc0b1ef0 0x7fb6cc15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 -- 172.21.15.55:0/3211152080 >> 172.21.15.55:0/3211152080 conn(0x7fb6cc09be10 msgr2=0x7fb6cc08bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 -- 172.21.15.55:0/3211152080 shutdown_connections 2024-05-20T22:08:44.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.149+0000 7fb6d412c640 1 -- 172.21.15.55:0/3211152080 wait complete. 2024-05-20T22:08:44.162 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:08:44.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.277+0000 7f81f5aca640 1 Processor -- start 2024-05-20T22:08:44.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.277+0000 7f81f5aca640 1 -- start start 2024-05-20T22:08:44.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.278+0000 7f81f5aca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f015f150 0x7f81f00cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.278+0000 7f81f5aca640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81f00d3780 con 0x7f81f00d32a0 2024-05-20T22:08:44.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.278+0000 7f81f5aca640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81f00d3a60 con 0x7f81f006b500 2024-05-20T22:08:44.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.278+0000 7f81f5aca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f81f00d3d40 con 0x7f81f015f150 2024-05-20T22:08:44.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.278+0000 7f81ef7fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f81f00d32a0 0x7f81f00cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60492/0 (socket says 172.21.15.55:60492) 2024-05-20T22:08:44.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.278+0000 7f81ef7fe640 1 -- 172.21.15.55:0/3830087848 learned_addr learned my addr 172.21.15.55:0/3830087848 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:44.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.278+0000 7f81eeffd640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f015f150 0x7f81f00cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.278+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1143764480 0 0) 0x7f81f00d3780 con 0x7f81f00d32a0 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.278+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81d8003520 con 0x7f81f00d32a0 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1366926640 0 0) 0x7f81f00d3a60 con 0x7f81f006b500 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81f00d3780 con 0x7f81f006b500 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1230387645 0 0) 0x7f81d8003520 con 0x7f81f00d32a0 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f015f150 msgr2=0x7f81f00cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f015f150 0x7f81f00cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 >> v1:172.21.15.138:6789/0 conn(0x7f81f006b500 legacy=0x7f81f006b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f00d4020 con 0x7f81f00d32a0 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f81e0001670 con 0x7f81f00d32a0 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 >> v1:172.21.15.55:6789/0 conn(0x7f81f00d32a0 legacy=0x7f81f00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f015f150 0x7f81d8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81d8004140 0x7f81d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81d8004d40 0x7f81d80051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f81f00d3d40 con 0x7f81d8004140 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81d800bbb0 con 0x7f81d8004d40 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f81d800bdb0 con 0x7f81f015f150 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81eeffd640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81d8004140 0x7f81d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81ef7fe640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f015f150 0x7f81d8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81effff640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81d8004d40 0x7f81d80051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81eeffd640 1 -- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f015f150 msgr2=0x7f81d8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81eeffd640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81f015f150 0x7f81d8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81eeffd640 1 -- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81d8004d40 msgr2=0x7f81d80051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81eeffd640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81d8004d40 0x7f81d80051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81eeffd640 1 -- 172.21.15.55:0/3830087848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d8003520 con 0x7f81d8004140 2024-05-20T22:08:44.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.279+0000 7f81effff640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81d8004d40 0x7f81d80051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.280+0000 7f81eeffd640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81d8004140 0x7f81d8004800 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009a90 tx=0x7f81d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5ddb175840262f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.280+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4036070 con 0x7f81d8004140 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.280+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e4031400 con 0x7f81d8004140 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.280+0000 7f81ee7fc640 1 -- 172.21.15.55:0/3830087848 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4009690 con 0x7f81d8004140 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.281+0000 7f81f5aca640 1 -- 172.21.15.55:0/3830087848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81d8004140 msgr2=0x7f81d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.281+0000 7f81f5aca640 1 --2- 172.21.15.55:0/3830087848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81d8004140 0x7f81d8004800 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009a90 tx=0x7f81d8005830 comp rx=0 tx=0).stop 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.281+0000 7f81effff640 1 -- 172.21.15.55:0/3830087848 reap_dead start 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.281+0000 7f81f5aca640 1 -- 172.21.15.55:0/3830087848 shutdown_connections 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.281+0000 7f81f5aca640 1 -- 172.21.15.55:0/3830087848 >> 172.21.15.55:0/3830087848 conn(0x7f81f00b6de0 msgr2=0x7f81f00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.281+0000 7f81f5aca640 1 -- 172.21.15.55:0/3830087848 shutdown_connections 2024-05-20T22:08:44.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.281+0000 7f81f5aca640 1 -- 172.21.15.55:0/3830087848 wait complete. 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.283+0000 7f81f5aca640 1 Processor -- start 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.283+0000 7f81f5aca640 1 -- start start 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.284+0000 7f81f5aca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81d8004140 0x7f81f014d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.284+0000 7f81f5aca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f01f8d30 0x7f81f01f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.284+0000 7f81f5aca640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81f015bbc0 0x7f81f015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.284+0000 7f81f5aca640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f81f00d4820 con 0x7f81f015bbc0 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.284+0000 7f81f5aca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81f015f560 con 0x7f81f01f8d30 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.284+0000 7f81f5aca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f81f01f9810 con 0x7f81d8004140 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.284+0000 7f81effff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81f015bbc0 0x7f81f015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.284+0000 7f81effff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81f015bbc0 0x7f81f015bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49646/0 (socket says 172.21.15.55:49646) 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.284+0000 7f81effff640 1 -- 172.21.15.55:0/860926546 learned_addr learned my addr 172.21.15.55:0/860926546 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81ef7fe640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81d8004140 0x7f81f014d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81eeffd640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f01f8d30 0x7f81f01f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81effff640 1 -- 172.21.15.55:0/860926546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81d8004140 msgr2=0x7f81f014d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81effff640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81d8004140 0x7f81f014d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81effff640 1 -- 172.21.15.55:0/860926546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f01f8d30 msgr2=0x7f81f01f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81effff640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f01f8d30 0x7f81f01f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81effff640 1 -- 172.21.15.55:0/860926546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f015c510 con 0x7f81f015bbc0 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81eeffd640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f01f8d30 0x7f81f01f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81effff640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81f015bbc0 0x7f81f015bfd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f81f00b7830 tx=0x7f81e800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81d7fff640 1 -- 172.21.15.55:0/860926546 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8019070 con 0x7f81f015bbc0 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81d7fff640 1 -- 172.21.15.55:0/860926546 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e8003b80 con 0x7f81f015bbc0 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81d7fff640 1 -- 172.21.15.55:0/860926546 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8006b50 con 0x7f81f015bbc0 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81f5aca640 1 -- 172.21.15.55:0/860926546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f01f9a60 con 0x7f81f015bbc0 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.285+0000 7f81f5aca640 1 -- 172.21.15.55:0/860926546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f01e8f30 con 0x7f81f015bbc0 2024-05-20T22:08:44.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.286+0000 7f81f5aca640 1 -- 172.21.15.55:0/860926546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f00d24f0 con 0x7f81f015bbc0 2024-05-20T22:08:44.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.287+0000 7f81d7fff640 1 -- 172.21.15.55:0/860926546 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81e801c070 con 0x7f81f015bbc0 2024-05-20T22:08:44.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.287+0000 7f81d7fff640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f81dc06be30 0x7f81dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.288+0000 7f81ef7fe640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f81dc06be30 0x7f81dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.288+0000 7f81d7fff640 1 -- 172.21.15.55:0/860926546 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f81e808a6c0 con 0x7f81f015bbc0 2024-05-20T22:08:44.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.288+0000 7f81ef7fe640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f81dc06be30 0x7f81dc06e340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f81e0007be0 tx=0x7f81e0001d70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:44.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.292+0000 7f81d7fff640 1 -- 172.21.15.55:0/860926546 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f81f00d24f0 con 0x7f81f015bbc0 2024-05-20T22:08:44.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.493+0000 7f81f5aca640 1 -- 172.21.15.55:0/860926546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f81f0069090 con 0x7f81f015bbc0 2024-05-20T22:08:44.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.493+0000 7f81d7fff640 1 -- 172.21.15.55:0/860926546 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f81e80584f0 con 0x7f81f015bbc0 2024-05-20T22:08:44.494 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-20T22:08:44.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 -- 172.21.15.55:0/860926546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f81dc06be30 msgr2=0x7f81dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f81dc06be30 0x7f81dc06e340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f81e0007be0 tx=0x7f81e0001d70 comp rx=0 tx=0).stop 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 -- 172.21.15.55:0/860926546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81f015bbc0 msgr2=0x7f81f015bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81f015bbc0 0x7f81f015bfd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f81f00b7830 tx=0x7f81e800ec60 comp rx=0 tx=0).stop 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 -- 172.21.15.55:0/860926546 shutdown_connections 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f81dc06be30 0x7f81dc06e340 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81f015bbc0 0x7f81f015bfd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f01f8d30 0x7f81f01f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 --2- 172.21.15.55:0/860926546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81d8004140 0x7f81f014d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 -- 172.21.15.55:0/860926546 >> 172.21.15.55:0/860926546 conn(0x7f81f00b6de0 msgr2=0x7f81f015d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 -- 172.21.15.55:0/860926546 shutdown_connections 2024-05-20T22:08:44.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.495+0000 7f81f5aca640 1 -- 172.21.15.55:0/860926546 wait complete. 2024-05-20T22:08:44.508 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:08:44.508 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:44.524 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:44.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.678+0000 7fe698db7640 1 Processor -- start 2024-05-20T22:08:44.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.678+0000 7fe698db7640 1 -- start start 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.679+0000 7fe698db7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6940cd840 0x7fe6940b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.679+0000 7fe698db7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe6940d2e10 0x7fe6940d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.679+0000 7fe698db7640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe69415da90 con 0x7fe6940b5830 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.679+0000 7fe698db7640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fe6940b6990 con 0x7fe6940d2e10 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.679+0000 7fe698db7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6940b5d80 con 0x7fe6940cd840 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.679+0000 7fe692d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe6940d2e10 0x7fe6940d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.679+0000 7fe691d9b640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe6940b5830 0x7fe6940d2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60496/0 (socket says 172.21.15.55:60496) 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.679+0000 7fe691d9b640 1 -- 172.21.15.55:0/3094779817 learned_addr learned my addr 172.21.15.55:0/3094779817 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.680+0000 7fe69259c640 1 --2- 172.21.15.55:0/3094779817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6940cd840 0x7fe6940b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.680+0000 7fe692d9d640 1 -- 172.21.15.55:0/3094779817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6940cd840 msgr2=0x7fe6940b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.680+0000 7fe692d9d640 1 --2- 172.21.15.55:0/3094779817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6940cd840 0x7fe6940b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.680+0000 7fe692d9d640 1 -- 172.21.15.55:0/3094779817 >> v1:172.21.15.55:6789/0 conn(0x7fe6940b5830 legacy=0x7fe6940d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.680+0000 7fe69159a640 1 -- 172.21.15.55:0/3094779817 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3250081150 0 0) 0x7fe69415da90 con 0x7fe6940b5830 2024-05-20T22:08:44.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.680+0000 7fe692d9d640 1 -- 172.21.15.55:0/3094779817 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6940b6060 con 0x7fe6940d2e10 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.681+0000 7fe692d9d640 1 --2- 172.21.15.55:0/3094779817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940d2e10 0x7fe6940d1890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe6940b6b10 tx=0x7fe68c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaeff080b1a9747 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.681+0000 7fe69159a640 1 -- 172.21.15.55:0/3094779817 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c031600 con 0x7fe6940d2e10 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.681+0000 7fe69159a640 1 -- 172.21.15.55:0/3094779817 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68c0317a0 con 0x7fe6940d2e10 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.681+0000 7fe69159a640 1 -- 172.21.15.55:0/3094779817 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c031b70 con 0x7fe6940d2e10 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.681+0000 7fe698db7640 1 -- 172.21.15.55:0/3094779817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940d2e10 msgr2=0x7fe6940d1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.681+0000 7fe698db7640 1 --2- 172.21.15.55:0/3094779817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940d2e10 0x7fe6940d1890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe6940b6b10 tx=0x7fe68c02a5d0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.682+0000 7fe698db7640 1 -- 172.21.15.55:0/3094779817 shutdown_connections 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.682+0000 7fe698db7640 1 --2- 172.21.15.55:0/3094779817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940d2e10 0x7fe6940d1890 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.682+0000 7fe698db7640 1 --2- 172.21.15.55:0/3094779817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6940cd840 0x7fe6940b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.682+0000 7fe698db7640 1 -- 172.21.15.55:0/3094779817 >> 172.21.15.55:0/3094779817 conn(0x7fe69408bff0 msgr2=0x7fe69408c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:44.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.682+0000 7fe698db7640 1 -- 172.21.15.55:0/3094779817 shutdown_connections 2024-05-20T22:08:44.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.682+0000 7fe698db7640 1 -- 172.21.15.55:0/3094779817 wait complete. 2024-05-20T22:08:44.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.683+0000 7fe698db7640 1 Processor -- start 2024-05-20T22:08:44.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.684+0000 7fe698db7640 1 -- start start 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe698db7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940b5830 0x7fe6941530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe698db7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6940cd840 0x7fe69406e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe698db7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6940d2e10 0x7fe6940b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe698db7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe6940b8160 con 0x7fe6940b5830 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe698db7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6940b6620 con 0x7fe6940cd840 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe698db7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6940b6e20 con 0x7fe6940d2e10 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe69259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940b5830 0x7fe6941530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe69259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940b5830 0x7fe6941530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49666/0 (socket says 172.21.15.55:49666) 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe69259c640 1 -- 172.21.15.55:0/982975672 learned_addr learned my addr 172.21.15.55:0/982975672 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe691d9b640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6940cd840 0x7fe69406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe692d9d640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6940d2e10 0x7fe6940b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe69259c640 1 -- 172.21.15.55:0/982975672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6940d2e10 msgr2=0x7fe6940b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe69259c640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6940d2e10 0x7fe6940b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe69259c640 1 -- 172.21.15.55:0/982975672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6940cd840 msgr2=0x7fe69406e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe69259c640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6940cd840 0x7fe69406e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe69259c640 1 -- 172.21.15.55:0/982975672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6941538c0 con 0x7fe6940b5830 2024-05-20T22:08:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe691d9b640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6940cd840 0x7fe69406e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.685+0000 7fe69259c640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940b5830 0x7fe6941530a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe68400d830 tx=0x7fe68400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.686+0000 7fe682ffd640 1 -- 172.21.15.55:0/982975672 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684005990 con 0x7fe6940b5830 2024-05-20T22:08:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.686+0000 7fe682ffd640 1 -- 172.21.15.55:0/982975672 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe684005b50 con 0x7fe6940b5830 2024-05-20T22:08:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.686+0000 7fe682ffd640 1 -- 172.21.15.55:0/982975672 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6840096a0 con 0x7fe6940b5830 2024-05-20T22:08:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.686+0000 7fe698db7640 1 -- 172.21.15.55:0/982975672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6940cbea0 con 0x7fe6940b5830 2024-05-20T22:08:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.686+0000 7fe698db7640 1 -- 172.21.15.55:0/982975672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe694160620 con 0x7fe6940b5830 2024-05-20T22:08:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.687+0000 7fe698db7640 1 -- 172.21.15.55:0/982975672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe694069090 con 0x7fe6940b5830 2024-05-20T22:08:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.688+0000 7fe682ffd640 1 -- 172.21.15.55:0/982975672 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe684009a50 con 0x7fe6940b5830 2024-05-20T22:08:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.688+0000 7fe682ffd640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe67406be30 0x7fe67406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.688+0000 7fe682ffd640 1 -- 172.21.15.55:0/982975672 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe68408b6c0 con 0x7fe6940b5830 2024-05-20T22:08:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.688+0000 7fe691d9b640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe67406be30 0x7fe67406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.689+0000 7fe691d9b640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe67406be30 0x7fe67406e340 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe688000f60 tx=0x7fe688007cc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.693+0000 7fe682ffd640 1 -- 172.21.15.55:0/982975672 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe684059d00 con 0x7fe6940b5830 2024-05-20T22:08:44.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.876+0000 7fe698db7640 1 -- 172.21.15.55:0/982975672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe69415ce80 con 0x7fe6940b5830 2024-05-20T22:08:44.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.878+0000 7fe682ffd640 1 -- 172.21.15.55:0/982975672 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7fe684059600 con 0x7fe6940b5830 2024-05-20T22:08:44.879 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:44.879 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:44.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 -- 172.21.15.55:0/982975672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe67406be30 msgr2=0x7fe67406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe67406be30 0x7fe67406e340 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe688000f60 tx=0x7fe688007cc0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 -- 172.21.15.55:0/982975672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940b5830 msgr2=0x7fe6941530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:44.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940b5830 0x7fe6941530a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe68400d830 tx=0x7fe68400dcf0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 -- 172.21.15.55:0/982975672 shutdown_connections 2024-05-20T22:08:44.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe67406be30 0x7fe67406e340 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6940d2e10 0x7fe6940b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6940cd840 0x7fe69406e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 --2- 172.21.15.55:0/982975672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6940b5830 0x7fe6941530a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:44.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 -- 172.21.15.55:0/982975672 >> 172.21.15.55:0/982975672 conn(0x7fe69408bff0 msgr2=0x7fe6940707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:44.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 -- 172.21.15.55:0/982975672 shutdown_connections 2024-05-20T22:08:44.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:44.880+0000 7fe698db7640 1 -- 172.21.15.55:0/982975672 wait complete. 2024-05-20T22:08:44.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:44.893 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.010+0000 7f9bc8651640 1 Processor -- start 2024-05-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.010+0000 7f9bc8651640 1 -- start start 2024-05-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.011+0000 7f9bc8651640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9bc00cd520 0x7f9bc008c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.011+0000 7f9bc8651640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9bc00cf270 0x7f9bc00cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.011+0000 7f9bc8651640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc00cfbc0 con 0x7f9bc00d0860 2024-05-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.011+0000 7f9bc8651640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc00d1020 con 0x7f9bc00cd520 2024-05-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.011+0000 7f9bc8651640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc00d1300 con 0x7f9bc00cf270 2024-05-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.011+0000 7f9bc63ed640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9bc00cd520 0x7f9bc008c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.012+0000 7f9bc5bec640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9bc00d0860 0x7f9bc00b19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60506/0 (socket says 172.21.15.55:60506) 2024-05-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.012+0000 7f9bc5bec640 1 -- 172.21.15.55:0/3581321033 learned_addr learned my addr 172.21.15.55:0/3581321033 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.012+0000 7f9bc6bee640 1 --2- 172.21.15.55:0/3581321033 >> v2:172.21.15.138:3300/0 conn(0x7f9bc00cf270 0x7f9bc00cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.012+0000 7f9bc63ed640 1 -- 172.21.15.55:0/3581321033 >> v2:172.21.15.138:3300/0 conn(0x7f9bc00cf270 msgr2=0x7f9bc00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.012+0000 7f9bc63ed640 1 --2- 172.21.15.55:0/3581321033 >> v2:172.21.15.138:3300/0 conn(0x7f9bc00cf270 0x7f9bc00cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.012+0000 7f9bc63ed640 1 -- 172.21.15.55:0/3581321033 >> v1:172.21.15.55:6789/0 conn(0x7f9bc00d0860 legacy=0x7f9bc00b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.012+0000 7f9bc63ed640 1 -- 172.21.15.55:0/3581321033 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc00d15e0 con 0x7f9bc00cd520 2024-05-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.012+0000 7f9bc53eb640 1 -- 172.21.15.55:0/3581321033 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151111779 0 0) 0x7f9bb0001110 con 0x7f9bc00d0860 2024-05-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.013+0000 7f9bc63ed640 1 --2- 172.21.15.55:0/3581321033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 0x7f9bc008c960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00d1b30 tx=0x7f9bbc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c676f80442a812 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.013+0000 7f9bc53eb640 1 -- 172.21.15.55:0/3581321033 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc037070 con 0x7f9bc00cd520 2024-05-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.013+0000 7f9bc53eb640 1 -- 172.21.15.55:0/3581321033 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bbc032440 con 0x7f9bc00cd520 2024-05-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.013+0000 7f9bc53eb640 1 -- 172.21.15.55:0/3581321033 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc02ae60 con 0x7f9bc00cd520 2024-05-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.013+0000 7f9bc8651640 1 -- 172.21.15.55:0/3581321033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 msgr2=0x7f9bc008c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.013+0000 7f9bc8651640 1 --2- 172.21.15.55:0/3581321033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 0x7f9bc008c960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00d1b30 tx=0x7f9bbc02a110 comp rx=0 tx=0).stop 2024-05-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.014+0000 7f9bc8651640 1 -- 172.21.15.55:0/3581321033 shutdown_connections 2024-05-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.014+0000 7f9bc8651640 1 --2- 172.21.15.55:0/3581321033 >> v2:172.21.15.138:3300/0 conn(0x7f9bc00cf270 0x7f9bc00cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.014+0000 7f9bc8651640 1 --2- 172.21.15.55:0/3581321033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 0x7f9bc008c960 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.014+0000 7f9bc8651640 1 -- 172.21.15.55:0/3581321033 >> 172.21.15.55:0/3581321033 conn(0x7f9bc00d2860 msgr2=0x7f9bc00d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.014+0000 7f9bc8651640 1 -- 172.21.15.55:0/3581321033 shutdown_connections 2024-05-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.014+0000 7f9bc8651640 1 -- 172.21.15.55:0/3581321033 wait complete. 2024-05-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.015+0000 7f9bc8651640 1 Processor -- start 2024-05-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.015+0000 7f9bc8651640 1 -- start start 2024-05-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc8651640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 0x7f9bc015db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc8651640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00cf270 0x7f9bc00cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc8651640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bc00d0860 0x7f9bc00cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc8651640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc00d0c70 con 0x7f9bc00cd520 2024-05-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc8651640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc00b2cb0 con 0x7f9bc00d0860 2024-05-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc8651640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc015f430 con 0x7f9bc00cf270 2024-05-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc63ed640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 0x7f9bc015db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc63ed640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 0x7f9bc015db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49672/0 (socket says 172.21.15.55:49672) 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc63ed640 1 -- 172.21.15.55:0/1481193330 learned_addr learned my addr 172.21.15.55:0/1481193330 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc5bec640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00cf270 0x7f9bc00cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc6bee640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bc00d0860 0x7f9bc00cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc63ed640 1 -- 172.21.15.55:0/1481193330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00cf270 msgr2=0x7f9bc00cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc63ed640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00cf270 0x7f9bc00cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc63ed640 1 -- 172.21.15.55:0/1481193330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bc00d0860 msgr2=0x7f9bc00cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc63ed640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bc00d0860 0x7f9bc00cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc63ed640 1 -- 172.21.15.55:0/1481193330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc015e8b0 con 0x7f9bc00cd520 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.017+0000 7f9bc63ed640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 0x7f9bc015db40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc000c00 tx=0x7f9bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.018+0000 7f9bb6ffd640 1 -- 172.21.15.55:0/1481193330 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc037040 con 0x7f9bc00cd520 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.018+0000 7f9bb6ffd640 1 -- 172.21.15.55:0/1481193330 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bbc042070 con 0x7f9bc00cd520 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.018+0000 7f9bc8651640 1 -- 172.21.15.55:0/1481193330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc015eb90 con 0x7f9bc00cd520 2024-05-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.018+0000 7f9bb6ffd640 1 -- 172.21.15.55:0/1481193330 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc030a50 con 0x7f9bc00cd520 2024-05-20T22:08:45.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.018+0000 7f9bc8651640 1 -- 172.21.15.55:0/1481193330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc015f150 con 0x7f9bc00cd520 2024-05-20T22:08:45.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.019+0000 7f9bc8651640 1 -- 172.21.15.55:0/1481193330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc0069090 con 0x7f9bc00cd520 2024-05-20T22:08:45.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.019+0000 7f9bb6ffd640 1 -- 172.21.15.55:0/1481193330 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bbc030bf0 con 0x7f9bc00cd520 2024-05-20T22:08:45.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.020+0000 7f9bb6ffd640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9ba406be60 0x7f9ba406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.020+0000 7f9bb6ffd640 1 -- 172.21.15.55:0/1481193330 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9bbc0ab9f0 con 0x7f9bc00cd520 2024-05-20T22:08:45.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.020+0000 7f9bc5bec640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9ba406be60 0x7f9ba406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.020+0000 7f9bc5bec640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9ba406be60 0x7f9ba406e370 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00afca0 tx=0x7f9bb0007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.024+0000 7f9bb6ffd640 1 -- 172.21.15.55:0/1481193330 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9bbc04c020 con 0x7f9bc00cd520 2024-05-20T22:08:45.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.207+0000 7f9bc8651640 1 -- 172.21.15.55:0/1481193330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9bc00cce00 con 0x7f9bc00cd520 2024-05-20T22:08:45.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.208+0000 7f9bb6ffd640 1 -- 172.21.15.55:0/1481193330 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f9bbc079950 con 0x7f9bc00cd520 2024-05-20T22:08:45.209 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:45.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 -- 172.21.15.55:0/1481193330 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9ba406be60 msgr2=0x7f9ba406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9ba406be60 0x7f9ba406e370 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00afca0 tx=0x7f9bb0007de0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 -- 172.21.15.55:0/1481193330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 msgr2=0x7f9bc015db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 0x7f9bc015db40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc000c00 tx=0x7f9bbc0046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 -- 172.21.15.55:0/1481193330 shutdown_connections 2024-05-20T22:08:45.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9ba406be60 0x7f9ba406e370 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bc00d0860 0x7f9bc00cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00cf270 0x7f9bc00cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 --2- 172.21.15.55:0/1481193330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bc00cd520 0x7f9bc015db40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 -- 172.21.15.55:0/1481193330 >> 172.21.15.55:0/1481193330 conn(0x7f9bc00d2860 msgr2=0x7f9bc00706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:45.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 -- 172.21.15.55:0/1481193330 shutdown_connections 2024-05-20T22:08:45.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.210+0000 7f9bc8651640 1 -- 172.21.15.55:0/1481193330 wait complete. 2024-05-20T22:08:45.222 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:45.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.339+0000 7f1c3a321640 1 Processor -- start 2024-05-20T22:08:45.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.339+0000 7f1c3a321640 1 -- start start 2024-05-20T22:08:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c3a321640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340d32a0 0x7f1c3415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c3a321640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1c340cf690 0x7f1c340cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c3a321640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c340d3780 con 0x7f1c340cffe0 2024-05-20T22:08:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c3a321640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1c3406c510 con 0x7f1c340cf690 2024-05-20T22:08:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c3a321640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c340d3a60 con 0x7f1c340d32a0 2024-05-20T22:08:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c33fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340d32a0 0x7f1c3415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c337fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1c340cf690 0x7f1c340cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c33fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340d32a0 0x7f1c3415f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:39098/0 (socket says 172.21.15.55:39098) 2024-05-20T22:08:45.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c33fff640 1 -- 172.21.15.55:0/3581398739 learned_addr learned my addr 172.21.15.55:0/3581398739 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:45.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c33fff640 1 -- 172.21.15.55:0/3581398739 >> v2:172.21.15.138:3300/0 conn(0x7f1c340cf690 msgr2=0x7f1c340cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:45.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c33fff640 1 --2- 172.21.15.55:0/3581398739 >> v2:172.21.15.138:3300/0 conn(0x7f1c340cf690 0x7f1c340cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c33fff640 1 -- 172.21.15.55:0/3581398739 >> v1:172.21.15.138:6789/0 conn(0x7f1c340cffe0 legacy=0x7f1c34162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.341+0000 7f1c33fff640 1 -- 172.21.15.55:0/3581398739 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c340d3d40 con 0x7f1c340d32a0 2024-05-20T22:08:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.342+0000 7f1c33fff640 1 --2- 172.21.15.55:0/3581398739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340d32a0 0x7f1c3415f150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009870 tx=0x7f1c2402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c39bfe015fd58a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.342+0000 7f1c32ffd640 1 -- 172.21.15.55:0/3581398739 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24037070 con 0x7f1c340d32a0 2024-05-20T22:08:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.342+0000 7f1c32ffd640 1 -- 172.21.15.55:0/3581398739 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c24032440 con 0x7f1c340d32a0 2024-05-20T22:08:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.342+0000 7f1c32ffd640 1 -- 172.21.15.55:0/3581398739 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24032810 con 0x7f1c340d32a0 2024-05-20T22:08:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.343+0000 7f1c3a321640 1 -- 172.21.15.55:0/3581398739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340d32a0 msgr2=0x7f1c3415f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.343+0000 7f1c3a321640 1 --2- 172.21.15.55:0/3581398739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340d32a0 0x7f1c3415f150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009870 tx=0x7f1c2402a250 comp rx=0 tx=0).stop 2024-05-20T22:08:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.343+0000 7f1c3a321640 1 -- 172.21.15.55:0/3581398739 shutdown_connections 2024-05-20T22:08:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.343+0000 7f1c3a321640 1 --2- 172.21.15.55:0/3581398739 >> v2:172.21.15.138:3300/0 conn(0x7f1c340cf690 0x7f1c340cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.343+0000 7f1c3a321640 1 --2- 172.21.15.55:0/3581398739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340d32a0 0x7f1c3415f150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.343+0000 7f1c3a321640 1 -- 172.21.15.55:0/3581398739 >> 172.21.15.55:0/3581398739 conn(0x7f1c340b6de0 msgr2=0x7f1c340d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.343+0000 7f1c3a321640 1 -- 172.21.15.55:0/3581398739 shutdown_connections 2024-05-20T22:08:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.343+0000 7f1c3a321640 1 -- 172.21.15.55:0/3581398739 wait complete. 2024-05-20T22:08:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.345+0000 7f1c3a321640 1 Processor -- start 2024-05-20T22:08:45.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.345+0000 7f1c3a321640 1 -- start start 2024-05-20T22:08:45.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c3a321640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340cf690 0x7f1c340c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c3a321640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340cffe0 0x7f1c340cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c3a321640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c340d32a0 0x7f1c3415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c3a321640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c340d03f0 con 0x7f1c340cffe0 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c3a321640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c340d4300 con 0x7f1c340d32a0 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c3a321640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c340d4820 con 0x7f1c340cf690 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c337fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340cffe0 0x7f1c340cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c337fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340cffe0 0x7f1c340cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49680/0 (socket says 172.21.15.55:49680) 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c337fe640 1 -- 172.21.15.55:0/2699275089 learned_addr learned my addr 172.21.15.55:0/2699275089 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c33fff640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340cf690 0x7f1c340c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c388be640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c340d32a0 0x7f1c3415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c337fe640 1 -- 172.21.15.55:0/2699275089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340cf690 msgr2=0x7f1c340c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c337fe640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340cf690 0x7f1c340c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c337fe640 1 -- 172.21.15.55:0/2699275089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c340d32a0 msgr2=0x7f1c3415f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c337fe640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c340d32a0 0x7f1c3415f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c337fe640 1 -- 172.21.15.55:0/2699275089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c340ccd20 con 0x7f1c340cffe0 2024-05-20T22:08:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.347+0000 7f1c337fe640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340cffe0 0x7f1c340cc3f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800e8a0 tx=0x7f1c2800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.348+0000 7f1c30ff9640 1 -- 172.21.15.55:0/2699275089 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28017070 con 0x7f1c340cffe0 2024-05-20T22:08:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.348+0000 7f1c30ff9640 1 -- 172.21.15.55:0/2699275089 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2800c8e0 con 0x7f1c340cffe0 2024-05-20T22:08:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.348+0000 7f1c3a321640 1 -- 172.21.15.55:0/2699275089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c340cd000 con 0x7f1c340cffe0 2024-05-20T22:08:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.348+0000 7f1c30ff9640 1 -- 172.21.15.55:0/2699275089 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800ccd0 con 0x7f1c340cffe0 2024-05-20T22:08:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.348+0000 7f1c3a321640 1 -- 172.21.15.55:0/2699275089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c340cd5c0 con 0x7f1c340cffe0 2024-05-20T22:08:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.349+0000 7f1c3a321640 1 -- 172.21.15.55:0/2699275089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c34069090 con 0x7f1c340cffe0 2024-05-20T22:08:45.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.349+0000 7f1c30ff9640 1 -- 172.21.15.55:0/2699275089 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c2801a070 con 0x7f1c340cffe0 2024-05-20T22:08:45.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.350+0000 7f1c30ff9640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c2006be30 0x7f1c2006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.350+0000 7f1c30ff9640 1 -- 172.21.15.55:0/2699275089 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1c2808b3e0 con 0x7f1c340cffe0 2024-05-20T22:08:45.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.350+0000 7f1c33fff640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c2006be30 0x7f1c2006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.350+0000 7f1c33fff640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c2006be30 0x7f1c2006e340 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1c24034000 tx=0x7f1c2402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.354+0000 7f1c30ff9640 1 -- 172.21.15.55:0/2699275089 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c28059340 con 0x7f1c340cffe0 2024-05-20T22:08:45.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.540+0000 7f1c3a321640 1 -- 172.21.15.55:0/2699275089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c341626e0 con 0x7f1c340cffe0 2024-05-20T22:08:45.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.541+0000 7f1c30ff9640 1 -- 172.21.15.55:0/2699275089 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f1c2805d640 con 0x7f1c340cffe0 2024-05-20T22:08:45.542 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.542+0000 7f1c3a321640 1 -- 172.21.15.55:0/2699275089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c2006be30 msgr2=0x7f1c2006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c2006be30 0x7f1c2006e340 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1c24034000 tx=0x7f1c2402e000 comp rx=0 tx=0).stop 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 -- 172.21.15.55:0/2699275089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340cffe0 msgr2=0x7f1c340cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340cffe0 0x7f1c340cc3f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800e8a0 tx=0x7f1c2800ed60 comp rx=0 tx=0).stop 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 -- 172.21.15.55:0/2699275089 shutdown_connections 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c2006be30 0x7f1c2006e340 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c340d32a0 0x7f1c3415f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340cffe0 0x7f1c340cc3f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 --2- 172.21.15.55:0/2699275089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340cf690 0x7f1c340c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 -- 172.21.15.55:0/2699275089 >> 172.21.15.55:0/2699275089 conn(0x7f1c340b6de0 msgr2=0x7f1c340b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 -- 172.21.15.55:0/2699275089 shutdown_connections 2024-05-20T22:08:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.543+0000 7f1c3a321640 1 -- 172.21.15.55:0/2699275089 wait complete. 2024-05-20T22:08:45.555 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:45.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.672+0000 7efe24244640 1 Processor -- start 2024-05-20T22:08:45.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.672+0000 7efe24244640 1 -- start start 2024-05-20T22:08:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.673+0000 7efe24244640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c167ec0 0x7efe1c16a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.673+0000 7efe24244640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1c06b500 con 0x7efe1c0b4080 2024-05-20T22:08:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.673+0000 7efe24244640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1c0b3e10 con 0x7efe1c0b35e0 2024-05-20T22:08:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.673+0000 7efe24244640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe1c163960 con 0x7efe1c167ec0 2024-05-20T22:08:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe21fe0640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efe1c0b4080 0x7efe1c15f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60520/0 (socket says 172.21.15.55:60520) 2024-05-20T22:08:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe21fe0640 1 -- 172.21.15.55:0/677087391 learned_addr learned my addr 172.21.15.55:0/677087391 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe227e1640 1 --2- 172.21.15.55:0/677087391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c167ec0 0x7efe1c16a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe20fde640 1 -- 172.21.15.55:0/677087391 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2109635928 0 0) 0x7efe18001110 con 0x7efe1c0b4080 2024-05-20T22:08:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe20fde640 1 -- 172.21.15.55:0/677087391 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe08003520 con 0x7efe1c0b4080 2024-05-20T22:08:45.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe20fde640 1 -- 172.21.15.55:0/677087391 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3032284380 0 0) 0x7efe1c0b3e10 con 0x7efe1c0b35e0 2024-05-20T22:08:45.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe20fde640 1 -- 172.21.15.55:0/677087391 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe18001110 con 0x7efe1c0b35e0 2024-05-20T22:08:45.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe20fde640 1 -- 172.21.15.55:0/677087391 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 384653662 0 0) 0x7efe08003520 con 0x7efe1c0b4080 2024-05-20T22:08:45.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe227e1640 1 -- 172.21.15.55:0/677087391 >> v1:172.21.15.138:6789/0 conn(0x7efe1c0b35e0 legacy=0x7efe1c15f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe227e1640 1 -- 172.21.15.55:0/677087391 >> v1:172.21.15.55:6789/0 conn(0x7efe1c0b4080 legacy=0x7efe1c15f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.674+0000 7efe227e1640 1 -- 172.21.15.55:0/677087391 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c0b3b30 con 0x7efe1c167ec0 2024-05-20T22:08:45.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.675+0000 7efe227e1640 1 --2- 172.21.15.55:0/677087391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c167ec0 0x7efe1c16a2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe14009dc0 tx=0x7efe1402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23248c755c71202a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:45.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.675+0000 7efe20fde640 1 -- 172.21.15.55:0/677087391 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18001840 con 0x7efe1c167ec0 2024-05-20T22:08:45.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.675+0000 7efe20fde640 1 -- 172.21.15.55:0/677087391 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe14007540 con 0x7efe1c167ec0 2024-05-20T22:08:45.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.675+0000 7efe20fde640 1 -- 172.21.15.55:0/677087391 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe140078f0 con 0x7efe1c167ec0 2024-05-20T22:08:45.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.676+0000 7efe24244640 1 -- 172.21.15.55:0/677087391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c167ec0 msgr2=0x7efe1c16a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.676+0000 7efe24244640 1 --2- 172.21.15.55:0/677087391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c167ec0 0x7efe1c16a2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe14009dc0 tx=0x7efe1402a660 comp rx=0 tx=0).stop 2024-05-20T22:08:45.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.676+0000 7efe24244640 1 -- 172.21.15.55:0/677087391 shutdown_connections 2024-05-20T22:08:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.676+0000 7efe24244640 1 --2- 172.21.15.55:0/677087391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c167ec0 0x7efe1c16a2e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.676+0000 7efe24244640 1 -- 172.21.15.55:0/677087391 >> 172.21.15.55:0/677087391 conn(0x7efe1c0d51b0 msgr2=0x7efe1c0b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.676+0000 7efe24244640 1 -- 172.21.15.55:0/677087391 shutdown_connections 2024-05-20T22:08:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.676+0000 7efe24244640 1 -- 172.21.15.55:0/677087391 wait complete. 2024-05-20T22:08:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.678+0000 7efe24244640 1 Processor -- start 2024-05-20T22:08:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.678+0000 7efe24244640 1 -- start start 2024-05-20T22:08:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe24244640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c0b35e0 0x7efe1c0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe24244640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe1c0b4080 0x7efe1c0b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe24244640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c167ec0 0x7efe1c0b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe24244640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c1635f0 con 0x7efe1c0b4080 2024-05-20T22:08:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe24244640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c1604f0 con 0x7efe1c167ec0 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe24244640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe1c0d42a0 con 0x7efe1c0b35e0 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe217df640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe1c0b4080 0x7efe1c0b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe217df640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe1c0b4080 0x7efe1c0b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49696/0 (socket says 172.21.15.55:49696) 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe217df640 1 -- 172.21.15.55:0/875916174 learned_addr learned my addr 172.21.15.55:0/875916174 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe227e1640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c167ec0 0x7efe1c0b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.679+0000 7efe21fe0640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c0b35e0 0x7efe1c0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe217df640 1 -- 172.21.15.55:0/875916174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c0b35e0 msgr2=0x7efe1c0cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe217df640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c0b35e0 0x7efe1c0cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe217df640 1 -- 172.21.15.55:0/875916174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c167ec0 msgr2=0x7efe1c0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe217df640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c167ec0 0x7efe1c0b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe217df640 1 -- 172.21.15.55:0/875916174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c0c9600 con 0x7efe1c0b4080 2024-05-20T22:08:45.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe21fe0640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c0b35e0 0x7efe1c0cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:45.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe227e1640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c167ec0 0x7efe1c0b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:45.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe217df640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe1c0b4080 0x7efe1c0b5550 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00c460 tx=0x7efe0c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:45.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe127fc640 1 -- 172.21.15.55:0/875916174 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c017070 con 0x7efe1c0b4080 2024-05-20T22:08:45.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe127fc640 1 -- 172.21.15.55:0/875916174 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0c0052f0 con 0x7efe1c0b4080 2024-05-20T22:08:45.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe127fc640 1 -- 172.21.15.55:0/875916174 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c003830 con 0x7efe1c0b4080 2024-05-20T22:08:45.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe24244640 1 -- 172.21.15.55:0/875916174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c0c98e0 con 0x7efe1c0b4080 2024-05-20T22:08:45.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.680+0000 7efe24244640 1 -- 172.21.15.55:0/875916174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c0c9ea0 con 0x7efe1c0b4080 2024-05-20T22:08:45.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.681+0000 7efe24244640 1 -- 172.21.15.55:0/875916174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1c069090 con 0x7efe1c0b4080 2024-05-20T22:08:45.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.682+0000 7efe127fc640 1 -- 172.21.15.55:0/875916174 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe0c01a070 con 0x7efe1c0b4080 2024-05-20T22:08:45.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.682+0000 7efe127fc640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe0006be30 0x7efe0006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:45.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.683+0000 7efe127fc640 1 -- 172.21.15.55:0/875916174 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efe0c08a6e0 con 0x7efe1c0b4080 2024-05-20T22:08:45.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.683+0000 7efe21fe0640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe0006be30 0x7efe0006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:45.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.683+0000 7efe21fe0640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe0006be30 0x7efe0006e340 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efe18005880 tx=0x7efe18009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:45.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.687+0000 7efe127fc640 1 -- 172.21.15.55:0/875916174 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efe0c058bf0 con 0x7efe1c0b4080 2024-05-20T22:08:45.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.876+0000 7efe24244640 1 -- 172.21.15.55:0/875916174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efe1c0d2ec0 con 0x7efe1c0b4080 2024-05-20T22:08:45.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.877+0000 7efe127fc640 1 -- 172.21.15.55:0/875916174 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7efe0c0584f0 con 0x7efe1c0b4080 2024-05-20T22:08:45.878 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:45.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 -- 172.21.15.55:0/875916174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe0006be30 msgr2=0x7efe0006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe0006be30 0x7efe0006e340 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efe18005880 tx=0x7efe18009040 comp rx=0 tx=0).stop 2024-05-20T22:08:45.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 -- 172.21.15.55:0/875916174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe1c0b4080 msgr2=0x7efe1c0b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:45.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe1c0b4080 0x7efe1c0b5550 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00c460 tx=0x7efe0c00c920 comp rx=0 tx=0).stop 2024-05-20T22:08:45.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 -- 172.21.15.55:0/875916174 shutdown_connections 2024-05-20T22:08:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe0006be30 0x7efe0006e340 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe1c167ec0 0x7efe1c0b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe1c0b4080 0x7efe1c0b5550 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 --2- 172.21.15.55:0/875916174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe1c0b35e0 0x7efe1c0cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 -- 172.21.15.55:0/875916174 >> 172.21.15.55:0/875916174 conn(0x7efe1c0d51b0 msgr2=0x7efe1c160700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 -- 172.21.15.55:0/875916174 shutdown_connections 2024-05-20T22:08:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:45.879+0000 7efe24244640 1 -- 172.21.15.55:0/875916174 wait complete. 2024-05-20T22:08:45.891 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:46.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.010+0000 7fb4a9d3e640 1 Processor -- start 2024-05-20T22:08:46.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.010+0000 7fb4a9d3e640 1 -- start start 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.011+0000 7fb4a9d3e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb4a409ad30 0x7fb4a415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.011+0000 7fb4a9d3e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a4157780 0x7fb4a40d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.011+0000 7fb4a9d3e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4a40d1c10 0x7fb4a4160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.011+0000 7fb4a9d3e640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a406e7e0 con 0x7fb4a409ad30 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.011+0000 7fb4a9d3e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a40cd6c0 con 0x7fb4a40d1c10 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.011+0000 7fb4a9d3e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a415e540 con 0x7fb4a4157780 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.011+0000 7fb4a37fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb4a409ad30 0x7fb4a415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.011+0000 7fb4a37fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb4a409ad30 0x7fb4a415e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49704/0 (socket says 172.21.15.55:49704) 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.011+0000 7fb4a37fe640 1 -- 172.21.15.55:0/3637706850 learned_addr learned my addr 172.21.15.55:0/3637706850 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.011+0000 7fb4a2ffd640 1 --2- 172.21.15.55:0/3637706850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a4157780 0x7fb4a40d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.012+0000 7fb4a3fff640 1 --2- 172.21.15.55:0/3637706850 >> v2:172.21.15.138:3300/0 conn(0x7fb4a40d1c10 0x7fb4a4160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.012+0000 7fb4a37fe640 1 -- 172.21.15.55:0/3637706850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a4157780 msgr2=0x7fb4a40d16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.012+0000 7fb4a37fe640 1 --2- 172.21.15.55:0/3637706850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a4157780 0x7fb4a40d16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.012+0000 7fb4a37fe640 1 -- 172.21.15.55:0/3637706850 >> v2:172.21.15.138:3300/0 conn(0x7fb4a40d1c10 msgr2=0x7fb4a4160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.012+0000 7fb4a37fe640 1 --2- 172.21.15.55:0/3637706850 >> v2:172.21.15.138:3300/0 conn(0x7fb4a40d1c10 0x7fb4a4160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.012+0000 7fb4a37fe640 1 -- 172.21.15.55:0/3637706850 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a40d2020 con 0x7fb4a409ad30 2024-05-20T22:08:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.012+0000 7fb4a2ffd640 1 --2- 172.21.15.55:0/3637706850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a4157780 0x7fb4a40d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:46.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.012+0000 7fb4a37fe640 1 --2- 172.21.15.55:0/3637706850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a409ad30 0x7fb4a415e000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4072990 tx=0x7fb49402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46db83ced21a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:46.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.012+0000 7fb4a27fc640 1 -- 172.21.15.55:0/3637706850 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494037070 con 0x7fb4a409ad30 2024-05-20T22:08:46.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a27fc640 1 -- 172.21.15.55:0/3637706850 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb494032440 con 0x7fb4a409ad30 2024-05-20T22:08:46.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a27fc640 1 -- 172.21.15.55:0/3637706850 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494032810 con 0x7fb4a409ad30 2024-05-20T22:08:46.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/3637706850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a409ad30 msgr2=0x7fb4a415e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a9d3e640 1 --2- 172.21.15.55:0/3637706850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a409ad30 0x7fb4a415e000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4072990 tx=0x7fb49402a1e0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/3637706850 shutdown_connections 2024-05-20T22:08:46.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a9d3e640 1 --2- 172.21.15.55:0/3637706850 >> v2:172.21.15.138:3300/0 conn(0x7fb4a40d1c10 0x7fb4a4160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a9d3e640 1 --2- 172.21.15.55:0/3637706850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a4157780 0x7fb4a40d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a9d3e640 1 --2- 172.21.15.55:0/3637706850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a409ad30 0x7fb4a415e000 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/3637706850 >> 172.21.15.55:0/3637706850 conn(0x7fb4a40736f0 msgr2=0x7fb4a40b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:46.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/3637706850 shutdown_connections 2024-05-20T22:08:46.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.013+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/3637706850 wait complete. 2024-05-20T22:08:46.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.015+0000 7fb4a9d3e640 1 Processor -- start 2024-05-20T22:08:46.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.015+0000 7fb4a9d3e640 1 -- start start 2024-05-20T22:08:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a9d3e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a409ad30 0x7fb4a40cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a9d3e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a40d1c10 0x7fb4a40c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a9d3e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a4157780 0x7fb4a414f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a9d3e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a40cd350 con 0x7fb4a4157780 2024-05-20T22:08:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a9d3e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a40b2d20 con 0x7fb4a409ad30 2024-05-20T22:08:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a9d3e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a40b3000 con 0x7fb4a40d1c10 2024-05-20T22:08:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a4157780 0x7fb4a414f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a4157780 0x7fb4a414f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49706/0 (socket says 172.21.15.55:49706) 2024-05-20T22:08:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a3fff640 1 -- 172.21.15.55:0/908188282 learned_addr learned my addr 172.21.15.55:0/908188282 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a2ffd640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a40d1c10 0x7fb4a40c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a37fe640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a409ad30 0x7fb4a40cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a3fff640 1 -- 172.21.15.55:0/908188282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a40d1c10 msgr2=0x7fb4a40c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a3fff640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a40d1c10 0x7fb4a40c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a3fff640 1 -- 172.21.15.55:0/908188282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a409ad30 msgr2=0x7fb4a40cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a3fff640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a409ad30 0x7fb4a40cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a3fff640 1 -- 172.21.15.55:0/908188282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a40c7d30 con 0x7fb4a4157780 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb4a3fff640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a4157780 0x7fb4a414f570 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00cbb0 tx=0x7fb49c00a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb48bfff640 1 -- 172.21.15.55:0/908188282 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c015070 con 0x7fb4a4157780 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.017+0000 7fb48bfff640 1 -- 172.21.15.55:0/908188282 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49c00abd0 con 0x7fb4a4157780 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.018+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/908188282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a40c8010 con 0x7fb4a4157780 2024-05-20T22:08:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.018+0000 7fb48bfff640 1 -- 172.21.15.55:0/908188282 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c00d040 con 0x7fb4a4157780 2024-05-20T22:08:46.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.018+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/908188282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a41e7290 con 0x7fb4a4157780 2024-05-20T22:08:46.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.018+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/908188282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a40c8820 con 0x7fb4a4157780 2024-05-20T22:08:46.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.019+0000 7fb48bfff640 1 -- 172.21.15.55:0/908188282 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb49c011410 con 0x7fb4a4157780 2024-05-20T22:08:46.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.020+0000 7fb48bfff640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb49006be60 0x7fb49006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.020+0000 7fb48bfff640 1 -- 172.21.15.55:0/908188282 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb49c08ae40 con 0x7fb4a4157780 2024-05-20T22:08:46.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.020+0000 7fb4a37fe640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb49006be60 0x7fb49006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.020+0000 7fb4a37fe640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb49006be60 0x7fb49006e370 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb494034000 tx=0x7fb49402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:46.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.024+0000 7fb48bfff640 1 -- 172.21.15.55:0/908188282 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb49c059360 con 0x7fb4a4157780 2024-05-20T22:08:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.208+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/908188282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb4a4069090 con 0x7fb4a4157780 2024-05-20T22:08:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.208+0000 7fb48bfff640 1 -- 172.21.15.55:0/908188282 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb49c058c60 con 0x7fb4a4157780 2024-05-20T22:08:46.209 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/908188282 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb49006be60 msgr2=0x7fb49006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb49006be60 0x7fb49006e370 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb494034000 tx=0x7fb49402e000 comp rx=0 tx=0).stop 2024-05-20T22:08:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/908188282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a4157780 msgr2=0x7fb4a414f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a4157780 0x7fb4a414f570 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00cbb0 tx=0x7fb49c00a4c0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/908188282 shutdown_connections 2024-05-20T22:08:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb49006be60 0x7fb49006e370 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4a4157780 0x7fb4a414f570 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4a40d1c10 0x7fb4a40c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 --2- 172.21.15.55:0/908188282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a409ad30 0x7fb4a40cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/908188282 >> 172.21.15.55:0/908188282 conn(0x7fb4a40736f0 msgr2=0x7fb4a40b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/908188282 shutdown_connections 2024-05-20T22:08:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.211+0000 7fb4a9d3e640 1 -- 172.21.15.55:0/908188282 wait complete. 2024-05-20T22:08:46.223 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:08:46.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.338+0000 7fad5d61b640 1 Processor -- start 2024-05-20T22:08:46.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.338+0000 7fad5d61b640 1 -- start start 2024-05-20T22:08:46.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.339+0000 7fad5d61b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad580d32a0 0x7fad5815f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.339+0000 7fad5d61b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad580cf690 0x7fad580cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad5d61b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad580d3780 con 0x7fad580cffe0 2024-05-20T22:08:46.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad5d61b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fad5806c510 con 0x7fad580d32a0 2024-05-20T22:08:46.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad5d61b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad580d3a60 con 0x7fad580cf690 2024-05-20T22:08:46.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad56ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad580d32a0 0x7fad5815f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad577fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fad580cffe0 0x7fad58162cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60528/0 (socket says 172.21.15.55:60528) 2024-05-20T22:08:46.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad577fe640 1 -- 172.21.15.55:0/3300688603 learned_addr learned my addr 172.21.15.55:0/3300688603 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:46.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad567fc640 1 --2- 172.21.15.55:0/3300688603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad580cf690 0x7fad580cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad56ffd640 1 -- 172.21.15.55:0/3300688603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad580cf690 msgr2=0x7fad580cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:46.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad56ffd640 1 --2- 172.21.15.55:0/3300688603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad580cf690 0x7fad580cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad56ffd640 1 -- 172.21.15.55:0/3300688603 >> v1:172.21.15.55:6789/0 conn(0x7fad580cffe0 legacy=0x7fad58162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad56ffd640 1 -- 172.21.15.55:0/3300688603 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad580d3d40 con 0x7fad580d32a0 2024-05-20T22:08:46.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.340+0000 7fad55ffb640 1 -- 172.21.15.55:0/3300688603 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4263106564 0 0) 0x7fad580d3780 con 0x7fad580cffe0 2024-05-20T22:08:46.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.341+0000 7fad56ffd640 1 --2- 172.21.15.55:0/3300688603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580d32a0 0x7fad5815f150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fad480097a0 tx=0x7fad4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625bf5317d10ef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:46.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.341+0000 7fad55ffb640 1 -- 172.21.15.55:0/3300688603 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48037070 con 0x7fad580d32a0 2024-05-20T22:08:46.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.341+0000 7fad55ffb640 1 -- 172.21.15.55:0/3300688603 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad48032440 con 0x7fad580d32a0 2024-05-20T22:08:46.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.341+0000 7fad55ffb640 1 -- 172.21.15.55:0/3300688603 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48032810 con 0x7fad580d32a0 2024-05-20T22:08:46.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.342+0000 7fad5d61b640 1 -- 172.21.15.55:0/3300688603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580d32a0 msgr2=0x7fad5815f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.342+0000 7fad5d61b640 1 --2- 172.21.15.55:0/3300688603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580d32a0 0x7fad5815f150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fad480097a0 tx=0x7fad4802a1a0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.342+0000 7fad5d61b640 1 -- 172.21.15.55:0/3300688603 shutdown_connections 2024-05-20T22:08:46.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.342+0000 7fad5d61b640 1 --2- 172.21.15.55:0/3300688603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad580cf690 0x7fad580cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.342+0000 7fad5d61b640 1 --2- 172.21.15.55:0/3300688603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580d32a0 0x7fad5815f150 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.342+0000 7fad5d61b640 1 -- 172.21.15.55:0/3300688603 >> 172.21.15.55:0/3300688603 conn(0x7fad580b6de0 msgr2=0x7fad580d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:46.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.342+0000 7fad5d61b640 1 -- 172.21.15.55:0/3300688603 shutdown_connections 2024-05-20T22:08:46.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.342+0000 7fad5d61b640 1 -- 172.21.15.55:0/3300688603 wait complete. 2024-05-20T22:08:46.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.344+0000 7fad5d61b640 1 Processor -- start 2024-05-20T22:08:46.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.344+0000 7fad5d61b640 1 -- start start 2024-05-20T22:08:46.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad5d61b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580cf690 0x7fad58156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad5d61b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad580cffe0 0x7fad580cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad5d61b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad580d32a0 0x7fad580b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad5d61b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fad580d03f0 con 0x7fad580cf690 2024-05-20T22:08:46.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad5d61b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad580d4300 con 0x7fad580cffe0 2024-05-20T22:08:46.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad5d61b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad580d4820 con 0x7fad580d32a0 2024-05-20T22:08:46.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad56ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580cf690 0x7fad58156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad56ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580cf690 0x7fad58156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49722/0 (socket says 172.21.15.55:49722) 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad56ffd640 1 -- 172.21.15.55:0/3869029346 learned_addr learned my addr 172.21.15.55:0/3869029346 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad577fe640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad580d32a0 0x7fad580b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad567fc640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad580cffe0 0x7fad580cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad56ffd640 1 -- 172.21.15.55:0/3869029346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad580d32a0 msgr2=0x7fad580b9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad56ffd640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad580d32a0 0x7fad580b9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad56ffd640 1 -- 172.21.15.55:0/3869029346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad580cffe0 msgr2=0x7fad580cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad56ffd640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad580cffe0 0x7fad580cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.345+0000 7fad56ffd640 1 -- 172.21.15.55:0/3869029346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad580cdcf0 con 0x7fad580cf690 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.346+0000 7fad56ffd640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580cf690 0x7fad58156350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad48000c00 tx=0x7fad480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.346+0000 7fad3f7fe640 1 -- 172.21.15.55:0/3869029346 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48037040 con 0x7fad580cf690 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.346+0000 7fad3f7fe640 1 -- 172.21.15.55:0/3869029346 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad48042070 con 0x7fad580cf690 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.346+0000 7fad5d61b640 1 -- 172.21.15.55:0/3869029346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad58161e90 con 0x7fad580cf690 2024-05-20T22:08:46.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.346+0000 7fad3f7fe640 1 -- 172.21.15.55:0/3869029346 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48030bb0 con 0x7fad580cf690 2024-05-20T22:08:46.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.346+0000 7fad5d61b640 1 -- 172.21.15.55:0/3869029346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad580cbeb0 con 0x7fad580cf690 2024-05-20T22:08:46.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.347+0000 7fad5d61b640 1 -- 172.21.15.55:0/3869029346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad58069090 con 0x7fad580cf690 2024-05-20T22:08:46.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.348+0000 7fad3f7fe640 1 -- 172.21.15.55:0/3869029346 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad48047460 con 0x7fad580cf690 2024-05-20T22:08:46.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.348+0000 7fad3f7fe640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fad4406be30 0x7fad4406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.348+0000 7fad3f7fe640 1 -- 172.21.15.55:0/3869029346 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fad480abc60 con 0x7fad580cf690 2024-05-20T22:08:46.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.348+0000 7fad567fc640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fad4406be30 0x7fad4406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.348+0000 7fad567fc640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fad4406be30 0x7fad4406e340 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fad4c006fd0 tx=0x7fad4c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:46.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.353+0000 7fad3f7fe640 1 -- 172.21.15.55:0/3869029346 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fad4804c020 con 0x7fad580cf690 2024-05-20T22:08:46.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.555+0000 7fad5d61b640 1 -- 172.21.15.55:0/3869029346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fad580d2e50 con 0x7fad580cf690 2024-05-20T22:08:46.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.555+0000 7fad3f7fe640 1 -- 172.21.15.55:0/3869029346 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fad480ab020 con 0x7fad580cf690 2024-05-20T22:08:46.556 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-20T22:08:46.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.557+0000 7fad5d61b640 1 -- 172.21.15.55:0/3869029346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fad4406be30 msgr2=0x7fad4406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.557+0000 7fad5d61b640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fad4406be30 0x7fad4406e340 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fad4c006fd0 tx=0x7fad4c008040 comp rx=0 tx=0).stop 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.557+0000 7fad5d61b640 1 -- 172.21.15.55:0/3869029346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580cf690 msgr2=0x7fad58156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.557+0000 7fad5d61b640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580cf690 0x7fad58156350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad48000c00 tx=0x7fad480046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.557+0000 7fad5d61b640 1 -- 172.21.15.55:0/3869029346 shutdown_connections 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.557+0000 7fad5d61b640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fad4406be30 0x7fad4406e340 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.557+0000 7fad5d61b640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad580d32a0 0x7fad580b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.557+0000 7fad5d61b640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad580cffe0 0x7fad580cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.557+0000 7fad5d61b640 1 --2- 172.21.15.55:0/3869029346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad580cf690 0x7fad58156350 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.557+0000 7fad5d61b640 1 -- 172.21.15.55:0/3869029346 >> 172.21.15.55:0/3869029346 conn(0x7fad580b6de0 msgr2=0x7fad580d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.558+0000 7fad5d61b640 1 -- 172.21.15.55:0/3869029346 shutdown_connections 2024-05-20T22:08:46.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.558+0000 7fad5d61b640 1 -- 172.21.15.55:0/3869029346 wait complete. 2024-05-20T22:08:46.570 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi055.front.sepia.ceph.com... 2024-05-20T22:08:46.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:46.570 DEBUG:teuthology.orchestra.run.smithi055:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-20T22:08:46.596 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:08:46.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:08:46.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:46.596 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T22:08:46.653 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:08:46.653 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:46.673 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:08:46.673 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:46.732 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:46.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.846+0000 7f2458f38640 1 Processor -- start 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.846+0000 7f2458f38640 1 -- start start 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2458f38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24540b3b30 0x7f24540b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2458f38640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f245415f100 0x7f245406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2458f38640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f245406c5c0 0x7f245406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2458f38640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f24540d3dd0 con 0x7f245406c5c0 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2458f38640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f24540d40b0 con 0x7f245415f100 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2458f38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f24540d4390 con 0x7f24540b3b30 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2452d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f245406c5c0 0x7f245406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2452d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f245406c5c0 0x7f245406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49724/0 (socket says 172.21.15.55:49724) 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2452d9d640 1 -- 172.21.15.55:0/3365805801 learned_addr learned my addr 172.21.15.55:0/3365805801 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:46.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2452d9d640 1 -- 172.21.15.55:0/3365805801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24540b3b30 msgr2=0x7f24540b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f245259c640 1 --2- 172.21.15.55:0/3365805801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24540b3b30 0x7f24540b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2452d9d640 1 --2- 172.21.15.55:0/3365805801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24540b3b30 0x7f24540b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2452d9d640 1 -- 172.21.15.55:0/3365805801 >> v2:172.21.15.138:3300/0 conn(0x7f245415f100 msgr2=0x7f245406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2452d9d640 1 --2- 172.21.15.55:0/3365805801 >> v2:172.21.15.138:3300/0 conn(0x7f245415f100 0x7f245406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.848+0000 7f2452d9d640 1 -- 172.21.15.55:0/3365805801 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245406b5a0 con 0x7f245406c5c0 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.849+0000 7f2452d9d640 1 --2- 172.21.15.55:0/3365805801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245406c5c0 0x7f245406c9d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24540b6c90 tx=0x7f244c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99246e95e95bdb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.849+0000 7f245159a640 1 -- 172.21.15.55:0/3365805801 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c037070 con 0x7f245406c5c0 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.849+0000 7f245159a640 1 -- 172.21.15.55:0/3365805801 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f244c032440 con 0x7f245406c5c0 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.849+0000 7f245159a640 1 -- 172.21.15.55:0/3365805801 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c032810 con 0x7f245406c5c0 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.850+0000 7f2458f38640 1 -- 172.21.15.55:0/3365805801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245406c5c0 msgr2=0x7f245406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.850+0000 7f2458f38640 1 --2- 172.21.15.55:0/3365805801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245406c5c0 0x7f245406c9d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24540b6c90 tx=0x7f244c02a1e0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.850+0000 7f2458f38640 1 -- 172.21.15.55:0/3365805801 shutdown_connections 2024-05-20T22:08:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.850+0000 7f2458f38640 1 --2- 172.21.15.55:0/3365805801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245406c5c0 0x7f245406c9d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.850+0000 7f2458f38640 1 --2- 172.21.15.55:0/3365805801 >> v2:172.21.15.138:3300/0 conn(0x7f245415f100 0x7f245406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.850+0000 7f2458f38640 1 --2- 172.21.15.55:0/3365805801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f24540b3b30 0x7f24540b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.850+0000 7f2458f38640 1 -- 172.21.15.55:0/3365805801 >> 172.21.15.55:0/3365805801 conn(0x7f24540d51d0 msgr2=0x7f24540b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.850+0000 7f2458f38640 1 -- 172.21.15.55:0/3365805801 shutdown_connections 2024-05-20T22:08:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.850+0000 7f2458f38640 1 -- 172.21.15.55:0/3365805801 wait complete. 2024-05-20T22:08:46.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.852+0000 7f2458f38640 1 Processor -- start 2024-05-20T22:08:46.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.852+0000 7f2458f38640 1 -- start start 2024-05-20T22:08:46.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.853+0000 7f2458f38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f245406c5c0 0x7f2454153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.853+0000 7f2458f38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540b3b30 0x7f2454150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.853+0000 7f2458f38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245415f100 0x7f245415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.853+0000 7f2458f38640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2454162b70 con 0x7f245415f100 2024-05-20T22:08:46.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.853+0000 7f2458f38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f245406bb60 con 0x7f24540b3b30 2024-05-20T22:08:46.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.853+0000 7f2458f38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2454162cf0 con 0x7f245406c5c0 2024-05-20T22:08:46.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.853+0000 7f2452d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245415f100 0x7f245415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.853+0000 7f2452d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245415f100 0x7f245415a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49740/0 (socket says 172.21.15.55:49740) 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.853+0000 7f2452d9d640 1 -- 172.21.15.55:0/3578398571 learned_addr learned my addr 172.21.15.55:0/3578398571 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f245259c640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f245406c5c0 0x7f2454153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2451d9b640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540b3b30 0x7f2454150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2452d9d640 1 -- 172.21.15.55:0/3578398571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f245406c5c0 msgr2=0x7f2454153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2452d9d640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f245406c5c0 0x7f2454153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2452d9d640 1 -- 172.21.15.55:0/3578398571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540b3b30 msgr2=0x7f2454150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2452d9d640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540b3b30 0x7f2454150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2452d9d640 1 -- 172.21.15.55:0/3578398571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24541f53a0 con 0x7f245415f100 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2452d9d640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245415f100 0x7f245415a000 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f244c0090b0 tx=0x7f244c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2442ffd640 1 -- 172.21.15.55:0/3578398571 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c041020 con 0x7f245415f100 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2442ffd640 1 -- 172.21.15.55:0/3578398571 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f244c030990 con 0x7f245415f100 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2442ffd640 1 -- 172.21.15.55:0/3578398571 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c037040 con 0x7f245415f100 2024-05-20T22:08:46.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2458f38640 1 -- 172.21.15.55:0/3578398571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24541f5680 con 0x7f245415f100 2024-05-20T22:08:46.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.854+0000 7f2458f38640 1 -- 172.21.15.55:0/3578398571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24541f5c40 con 0x7f245415f100 2024-05-20T22:08:46.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.855+0000 7f2458f38640 1 -- 172.21.15.55:0/3578398571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2454069090 con 0x7f245415f100 2024-05-20T22:08:46.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.856+0000 7f2442ffd640 1 -- 172.21.15.55:0/3578398571 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f244c044020 con 0x7f245415f100 2024-05-20T22:08:46.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.856+0000 7f2442ffd640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f243406be30 0x7f243406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:46.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.856+0000 7f2442ffd640 1 -- 172.21.15.55:0/3578398571 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f244c0abac0 con 0x7f245415f100 2024-05-20T22:08:46.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.856+0000 7f245259c640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f243406be30 0x7f243406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:46.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.857+0000 7f245259c640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f243406be30 0x7f243406e340 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f245406d0c0 tx=0x7f2444008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:46.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:46.861+0000 7f2442ffd640 1 -- 172.21.15.55:0/3578398571 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f244c04b030 con 0x7f245415f100 2024-05-20T22:08:47.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.045+0000 7f2458f38640 1 -- 172.21.15.55:0/3578398571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24540d2ee0 con 0x7f245415f100 2024-05-20T22:08:47.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.047+0000 7f2442ffd640 1 -- 172.21.15.55:0/3578398571 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7f244c079a20 con 0x7f245415f100 2024-05-20T22:08:47.048 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:47.048 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.048+0000 7f2458f38640 1 -- 172.21.15.55:0/3578398571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f243406be30 msgr2=0x7f243406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.048+0000 7f2458f38640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f243406be30 0x7f243406e340 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f245406d0c0 tx=0x7f2444008040 comp rx=0 tx=0).stop 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.048+0000 7f2458f38640 1 -- 172.21.15.55:0/3578398571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245415f100 msgr2=0x7f245415a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.048+0000 7f2458f38640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245415f100 0x7f245415a000 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f244c0090b0 tx=0x7f244c0046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.049+0000 7f2458f38640 1 -- 172.21.15.55:0/3578398571 shutdown_connections 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.049+0000 7f2458f38640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f243406be30 0x7f243406e340 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.049+0000 7f2458f38640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f245415f100 0x7f245415a000 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.049+0000 7f2458f38640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24540b3b30 0x7f2454150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.049+0000 7f2458f38640 1 --2- 172.21.15.55:0/3578398571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f245406c5c0 0x7f2454153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.049+0000 7f2458f38640 1 -- 172.21.15.55:0/3578398571 >> 172.21.15.55:0/3578398571 conn(0x7f24540d51d0 msgr2=0x7f24540cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.049+0000 7f2458f38640 1 -- 172.21.15.55:0/3578398571 shutdown_connections 2024-05-20T22:08:47.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.049+0000 7f2458f38640 1 -- 172.21.15.55:0/3578398571 wait complete. 2024-05-20T22:08:47.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:47.062 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:47.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.179+0000 7f682880d640 1 Processor -- start 2024-05-20T22:08:47.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.179+0000 7f682880d640 1 -- start start 2024-05-20T22:08:47.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.180+0000 7f682880d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68240b4080 0x7f682415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.180+0000 7f682880d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f682406b500 con 0x7f68240b35e0 2024-05-20T22:08:47.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.180+0000 7f682880d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68240b3e10 con 0x7f68240d3290 2024-05-20T22:08:47.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.180+0000 7f682880d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f6824163960 con 0x7f68240b4080 2024-05-20T22:08:47.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.180+0000 7f682259c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68240b4080 0x7f682415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.180+0000 7f681bfff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f68240b35e0 0x7f682415f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60544/0 (socket says 172.21.15.55:60544) 2024-05-20T22:08:47.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.180+0000 7f681bfff640 1 -- 172.21.15.55:0/1306268412 learned_addr learned my addr 172.21.15.55:0/1306268412 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:47.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.181+0000 7f682259c640 1 -- 172.21.15.55:0/1306268412 >> v1:172.21.15.138:6789/0 conn(0x7f68240d3290 legacy=0x7f68240d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.181+0000 7f682259c640 1 -- 172.21.15.55:0/1306268412 >> v1:172.21.15.55:6789/0 conn(0x7f68240b35e0 legacy=0x7f682415f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.181+0000 7f682259c640 1 -- 172.21.15.55:0/1306268412 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68240b3b30 con 0x7f68240b4080 2024-05-20T22:08:47.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.181+0000 7f6821d9b640 1 -- 172.21.15.55:0/1306268412 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 461465677 0 0) 0x7f680c001110 con 0x7f68240b35e0 2024-05-20T22:08:47.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.181+0000 7f682259c640 1 --2- 172.21.15.55:0/1306268412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b4080 0x7f682415f2e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f68140097a0 tx=0x7f681402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e589549ca0024852 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:47.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.182+0000 7f6821d9b640 1 -- 172.21.15.55:0/1306268412 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814037070 con 0x7f68240b4080 2024-05-20T22:08:47.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.182+0000 7f6821d9b640 1 -- 172.21.15.55:0/1306268412 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6814032440 con 0x7f68240b4080 2024-05-20T22:08:47.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.182+0000 7f6821d9b640 1 -- 172.21.15.55:0/1306268412 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814032810 con 0x7f68240b4080 2024-05-20T22:08:47.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.182+0000 7f682880d640 1 -- 172.21.15.55:0/1306268412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b4080 msgr2=0x7f682415f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.182+0000 7f682880d640 1 --2- 172.21.15.55:0/1306268412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b4080 0x7f682415f2e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f68140097a0 tx=0x7f681402a230 comp rx=0 tx=0).stop 2024-05-20T22:08:47.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.182+0000 7f682880d640 1 -- 172.21.15.55:0/1306268412 shutdown_connections 2024-05-20T22:08:47.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.182+0000 7f682880d640 1 --2- 172.21.15.55:0/1306268412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b4080 0x7f682415f2e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.182+0000 7f682880d640 1 -- 172.21.15.55:0/1306268412 >> 172.21.15.55:0/1306268412 conn(0x7f68240d51b0 msgr2=0x7f68240b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:47.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.182+0000 7f682880d640 1 -- 172.21.15.55:0/1306268412 shutdown_connections 2024-05-20T22:08:47.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.182+0000 7f682880d640 1 -- 172.21.15.55:0/1306268412 wait complete. 2024-05-20T22:08:47.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.184+0000 7f682880d640 1 Processor -- start 2024-05-20T22:08:47.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.184+0000 7f682880d640 1 -- start start 2024-05-20T22:08:47.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.185+0000 7f682880d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b35e0 0x7f6824158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682880d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68240b4080 0x7f682415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682880d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68240d3290 0x7f682415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682880d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f68241635f0 con 0x7f68240b35e0 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682880d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68241604f0 con 0x7f68240d3290 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682880d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f682415ff30 con 0x7f68240b4080 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b35e0 0x7f6824158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b35e0 0x7f6824158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49758/0 (socket says 172.21.15.55:49758) 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682259c640 1 -- 172.21.15.55:0/3255982697 learned_addr learned my addr 172.21.15.55:0/3255982697 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f6822d9d640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68240d3290 0x7f682415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f681bfff640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68240b4080 0x7f682415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682259c640 1 -- 172.21.15.55:0/3255982697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68240b4080 msgr2=0x7f682415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682259c640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68240b4080 0x7f682415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682259c640 1 -- 172.21.15.55:0/3255982697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68240d3290 msgr2=0x7f682415a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682259c640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68240d3290 0x7f682415a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682259c640 1 -- 172.21.15.55:0/3255982697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68240c9dd0 con 0x7f68240b35e0 2024-05-20T22:08:47.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.186+0000 7f682259c640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b35e0 0x7f6824158e40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6814000c00 tx=0x7f68140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.187+0000 7f681affd640 1 -- 172.21.15.55:0/3255982697 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814041070 con 0x7f68240b35e0 2024-05-20T22:08:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.187+0000 7f681affd640 1 -- 172.21.15.55:0/3255982697 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681403c930 con 0x7f68240b35e0 2024-05-20T22:08:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.187+0000 7f682880d640 1 -- 172.21.15.55:0/3255982697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68240c9530 con 0x7f68240b35e0 2024-05-20T22:08:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.187+0000 7f681affd640 1 -- 172.21.15.55:0/3255982697 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814037040 con 0x7f68240b35e0 2024-05-20T22:08:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.187+0000 7f682880d640 1 -- 172.21.15.55:0/3255982697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68240c9af0 con 0x7f68240b35e0 2024-05-20T22:08:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.188+0000 7f682880d640 1 -- 172.21.15.55:0/3255982697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6824069090 con 0x7f68240b35e0 2024-05-20T22:08:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.188+0000 7f681affd640 1 -- 172.21.15.55:0/3255982697 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6814044070 con 0x7f68240b35e0 2024-05-20T22:08:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.189+0000 7f681affd640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f680406be30 0x7f680406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.189+0000 7f681affd640 1 -- 172.21.15.55:0/3255982697 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f68140abe80 con 0x7f68240b35e0 2024-05-20T22:08:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.189+0000 7f681bfff640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f680406be30 0x7f680406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.189+0000 7f681bfff640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f680406be30 0x7f680406e340 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f680c000f60 tx=0x7f680c007e40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:47.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.194+0000 7f681affd640 1 -- 172.21.15.55:0/3255982697 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f68240c9dd0 con 0x7f68240b35e0 2024-05-20T22:08:47.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.379+0000 7f682880d640 1 -- 172.21.15.55:0/3255982697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f682415eb20 con 0x7f68240b35e0 2024-05-20T22:08:47.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.379+0000 7f681affd640 1 -- 172.21.15.55:0/3255982697 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f6814079cb0 con 0x7f68240b35e0 2024-05-20T22:08:47.380 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:47.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.381+0000 7f682880d640 1 -- 172.21.15.55:0/3255982697 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f680406be30 msgr2=0x7f680406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.381+0000 7f682880d640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f680406be30 0x7f680406e340 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f680c000f60 tx=0x7f680c007e40 comp rx=0 tx=0).stop 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.381+0000 7f682880d640 1 -- 172.21.15.55:0/3255982697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b35e0 msgr2=0x7f6824158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.381+0000 7f682880d640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b35e0 0x7f6824158e40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6814000c00 tx=0x7f68140046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.381+0000 7f682880d640 1 -- 172.21.15.55:0/3255982697 shutdown_connections 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.381+0000 7f682880d640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f680406be30 0x7f680406e340 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.381+0000 7f682880d640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68240d3290 0x7f682415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.381+0000 7f682880d640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68240b4080 0x7f682415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.382+0000 7f682880d640 1 --2- 172.21.15.55:0/3255982697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68240b35e0 0x7f6824158e40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.382+0000 7f682880d640 1 -- 172.21.15.55:0/3255982697 >> 172.21.15.55:0/3255982697 conn(0x7f68240d51b0 msgr2=0x7f6824161960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.382+0000 7f682880d640 1 -- 172.21.15.55:0/3255982697 shutdown_connections 2024-05-20T22:08:47.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.382+0000 7f682880d640 1 -- 172.21.15.55:0/3255982697 wait complete. 2024-05-20T22:08:47.394 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:47.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.512+0000 7f7b9fc8b640 1 Processor -- start 2024-05-20T22:08:47.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.512+0000 7f7b9fc8b640 1 -- start start 2024-05-20T22:08:47.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.513+0000 7f7b9fc8b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7b9808c960 0x7f7b980b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.513+0000 7f7b9fc8b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b9808cd70 con 0x7f7b980cd520 2024-05-20T22:08:47.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.513+0000 7f7b9fc8b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b980b1f30 con 0x7f7b980cf270 2024-05-20T22:08:47.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.513+0000 7f7b9fc8b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7b980d1020 con 0x7f7b9808c960 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.513+0000 7f7b9da27640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7b980cd520 0x7f7b980d0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60560/0 (socket says 172.21.15.55:60560) 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.513+0000 7f7b9da27640 1 -- 172.21.15.55:0/1358115479 learned_addr learned my addr 172.21.15.55:0/1358115479 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1507567130 0 0) 0x7f7b90001110 con 0x7f7b980cd520 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b80003520 con 0x7f7b980cd520 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9d226640 1 --2- 172.21.15.55:0/1358115479 >> v2:172.21.15.138:3300/0 conn(0x7f7b9808c960 0x7f7b980b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2305421013 0 0) 0x7f7b80003520 con 0x7f7b980cd520 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 >> v2:172.21.15.138:3300/0 conn(0x7f7b9808c960 msgr2=0x7f7b980b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 --2- 172.21.15.55:0/1358115479 >> v2:172.21.15.138:3300/0 conn(0x7f7b9808c960 0x7f7b980b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 >> v1:172.21.15.138:6789/0 conn(0x7f7b980cf270 legacy=0x7f7b980cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b980d1300 con 0x7f7b980cd520 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f7b90001840 con 0x7f7b980cd520 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 >> v1:172.21.15.55:6789/0 conn(0x7f7b980cd520 legacy=0x7f7b980d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b800039b0 0x7f7b80003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b80004800 0x7f7b80004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b80005250 0x7f7b8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7b980d1020 con 0x7f7b800039b0 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8000c1e0 con 0x7f7b80005250 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8000c3e0 con 0x7f7b80004800 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9da27640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b800039b0 0x7f7b80003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9d226640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b80004800 0x7f7b80004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9e228640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b80005250 0x7f7b8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9da27640 1 -- 172.21.15.55:0/1358115479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b80004800 msgr2=0x7f7b80004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9da27640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b80004800 0x7f7b80004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9da27640 1 -- 172.21.15.55:0/1358115479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b80005250 msgr2=0x7f7b8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9da27640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b80005250 0x7f7b8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9e228640 1 -- 172.21.15.55:0/1358115479 reap_dead start 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.514+0000 7f7b9da27640 1 -- 172.21.15.55:0/1358115479 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b80003520 con 0x7f7b800039b0 2024-05-20T22:08:47.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.515+0000 7f7b9da27640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b800039b0 0x7f7b80003e20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000c4c0 tx=0x7f7b80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aca636e6f63ff3a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:47.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.515+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000c0e0 con 0x7f7b800039b0 2024-05-20T22:08:47.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.515+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b90034070 con 0x7f7b800039b0 2024-05-20T22:08:47.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.515+0000 7f7b9ca25640 1 -- 172.21.15.55:0/1358115479 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90005d70 con 0x7f7b800039b0 2024-05-20T22:08:47.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.516+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1358115479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b800039b0 msgr2=0x7f7b80003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.516+0000 7f7b9fc8b640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b800039b0 0x7f7b80003e20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000c4c0 tx=0x7f7b80005830 comp rx=0 tx=0).stop 2024-05-20T22:08:47.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.516+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1358115479 shutdown_connections 2024-05-20T22:08:47.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.516+0000 7f7b9fc8b640 1 --2- 172.21.15.55:0/1358115479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b800039b0 0x7f7b80003e20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.516+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1358115479 >> 172.21.15.55:0/1358115479 conn(0x7f7b980d2860 msgr2=0x7f7b980d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:47.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.516+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1358115479 shutdown_connections 2024-05-20T22:08:47.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.516+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1358115479 wait complete. 2024-05-20T22:08:47.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.518+0000 7f7b9fc8b640 1 Processor -- start 2024-05-20T22:08:47.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.518+0000 7f7b9fc8b640 1 -- start start 2024-05-20T22:08:47.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.519+0000 7f7b9fc8b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b800039b0 0x7f7b980cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.519+0000 7f7b9fc8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9815f5a0 0x7f7b980b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.519+0000 7f7b9fc8b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b980b0f70 0x7f7b981574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.519+0000 7f7b9fc8b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7b980b2cb0 con 0x7f7b980b0f70 2024-05-20T22:08:47.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.519+0000 7f7b9fc8b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b980d18c0 con 0x7f7b9815f5a0 2024-05-20T22:08:47.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.519+0000 7f7b9fc8b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7b980b19f0 con 0x7f7b800039b0 2024-05-20T22:08:47.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.519+0000 7f7b9e228640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b980b0f70 0x7f7b981574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9e228640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b980b0f70 0x7f7b981574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49768/0 (socket says 172.21.15.55:49768) 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9e228640 1 -- 172.21.15.55:0/1190189031 learned_addr learned my addr 172.21.15.55:0/1190189031 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9da27640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b800039b0 0x7f7b980cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9d226640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9815f5a0 0x7f7b980b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9e228640 1 -- 172.21.15.55:0/1190189031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b800039b0 msgr2=0x7f7b980cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9e228640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b800039b0 0x7f7b980cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9e228640 1 -- 172.21.15.55:0/1190189031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9815f5a0 msgr2=0x7f7b980b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9e228640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9815f5a0 0x7f7b980b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9e228640 1 -- 172.21.15.55:0/1190189031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b980b1380 con 0x7f7b980b0f70 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9d226640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9815f5a0 0x7f7b980b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9e228640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b980b0f70 0x7f7b981574f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7b980d2400 tx=0x7f7b8800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b967fc640 1 -- 172.21.15.55:0/1190189031 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88019070 con 0x7f7b980b0f70 2024-05-20T22:08:47.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b967fc640 1 -- 172.21.15.55:0/1190189031 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b88006b50 con 0x7f7b980b0f70 2024-05-20T22:08:47.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b967fc640 1 -- 172.21.15.55:0/1190189031 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88003970 con 0x7f7b980b0f70 2024-05-20T22:08:47.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1190189031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b980ccc60 con 0x7f7b980b0f70 2024-05-20T22:08:47.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.520+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1190189031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9815ec10 con 0x7f7b980b0f70 2024-05-20T22:08:47.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.521+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1190189031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9815e6a0 con 0x7f7b980b0f70 2024-05-20T22:08:47.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.522+0000 7f7b967fc640 1 -- 172.21.15.55:0/1190189031 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b8801c070 con 0x7f7b980b0f70 2024-05-20T22:08:47.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.522+0000 7f7b967fc640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7b8406be30 0x7f7b8406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.523+0000 7f7b967fc640 1 -- 172.21.15.55:0/1190189031 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7b8808a730 con 0x7f7b980b0f70 2024-05-20T22:08:47.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.523+0000 7f7b9da27640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7b8406be30 0x7f7b8406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.523+0000 7f7b9da27640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7b8406be30 0x7f7b8406e340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b90033000 tx=0x7f7b9002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:47.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.527+0000 7f7b967fc640 1 -- 172.21.15.55:0/1190189031 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7b980b1380 con 0x7f7b980b0f70 2024-05-20T22:08:47.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.711+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1190189031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b980b27c0 con 0x7f7b980b0f70 2024-05-20T22:08:47.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.711+0000 7f7b967fc640 1 -- 172.21.15.55:0/1190189031 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7b88058690 con 0x7f7b980b0f70 2024-05-20T22:08:47.712 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1190189031 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7b8406be30 msgr2=0x7f7b8406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7b8406be30 0x7f7b8406e340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b90033000 tx=0x7f7b9002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1190189031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b980b0f70 msgr2=0x7f7b981574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b980b0f70 0x7f7b981574f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7b980d2400 tx=0x7f7b8800eb90 comp rx=0 tx=0).stop 2024-05-20T22:08:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1190189031 shutdown_connections 2024-05-20T22:08:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7b8406be30 0x7f7b8406e340 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b980b0f70 0x7f7b981574f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9815f5a0 0x7f7b980b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 --2- 172.21.15.55:0/1190189031 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7b800039b0 0x7f7b980cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1190189031 >> 172.21.15.55:0/1190189031 conn(0x7f7b980d2860 msgr2=0x7f7b980706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:47.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1190189031 shutdown_connections 2024-05-20T22:08:47.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.713+0000 7f7b9fc8b640 1 -- 172.21.15.55:0/1190189031 wait complete. 2024-05-20T22:08:47.725 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:47.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.841+0000 7fac44365640 1 Processor -- start 2024-05-20T22:08:47.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.841+0000 7fac44365640 1 -- start start 2024-05-20T22:08:47.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac44365640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fac3c15f150 0x7fac3c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac44365640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac3c06b500 0x7fac3c06b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac44365640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac3c0d3780 con 0x7fac3c0d32a0 2024-05-20T22:08:47.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac44365640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fac3c0d3a60 con 0x7fac3c15f150 2024-05-20T22:08:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac44365640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fac3c0d3d40 con 0x7fac3c06b500 2024-05-20T22:08:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac41900640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fac3c15f150 0x7fac3c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac41900640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fac3c15f150 0x7fac3c0cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49778/0 (socket says 172.21.15.55:49778) 2024-05-20T22:08:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac41900640 1 -- 172.21.15.55:0/1904830673 learned_addr learned my addr 172.21.15.55:0/1904830673 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac42902640 1 --2- 172.21.15.55:0/1904830673 >> v2:172.21.15.138:3300/0 conn(0x7fac3c06b500 0x7fac3c06b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac41900640 1 -- 172.21.15.55:0/1904830673 >> v2:172.21.15.138:3300/0 conn(0x7fac3c06b500 msgr2=0x7fac3c06b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac41900640 1 --2- 172.21.15.55:0/1904830673 >> v2:172.21.15.138:3300/0 conn(0x7fac3c06b500 0x7fac3c06b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac41900640 1 -- 172.21.15.55:0/1904830673 >> v1:172.21.15.138:6789/0 conn(0x7fac3c0d32a0 legacy=0x7fac3c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.843+0000 7fac41900640 1 -- 172.21.15.55:0/1904830673 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c0d4020 con 0x7fac3c15f150 2024-05-20T22:08:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.844+0000 7fac41900640 1 --2- 172.21.15.55:0/1904830673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c15f150 0x7fac3c0cfda0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fac3c163d70 tx=0x7fac3002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c37dceebe358f47 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.844+0000 7fac410ff640 1 -- 172.21.15.55:0/1904830673 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30037070 con 0x7fac3c15f150 2024-05-20T22:08:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.844+0000 7fac410ff640 1 -- 172.21.15.55:0/1904830673 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac30032440 con 0x7fac3c15f150 2024-05-20T22:08:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.844+0000 7fac410ff640 1 -- 172.21.15.55:0/1904830673 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30032810 con 0x7fac3c15f150 2024-05-20T22:08:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.845+0000 7fac44365640 1 -- 172.21.15.55:0/1904830673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c15f150 msgr2=0x7fac3c0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.845+0000 7fac44365640 1 --2- 172.21.15.55:0/1904830673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c15f150 0x7fac3c0cfda0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fac3c163d70 tx=0x7fac3002a230 comp rx=0 tx=0).stop 2024-05-20T22:08:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.845+0000 7fac44365640 1 -- 172.21.15.55:0/1904830673 shutdown_connections 2024-05-20T22:08:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.845+0000 7fac44365640 1 --2- 172.21.15.55:0/1904830673 >> v2:172.21.15.138:3300/0 conn(0x7fac3c06b500 0x7fac3c06b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.845+0000 7fac44365640 1 --2- 172.21.15.55:0/1904830673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c15f150 0x7fac3c0cfda0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.845+0000 7fac44365640 1 -- 172.21.15.55:0/1904830673 >> 172.21.15.55:0/1904830673 conn(0x7fac3c0b6de0 msgr2=0x7fac3c0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.845+0000 7fac44365640 1 -- 172.21.15.55:0/1904830673 shutdown_connections 2024-05-20T22:08:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.845+0000 7fac44365640 1 -- 172.21.15.55:0/1904830673 wait complete. 2024-05-20T22:08:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.847+0000 7fac44365640 1 Processor -- start 2024-05-20T22:08:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.847+0000 7fac44365640 1 -- start start 2024-05-20T22:08:47.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.848+0000 7fac44365640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c06b500 0x7fac3c0c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.848+0000 7fac44365640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c0d32a0 0x7fac3c0cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.848+0000 7fac44365640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c15f150 0x7fac3c1f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.848+0000 7fac44365640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c0d4820 con 0x7fac3c06b500 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.848+0000 7fac44365640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c15f560 con 0x7fac3c0d32a0 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.848+0000 7fac44365640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac3c0cc450 con 0x7fac3c15f150 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.848+0000 7fac42101640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c06b500 0x7fac3c0c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.848+0000 7fac42101640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c06b500 0x7fac3c0c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49790/0 (socket says 172.21.15.55:49790) 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.848+0000 7fac42101640 1 -- 172.21.15.55:0/2431662343 learned_addr learned my addr 172.21.15.55:0/2431662343 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac42902640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c15f150 0x7fac3c1f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac41900640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c0d32a0 0x7fac3c0cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac42101640 1 -- 172.21.15.55:0/2431662343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c15f150 msgr2=0x7fac3c1f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac42101640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c15f150 0x7fac3c1f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac42101640 1 -- 172.21.15.55:0/2431662343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c0d32a0 msgr2=0x7fac3c0cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac42101640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c0d32a0 0x7fac3c0cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac42101640 1 -- 172.21.15.55:0/2431662343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c14aa50 con 0x7fac3c06b500 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac42101640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c06b500 0x7fac3c0c87d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00ba50 tx=0x7fac2c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac3affd640 1 -- 172.21.15.55:0/2431662343 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c019070 con 0x7fac3c06b500 2024-05-20T22:08:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac3affd640 1 -- 172.21.15.55:0/2431662343 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac2c00cbc0 con 0x7fac3c06b500 2024-05-20T22:08:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac44365640 1 -- 172.21.15.55:0/2431662343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c0cb2d0 con 0x7fac3c06b500 2024-05-20T22:08:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac3affd640 1 -- 172.21.15.55:0/2431662343 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c014440 con 0x7fac3c06b500 2024-05-20T22:08:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.849+0000 7fac44365640 1 -- 172.21.15.55:0/2431662343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c161860 con 0x7fac3c06b500 2024-05-20T22:08:47.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.850+0000 7fac44365640 1 -- 172.21.15.55:0/2431662343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3c0b4a40 con 0x7fac3c06b500 2024-05-20T22:08:47.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.851+0000 7fac3affd640 1 -- 172.21.15.55:0/2431662343 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac2c01c070 con 0x7fac3c06b500 2024-05-20T22:08:47.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.851+0000 7fac3affd640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac2806be60 0x7fac2806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:47.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.851+0000 7fac3affd640 1 -- 172.21.15.55:0/2431662343 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fac2c08af10 con 0x7fac3c06b500 2024-05-20T22:08:47.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.852+0000 7fac41900640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac2806be60 0x7fac2806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:47.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.852+0000 7fac41900640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac2806be60 0x7fac2806e370 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fac30034000 tx=0x7fac3002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:47.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:47.856+0000 7fac3affd640 1 -- 172.21.15.55:0/2431662343 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fac3c0b4a40 con 0x7fac3c06b500 2024-05-20T22:08:48.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.040+0000 7fac44365640 1 -- 172.21.15.55:0/2431662343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fac3c069090 con 0x7fac3c06b500 2024-05-20T22:08:48.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.040+0000 7fac3affd640 1 -- 172.21.15.55:0/2431662343 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fac2c058dd0 con 0x7fac3c06b500 2024-05-20T22:08:48.041 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:48.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 -- 172.21.15.55:0/2431662343 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac2806be60 msgr2=0x7fac2806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac2806be60 0x7fac2806e370 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fac30034000 tx=0x7fac3002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:48.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 -- 172.21.15.55:0/2431662343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c06b500 msgr2=0x7fac3c0c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c06b500 0x7fac3c0c87d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00ba50 tx=0x7fac2c00bf10 comp rx=0 tx=0).stop 2024-05-20T22:08:48.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 -- 172.21.15.55:0/2431662343 shutdown_connections 2024-05-20T22:08:48.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fac2806be60 0x7fac2806e370 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c15f150 0x7fac3c1f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c0d32a0 0x7fac3c0cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 --2- 172.21.15.55:0/2431662343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac3c06b500 0x7fac3c0c87d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 -- 172.21.15.55:0/2431662343 >> 172.21.15.55:0/2431662343 conn(0x7fac3c0b6de0 msgr2=0x7fac3c0b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:48.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 -- 172.21.15.55:0/2431662343 shutdown_connections 2024-05-20T22:08:48.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.042+0000 7fac44365640 1 -- 172.21.15.55:0/2431662343 wait complete. 2024-05-20T22:08:48.054 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:48.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.171+0000 7f77f3d1d640 1 Processor -- start 2024-05-20T22:08:48.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.171+0000 7f77f3d1d640 1 -- start start 2024-05-20T22:08:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f3d1d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f77ec0d32a0 0x7f77ec15f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f3d1d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d01b0 0x7f77ec06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f3d1d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77ec0d3780 con 0x7f77ec0cf690 2024-05-20T22:08:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f3d1d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f77ec0d3a60 con 0x7f77ec0d32a0 2024-05-20T22:08:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f3d1d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f77ec0d3d40 con 0x7f77ec0d01b0 2024-05-20T22:08:48.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f1ab9640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f77ec0d32a0 0x7f77ec15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f1ab9640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f77ec0d32a0 0x7f77ec15f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49792/0 (socket says 172.21.15.55:49792) 2024-05-20T22:08:48.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f1ab9640 1 -- 172.21.15.55:0/3054293949 learned_addr learned my addr 172.21.15.55:0/3054293949 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:48.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f22ba640 1 --2- 172.21.15.55:0/3054293949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d01b0 0x7f77ec06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f1ab9640 1 -- 172.21.15.55:0/3054293949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d01b0 msgr2=0x7f77ec06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:48.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f1ab9640 1 --2- 172.21.15.55:0/3054293949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d01b0 0x7f77ec06b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f1ab9640 1 -- 172.21.15.55:0/3054293949 >> v1:172.21.15.138:6789/0 conn(0x7f77ec0cf690 legacy=0x7f77ec0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:48.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f1ab9640 1 -- 172.21.15.55:0/3054293949 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec0d4020 con 0x7f77ec0d32a0 2024-05-20T22:08:48.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.173+0000 7f77f22ba640 1 --2- 172.21.15.55:0/3054293949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d01b0 0x7f77ec06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:48.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.174+0000 7f77f1ab9640 1 --2- 172.21.15.55:0/3054293949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d32a0 0x7f77ec15f150 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009870 tx=0x7f77e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc696a7767f7c217 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:48.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.174+0000 7f77f0ab7640 1 -- 172.21.15.55:0/3054293949 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4037070 con 0x7f77ec0d32a0 2024-05-20T22:08:48.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.174+0000 7f77f0ab7640 1 -- 172.21.15.55:0/3054293949 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e4032440 con 0x7f77ec0d32a0 2024-05-20T22:08:48.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.174+0000 7f77f0ab7640 1 -- 172.21.15.55:0/3054293949 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4032810 con 0x7f77ec0d32a0 2024-05-20T22:08:48.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.175+0000 7f77f3d1d640 1 -- 172.21.15.55:0/3054293949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d32a0 msgr2=0x7f77ec15f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.175+0000 7f77f3d1d640 1 --2- 172.21.15.55:0/3054293949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d32a0 0x7f77ec15f150 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009870 tx=0x7f77e402a210 comp rx=0 tx=0).stop 2024-05-20T22:08:48.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.175+0000 7f77f3d1d640 1 -- 172.21.15.55:0/3054293949 shutdown_connections 2024-05-20T22:08:48.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.175+0000 7f77f3d1d640 1 --2- 172.21.15.55:0/3054293949 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d01b0 0x7f77ec06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.175+0000 7f77f3d1d640 1 --2- 172.21.15.55:0/3054293949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d32a0 0x7f77ec15f150 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.175+0000 7f77f3d1d640 1 -- 172.21.15.55:0/3054293949 >> 172.21.15.55:0/3054293949 conn(0x7f77ec0b6de0 msgr2=0x7f77ec0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:48.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.175+0000 7f77f3d1d640 1 -- 172.21.15.55:0/3054293949 shutdown_connections 2024-05-20T22:08:48.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.175+0000 7f77f3d1d640 1 -- 172.21.15.55:0/3054293949 wait complete. 2024-05-20T22:08:48.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.177+0000 7f77f3d1d640 1 Processor -- start 2024-05-20T22:08:48.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.177+0000 7f77f3d1d640 1 -- start start 2024-05-20T22:08:48.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f3d1d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ec0cf690 0x7f77ec0cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f3d1d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d01b0 0x7f77ec1f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f3d1d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d32a0 0x7f77ec0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f3d1d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec0d4820 con 0x7f77ec0d01b0 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f3d1d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec0b4350 con 0x7f77ec0cf690 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f3d1d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f77ec14e4f0 con 0x7f77ec0d32a0 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f12b8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d01b0 0x7f77ec1f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f12b8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d01b0 0x7f77ec1f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49806/0 (socket says 172.21.15.55:49806) 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f12b8640 1 -- 172.21.15.55:0/2985139922 learned_addr learned my addr 172.21.15.55:0/2985139922 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f22ba640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d32a0 0x7f77ec0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.178+0000 7f77f1ab9640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ec0cf690 0x7f77ec0cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77f12b8640 1 -- 172.21.15.55:0/2985139922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d32a0 msgr2=0x7f77ec0cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77f12b8640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d32a0 0x7f77ec0cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77f12b8640 1 -- 172.21.15.55:0/2985139922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ec0cf690 msgr2=0x7f77ec0cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77f12b8640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ec0cf690 0x7f77ec0cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77f12b8640 1 -- 172.21.15.55:0/2985139922 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec1f5770 con 0x7f77ec0d01b0 2024-05-20T22:08:48.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77f12b8640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d01b0 0x7f77ec1f5230 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f77e000b3f0 tx=0x7f77e000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:48.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77ea7fc640 1 -- 172.21.15.55:0/2985139922 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0015020 con 0x7f77ec0d01b0 2024-05-20T22:08:48.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77ea7fc640 1 -- 172.21.15.55:0/2985139922 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e0011830 con 0x7f77ec0d01b0 2024-05-20T22:08:48.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77f3d1d640 1 -- 172.21.15.55:0/2985139922 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ec0cc150 con 0x7f77ec0d01b0 2024-05-20T22:08:48.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77ea7fc640 1 -- 172.21.15.55:0/2985139922 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0011c20 con 0x7f77ec0d01b0 2024-05-20T22:08:48.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.179+0000 7f77f3d1d640 1 -- 172.21.15.55:0/2985139922 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ec0cc710 con 0x7f77ec0d01b0 2024-05-20T22:08:48.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.180+0000 7f77f3d1d640 1 -- 172.21.15.55:0/2985139922 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ec069090 con 0x7f77ec0d01b0 2024-05-20T22:08:48.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.181+0000 7f77ea7fc640 1 -- 172.21.15.55:0/2985139922 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77e0018020 con 0x7f77ec0d01b0 2024-05-20T22:08:48.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.181+0000 7f77ea7fc640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f77d806be30 0x7f77d806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.181+0000 7f77ea7fc640 1 -- 172.21.15.55:0/2985139922 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f77e008ae70 con 0x7f77ec0d01b0 2024-05-20T22:08:48.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.181+0000 7f77f1ab9640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f77d806be30 0x7f77d806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.182+0000 7f77f1ab9640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f77d806be30 0x7f77d806e340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f77e4034000 tx=0x7f77e402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:48.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.186+0000 7f77ea7fc640 1 -- 172.21.15.55:0/2985139922 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f77e001f030 con 0x7f77ec0d01b0 2024-05-20T22:08:48.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.371+0000 7f77f3d1d640 1 -- 172.21.15.55:0/2985139922 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f77ec0d2e50 con 0x7f77ec0d01b0 2024-05-20T22:08:48.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.371+0000 7f77ea7fc640 1 -- 172.21.15.55:0/2985139922 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f77e00172c0 con 0x7f77ec0d01b0 2024-05-20T22:08:48.372 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:48.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.373+0000 7f77f3d1d640 1 -- 172.21.15.55:0/2985139922 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f77d806be30 msgr2=0x7f77d806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.373+0000 7f77f3d1d640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f77d806be30 0x7f77d806e340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f77e4034000 tx=0x7f77e402e000 comp rx=0 tx=0).stop 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.373+0000 7f77f3d1d640 1 -- 172.21.15.55:0/2985139922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d01b0 msgr2=0x7f77ec1f5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.373+0000 7f77f3d1d640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d01b0 0x7f77ec1f5230 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f77e000b3f0 tx=0x7f77e000b8b0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.373+0000 7f77f3d1d640 1 -- 172.21.15.55:0/2985139922 shutdown_connections 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.373+0000 7f77f3d1d640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f77d806be30 0x7f77d806e340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.373+0000 7f77f3d1d640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77ec0d32a0 0x7f77ec0cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.373+0000 7f77f3d1d640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77ec0d01b0 0x7f77ec1f5230 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.373+0000 7f77f3d1d640 1 --2- 172.21.15.55:0/2985139922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ec0cf690 0x7f77ec0cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.373+0000 7f77f3d1d640 1 -- 172.21.15.55:0/2985139922 >> 172.21.15.55:0/2985139922 conn(0x7f77ec0b6de0 msgr2=0x7f77ec06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.374+0000 7f77f3d1d640 1 -- 172.21.15.55:0/2985139922 shutdown_connections 2024-05-20T22:08:48.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.374+0000 7f77f3d1d640 1 -- 172.21.15.55:0/2985139922 wait complete. 2024-05-20T22:08:48.386 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:08:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.501+0000 7f91bd4f0640 1 Processor -- start 2024-05-20T22:08:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.501+0000 7f91bd4f0640 1 -- start start 2024-05-20T22:08:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.502+0000 7f91bd4f0640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f91b80b7600 0x7f91b8167c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.502+0000 7f91bd4f0640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b80b40d0 con 0x7f91b80b7d90 2024-05-20T22:08:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.502+0000 7f91bd4f0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b80b5b10 con 0x7f91b80b1ef0 2024-05-20T22:08:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.502+0000 7f91bd4f0640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f91b80b6160 con 0x7f91b80b7600 2024-05-20T22:08:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.502+0000 7f91b77fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f91b80b7600 0x7f91b8167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.502+0000 7f91b6ffd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f91b80b7d90 0x7f91b80b3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60564/0 (socket says 172.21.15.55:60564) 2024-05-20T22:08:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.502+0000 7f91b6ffd640 1 -- 172.21.15.55:0/1704985714 learned_addr learned my addr 172.21.15.55:0/1704985714 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.503+0000 7f91b77fe640 1 -- 172.21.15.55:0/1704985714 >> v1:172.21.15.138:6789/0 conn(0x7f91b80b1ef0 legacy=0x7f91b80d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.503+0000 7f91b77fe640 1 -- 172.21.15.55:0/1704985714 >> v1:172.21.15.55:6789/0 conn(0x7f91b80b7d90 legacy=0x7f91b80b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.503+0000 7f91b77fe640 1 -- 172.21.15.55:0/1704985714 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b80b5df0 con 0x7f91b80b7600 2024-05-20T22:08:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.503+0000 7f91b5ffb640 1 -- 172.21.15.55:0/1704985714 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3240037452 0 0) 0x7f91b80b40d0 con 0x7f91b80b7d90 2024-05-20T22:08:48.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.503+0000 7f91b77fe640 1 --2- 172.21.15.55:0/1704985714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b7600 0x7f91b8167c00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009d90 tx=0x7f91b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3479cccb828fa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:48.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.504+0000 7f91b5ffb640 1 -- 172.21.15.55:0/1704985714 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0007740 con 0x7f91b80b7600 2024-05-20T22:08:48.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.504+0000 7f91b5ffb640 1 -- 172.21.15.55:0/1704985714 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b00078e0 con 0x7f91b80b7600 2024-05-20T22:08:48.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.504+0000 7f91b5ffb640 1 -- 172.21.15.55:0/1704985714 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0007cb0 con 0x7f91b80b7600 2024-05-20T22:08:48.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.504+0000 7f91bd4f0640 1 -- 172.21.15.55:0/1704985714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b7600 msgr2=0x7f91b8167c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.504+0000 7f91bd4f0640 1 --2- 172.21.15.55:0/1704985714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b7600 0x7f91b8167c00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009d90 tx=0x7f91b002a6a0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.504+0000 7f91bd4f0640 1 -- 172.21.15.55:0/1704985714 shutdown_connections 2024-05-20T22:08:48.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.504+0000 7f91bd4f0640 1 --2- 172.21.15.55:0/1704985714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b7600 0x7f91b8167c00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.504+0000 7f91bd4f0640 1 -- 172.21.15.55:0/1704985714 >> 172.21.15.55:0/1704985714 conn(0x7f91b809be10 msgr2=0x7f91b80d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:48.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.504+0000 7f91bd4f0640 1 -- 172.21.15.55:0/1704985714 shutdown_connections 2024-05-20T22:08:48.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.504+0000 7f91bd4f0640 1 -- 172.21.15.55:0/1704985714 wait complete. 2024-05-20T22:08:48.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.506+0000 7f91bd4f0640 1 Processor -- start 2024-05-20T22:08:48.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.506+0000 7f91bd4f0640 1 -- start start 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91bd4f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b1ef0 0x7f91b814a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91bd4f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91b80b7600 0x7f91b81588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91bd4f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b80b7d90 0x7f91b815e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91bd4f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8168140 con 0x7f91b80b1ef0 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91bd4f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91b80d1920 con 0x7f91b80b7d90 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91bd4f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f91b81682c0 con 0x7f91b80b7600 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b1ef0 0x7f91b814a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b1ef0 0x7f91b814a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49816/0 (socket says 172.21.15.55:49816) 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b6ffd640 1 -- 172.21.15.55:0/3615259289 learned_addr learned my addr 172.21.15.55:0/3615259289 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b67fc640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91b80b7600 0x7f91b81588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b77fe640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b80b7d90 0x7f91b815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b6ffd640 1 -- 172.21.15.55:0/3615259289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91b80b7600 msgr2=0x7f91b81588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b6ffd640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91b80b7600 0x7f91b81588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b6ffd640 1 -- 172.21.15.55:0/3615259289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b80b7d90 msgr2=0x7f91b815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b6ffd640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b80b7d90 0x7f91b815e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b6ffd640 1 -- 172.21.15.55:0/3615259289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b80ce4e0 con 0x7f91b80b1ef0 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b77fe640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b80b7d90 0x7f91b815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91b6ffd640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b1ef0 0x7f91b814a850 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f91a800c410 tx=0x7f91a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f919f7fe640 1 -- 172.21.15.55:0/3615259289 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a80074e0 con 0x7f91b80b1ef0 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f919f7fe640 1 -- 172.21.15.55:0/3615259289 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a80047e0 con 0x7f91b80b1ef0 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f91bd4f0640 1 -- 172.21.15.55:0/3615259289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b80ce7c0 con 0x7f91b80b1ef0 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.508+0000 7f919f7fe640 1 -- 172.21.15.55:0/3615259289 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8004bd0 con 0x7f91b80b1ef0 2024-05-20T22:08:48.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.509+0000 7f91bd4f0640 1 -- 172.21.15.55:0/3615259289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b80ced80 con 0x7f91b80b1ef0 2024-05-20T22:08:48.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.509+0000 7f91bd4f0640 1 -- 172.21.15.55:0/3615259289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b8069090 con 0x7f91b80b1ef0 2024-05-20T22:08:48.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.510+0000 7f919f7fe640 1 -- 172.21.15.55:0/3615259289 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91a8011070 con 0x7f91b80b1ef0 2024-05-20T22:08:48.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.510+0000 7f919f7fe640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f91a406be60 0x7f91a406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.511+0000 7f919f7fe640 1 -- 172.21.15.55:0/3615259289 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f91a808adb0 con 0x7f91b80b1ef0 2024-05-20T22:08:48.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.511+0000 7f91b67fc640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f91a406be60 0x7f91a406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.511+0000 7f91b67fc640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f91a406be60 0x7f91a406e370 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f91ac006fd0 tx=0x7f91ac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:48.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.515+0000 7f919f7fe640 1 -- 172.21.15.55:0/3615259289 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f91a800e280 con 0x7f91b80b1ef0 2024-05-20T22:08:48.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.711+0000 7f91bd4f0640 1 -- 172.21.15.55:0/3615259289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f91b8167f20 con 0x7f91b80b1ef0 2024-05-20T22:08:48.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.712+0000 7f919f7fe640 1 -- 172.21.15.55:0/3615259289 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f91a80593f0 con 0x7f91b80b1ef0 2024-05-20T22:08:48.713 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-20T22:08:48.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 -- 172.21.15.55:0/3615259289 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f91a406be60 msgr2=0x7f91a406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f91a406be60 0x7f91a406e370 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f91ac006fd0 tx=0x7f91ac008040 comp rx=0 tx=0).stop 2024-05-20T22:08:48.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 -- 172.21.15.55:0/3615259289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b1ef0 msgr2=0x7f91b814a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b1ef0 0x7f91b814a850 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f91a800c410 tx=0x7f91a800c8d0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 -- 172.21.15.55:0/3615259289 shutdown_connections 2024-05-20T22:08:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f91a406be60 0x7f91a406e370 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b80b7d90 0x7f91b815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91b80b7600 0x7f91b81588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 --2- 172.21.15.55:0/3615259289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b80b1ef0 0x7f91b814a850 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 -- 172.21.15.55:0/3615259289 >> 172.21.15.55:0/3615259289 conn(0x7f91b809be10 msgr2=0x7f91b8071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 -- 172.21.15.55:0/3615259289 shutdown_connections 2024-05-20T22:08:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.714+0000 7f91bd4f0640 1 -- 172.21.15.55:0/3615259289 wait complete. 2024-05-20T22:08:48.726 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:08:48.726 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:48.746 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:48.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.859+0000 7f0172707640 1 Processor -- start 2024-05-20T22:08:48.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.859+0000 7f0172707640 1 -- start start 2024-05-20T22:08:48.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f0172707640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f016c0b7fe0 0x7f016c169f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f0172707640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016c0b83f0 con 0x7f016c15f100 2024-05-20T22:08:48.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f0172707640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016c0d3dd0 con 0x7f016c0b3b30 2024-05-20T22:08:48.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f0172707640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f016c06cea0 con 0x7f016c0b7fe0 2024-05-20T22:08:48.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f0170ca4640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f016c0b7fe0 0x7f016c169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f016b7fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f016c15f100 0x7f016c06c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53264/0 (socket says 172.21.15.55:53264) 2024-05-20T22:08:48.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f016b7fe640 1 -- 172.21.15.55:0/3891790162 learned_addr learned my addr 172.21.15.55:0/3891790162 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:48.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f0170ca4640 1 -- 172.21.15.55:0/3891790162 >> v1:172.21.15.138:6789/0 conn(0x7f016c0b3b30 legacy=0x7f016c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:48.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f0170ca4640 1 -- 172.21.15.55:0/3891790162 >> v1:172.21.15.55:6789/0 conn(0x7f016c15f100 legacy=0x7f016c06c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f0170ca4640 1 -- 172.21.15.55:0/3891790162 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c0d40b0 con 0x7f016c0b7fe0 2024-05-20T22:08:48.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.861+0000 7f016affd640 1 -- 172.21.15.55:0/3891790162 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917002169 0 0) 0x7f016c0b83f0 con 0x7f016c15f100 2024-05-20T22:08:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.862+0000 7f0170ca4640 1 --2- 172.21.15.55:0/3891790162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c0b7fe0 0x7f016c169f00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0164009d90 tx=0x7f016402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7a4785c1f45ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.862+0000 7f016affd640 1 -- 172.21.15.55:0/3891790162 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164007740 con 0x7f016c0b7fe0 2024-05-20T22:08:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.862+0000 7f016affd640 1 -- 172.21.15.55:0/3891790162 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01640078e0 con 0x7f016c0b7fe0 2024-05-20T22:08:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.862+0000 7f016affd640 1 -- 172.21.15.55:0/3891790162 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164007cb0 con 0x7f016c0b7fe0 2024-05-20T22:08:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.863+0000 7f0172707640 1 -- 172.21.15.55:0/3891790162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c0b7fe0 msgr2=0x7f016c169f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.863+0000 7f0172707640 1 --2- 172.21.15.55:0/3891790162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c0b7fe0 0x7f016c169f00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0164009d90 tx=0x7f016402a6a0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.863+0000 7f0172707640 1 -- 172.21.15.55:0/3891790162 shutdown_connections 2024-05-20T22:08:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.863+0000 7f0172707640 1 --2- 172.21.15.55:0/3891790162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c0b7fe0 0x7f016c169f00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.863+0000 7f0172707640 1 -- 172.21.15.55:0/3891790162 >> 172.21.15.55:0/3891790162 conn(0x7f016c0d51d0 msgr2=0x7f016c0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.863+0000 7f0172707640 1 -- 172.21.15.55:0/3891790162 shutdown_connections 2024-05-20T22:08:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.863+0000 7f0172707640 1 -- 172.21.15.55:0/3891790162 wait complete. 2024-05-20T22:08:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.865+0000 7f0172707640 1 Processor -- start 2024-05-20T22:08:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.865+0000 7f0172707640 1 -- start start 2024-05-20T22:08:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.866+0000 7f0172707640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016c0b3b30 0x7f016c156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.866+0000 7f0172707640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016c0b7fe0 0x7f016c1f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.866+0000 7f0172707640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c15f100 0x7f016c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.866+0000 7f0172707640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f016c06b880 con 0x7f016c15f100 2024-05-20T22:08:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.866+0000 7f0172707640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f016c06b5a0 con 0x7f016c0b7fe0 2024-05-20T22:08:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.866+0000 7f0172707640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f016c06bb60 con 0x7f016c0b3b30 2024-05-20T22:08:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.866+0000 7f0170ca4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c15f100 0x7f016c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.866+0000 7f0170ca4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c15f100 0x7f016c158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60154/0 (socket says 172.21.15.55:60154) 2024-05-20T22:08:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.866+0000 7f0170ca4640 1 -- 172.21.15.55:0/394676159 learned_addr learned my addr 172.21.15.55:0/394676159 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f016b7fe640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016c0b7fe0 0x7f016c1f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f016bfff640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016c0b3b30 0x7f016c156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0170ca4640 1 -- 172.21.15.55:0/394676159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016c0b3b30 msgr2=0x7f016c156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0170ca4640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016c0b3b30 0x7f016c156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0170ca4640 1 -- 172.21.15.55:0/394676159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016c0b7fe0 msgr2=0x7f016c1f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0170ca4640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016c0b7fe0 0x7f016c1f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0170ca4640 1 -- 172.21.15.55:0/394676159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c1f61c0 con 0x7f016c15f100 2024-05-20T22:08:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f016b7fe640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016c0b7fe0 0x7f016c1f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f016bfff640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016c0b3b30 0x7f016c156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0170ca4640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c15f100 0x7f016c158500 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f016402ac90 tx=0x7f0164004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0168ff9640 1 -- 172.21.15.55:0/394676159 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164040070 con 0x7f016c15f100 2024-05-20T22:08:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0168ff9640 1 -- 172.21.15.55:0/394676159 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0164031c90 con 0x7f016c15f100 2024-05-20T22:08:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0172707640 1 -- 172.21.15.55:0/394676159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c1f64a0 con 0x7f016c15f100 2024-05-20T22:08:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0168ff9640 1 -- 172.21.15.55:0/394676159 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164045680 con 0x7f016c15f100 2024-05-20T22:08:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.867+0000 7f0172707640 1 -- 172.21.15.55:0/394676159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c1f9200 con 0x7f016c15f100 2024-05-20T22:08:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.868+0000 7f0172707640 1 -- 172.21.15.55:0/394676159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016c069090 con 0x7f016c15f100 2024-05-20T22:08:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.869+0000 7f0168ff9640 1 -- 172.21.15.55:0/394676159 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0164039040 con 0x7f016c15f100 2024-05-20T22:08:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.869+0000 7f0168ff9640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f015806be60 0x7f015806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.870+0000 7f0168ff9640 1 -- 172.21.15.55:0/394676159 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f01640acc60 con 0x7f016c15f100 2024-05-20T22:08:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.870+0000 7f016bfff640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f015806be60 0x7f015806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.870+0000 7f016bfff640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f015806be60 0x7f015806e370 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f016c06d0c0 tx=0x7f015c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:48.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:48.874+0000 7f0168ff9640 1 -- 172.21.15.55:0/394676159 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f016407b2d0 con 0x7f016c15f100 2024-05-20T22:08:49.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.060+0000 7f0172707640 1 -- 172.21.15.55:0/394676159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f016c1539c0 con 0x7f016c15f100 2024-05-20T22:08:49.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.062+0000 7f0168ff9640 1 -- 172.21.15.55:0/394676159 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7f01640ac020 con 0x7f016c15f100 2024-05-20T22:08:49.063 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:49.063 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 -- 172.21.15.55:0/394676159 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f015806be60 msgr2=0x7f015806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f015806be60 0x7f015806e370 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f016c06d0c0 tx=0x7f015c008040 comp rx=0 tx=0).stop 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 -- 172.21.15.55:0/394676159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c15f100 msgr2=0x7f016c158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c15f100 0x7f016c158500 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f016402ac90 tx=0x7f0164004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 -- 172.21.15.55:0/394676159 shutdown_connections 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f015806be60 0x7f015806e370 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f016c15f100 0x7f016c158500 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016c0b7fe0 0x7f016c1f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 --2- 172.21.15.55:0/394676159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016c0b3b30 0x7f016c156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 -- 172.21.15.55:0/394676159 >> 172.21.15.55:0/394676159 conn(0x7f016c0d51d0 msgr2=0x7f016c06c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 -- 172.21.15.55:0/394676159 shutdown_connections 2024-05-20T22:08:49.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.064+0000 7f0172707640 1 -- 172.21.15.55:0/394676159 wait complete. 2024-05-20T22:08:49.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:49.076 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:49.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.194+0000 7fa88dfc7640 1 Processor -- start 2024-05-20T22:08:49.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.194+0000 7fa88dfc7640 1 -- start start 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.195+0000 7fa88dfc7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa8880cf680 0x7fa88815f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa88dfc7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88806cb70 0x7fa8880b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa88dfc7640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa88806b6a0 con 0x7fa8880b7fb0 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa88dfc7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa88806b980 con 0x7fa8880cf680 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa88dfc7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa88806bc60 con 0x7fa88806cb70 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa87effd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa8880b7fb0 0x7fa88806c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53266/0 (socket says 172.21.15.55:53266) 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa87effd640 1 -- 172.21.15.55:0/4173672560 learned_addr learned my addr 172.21.15.55:0/4173672560 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1797664552 0 0) 0x7fa88806b6a0 con 0x7fa8880b7fb0 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa8877fe640 1 --2- 172.21.15.55:0/4173672560 >> v2:172.21.15.138:3300/0 conn(0x7fa8880cf680 0x7fa88815f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa887fff640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88806cb70 0x7fa8880b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa874003520 con 0x7fa8880b7fb0 2024-05-20T22:08:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3924767000 0 0) 0x7fa874003520 con 0x7fa8880b7fb0 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88806cb70 msgr2=0x7fa8880b83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88806cb70 0x7fa8880b83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 >> v2:172.21.15.138:3300/0 conn(0x7fa8880cf680 msgr2=0x7fa88815f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 --2- 172.21.15.55:0/4173672560 >> v2:172.21.15.138:3300/0 conn(0x7fa8880cf680 0x7fa88815f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88806bf40 con 0x7fa8880b7fb0 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fa870001670 con 0x7fa8880b7fb0 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 >> v1:172.21.15.55:6789/0 conn(0x7fa8880b7fb0 legacy=0x7fa88806c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740039b0 0x7fa874003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88806cb70 0x7fa874004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa874004d40 0x7fa874005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa88806b980 con 0x7fa8740039b0 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa88806bc60 con 0x7fa874004d40 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.196+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa87400ba70 con 0x7fa88806cb70 2024-05-20T22:08:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.197+0000 7fa8877fe640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740039b0 0x7fa874003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.197+0000 7fa87effd640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88806cb70 0x7fa874004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.197+0000 7fa887fff640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa874004d40 0x7fa874005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.197+0000 7fa8877fe640 1 -- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88806cb70 msgr2=0x7fa874004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.197+0000 7fa8877fe640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88806cb70 0x7fa874004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.197+0000 7fa8877fe640 1 -- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa874004d40 msgr2=0x7fa874005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.197+0000 7fa8877fe640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa874004d40 0x7fa874005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.197+0000 7fa8877fe640 1 -- 172.21.15.55:0/4173672560 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa874003520 con 0x7fa8740039b0 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.198+0000 7fa8877fe640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740039b0 0x7fa874003e20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa878009ac0 tx=0x7fa874005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a3fd283ff654e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.198+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878036070 con 0x7fa8740039b0 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.198+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa878031400 con 0x7fa8740039b0 2024-05-20T22:08:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.198+0000 7fa886ffd640 1 -- 172.21.15.55:0/4173672560 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878009690 con 0x7fa8740039b0 2024-05-20T22:08:49.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.198+0000 7fa88dfc7640 1 -- 172.21.15.55:0/4173672560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740039b0 msgr2=0x7fa874003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.198+0000 7fa88dfc7640 1 --2- 172.21.15.55:0/4173672560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740039b0 0x7fa874003e20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa878009ac0 tx=0x7fa874005830 comp rx=0 tx=0).stop 2024-05-20T22:08:49.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.198+0000 7fa887fff640 1 -- 172.21.15.55:0/4173672560 reap_dead start 2024-05-20T22:08:49.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.198+0000 7fa88dfc7640 1 -- 172.21.15.55:0/4173672560 shutdown_connections 2024-05-20T22:08:49.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.198+0000 7fa88dfc7640 1 -- 172.21.15.55:0/4173672560 >> 172.21.15.55:0/4173672560 conn(0x7fa8880d5450 msgr2=0x7fa8880b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.198+0000 7fa88dfc7640 1 -- 172.21.15.55:0/4173672560 shutdown_connections 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.199+0000 7fa88dfc7640 1 -- 172.21.15.55:0/4173672560 wait complete. 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.200+0000 7fa88dfc7640 1 Processor -- start 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.200+0000 7fa88dfc7640 1 -- start start 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa88dfc7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8740039b0 0x7fa8881ea810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa88dfc7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8881f8670 0x7fa8881f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa88dfc7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88815cb90 0x7fa88815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa88dfc7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa8880b8540 con 0x7fa8881f8670 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa88dfc7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8880b83c0 con 0x7fa8740039b0 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa88dfc7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa8880b86c0 con 0x7fa88815cb90 2024-05-20T22:08:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa87effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8881f8670 0x7fa8881f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa87effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8881f8670 0x7fa8881f8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60174/0 (socket says 172.21.15.55:60174) 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa87effd640 1 -- 172.21.15.55:0/361965215 learned_addr learned my addr 172.21.15.55:0/361965215 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa8877fe640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8740039b0 0x7fa8881ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa887fff640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88815cb90 0x7fa88815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa87effd640 1 -- 172.21.15.55:0/361965215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88815cb90 msgr2=0x7fa88815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa87effd640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88815cb90 0x7fa88815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa87effd640 1 -- 172.21.15.55:0/361965215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8740039b0 msgr2=0x7fa8881ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa87effd640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8740039b0 0x7fa8881ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa87effd640 1 -- 172.21.15.55:0/361965215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88815d4e0 con 0x7fa8881f8670 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.202+0000 7fa87effd640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8881f8670 0x7fa8881f8a80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa870007340 tx=0x7fa870007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.203+0000 7fa884ff9640 1 -- 172.21.15.55:0/361965215 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87000f940 con 0x7fa8881f8670 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.203+0000 7fa884ff9640 1 -- 172.21.15.55:0/361965215 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870018070 con 0x7fa8881f8670 2024-05-20T22:08:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.203+0000 7fa88dfc7640 1 -- 172.21.15.55:0/361965215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8881f9210 con 0x7fa8881f8670 2024-05-20T22:08:49.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.203+0000 7fa884ff9640 1 -- 172.21.15.55:0/361965215 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87000fd40 con 0x7fa8881f8670 2024-05-20T22:08:49.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.203+0000 7fa88dfc7640 1 -- 172.21.15.55:0/361965215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8881f6250 con 0x7fa8881f8670 2024-05-20T22:08:49.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.204+0000 7fa88dfc7640 1 -- 172.21.15.55:0/361965215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa888069090 con 0x7fa8881f8670 2024-05-20T22:08:49.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.204+0000 7fa884ff9640 1 -- 172.21.15.55:0/361965215 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa87001d460 con 0x7fa8881f8670 2024-05-20T22:08:49.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.205+0000 7fa884ff9640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa86806be30 0x7fa86806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.205+0000 7fa884ff9640 1 -- 172.21.15.55:0/361965215 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa87008b1c0 con 0x7fa8881f8670 2024-05-20T22:08:49.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.205+0000 7fa8877fe640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa86806be30 0x7fa86806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.205+0000 7fa8877fe640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa86806be30 0x7fa86806e340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa878033000 tx=0x7fa878008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:49.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.209+0000 7fa884ff9640 1 -- 172.21.15.55:0/361965215 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa88815d4e0 con 0x7fa8881f8670 2024-05-20T22:08:49.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.392+0000 7fa88dfc7640 1 -- 172.21.15.55:0/361965215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa8880d2e90 con 0x7fa8881f8670 2024-05-20T22:08:49.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.392+0000 7fa884ff9640 1 -- 172.21.15.55:0/361965215 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fa870059120 con 0x7fa8881f8670 2024-05-20T22:08:49.393 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:49.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 -- 172.21.15.55:0/361965215 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa86806be30 msgr2=0x7fa86806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa86806be30 0x7fa86806e340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa878033000 tx=0x7fa878008030 comp rx=0 tx=0).stop 2024-05-20T22:08:49.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 -- 172.21.15.55:0/361965215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8881f8670 msgr2=0x7fa8881f8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8881f8670 0x7fa8881f8a80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa870007340 tx=0x7fa870007800 comp rx=0 tx=0).stop 2024-05-20T22:08:49.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 -- 172.21.15.55:0/361965215 shutdown_connections 2024-05-20T22:08:49.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa86806be30 0x7fa86806e340 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa88815cb90 0x7fa88815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8881f8670 0x7fa8881f8a80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 --2- 172.21.15.55:0/361965215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8740039b0 0x7fa8881ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 -- 172.21.15.55:0/361965215 >> 172.21.15.55:0/361965215 conn(0x7fa8880d5450 msgr2=0x7fa8880b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:49.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 -- 172.21.15.55:0/361965215 shutdown_connections 2024-05-20T22:08:49.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.395+0000 7fa88dfc7640 1 -- 172.21.15.55:0/361965215 wait complete. 2024-05-20T22:08:49.407 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:49.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.524+0000 7f8d4e55e640 1 Processor -- start 2024-05-20T22:08:49.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.524+0000 7f8d4e55e640 1 -- start start 2024-05-20T22:08:49.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.525+0000 7f8d4e55e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d480d32a0 0x7f8d4815f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.525+0000 7f8d4e55e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d480cf690 0x7f8d480cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.525+0000 7f8d4e55e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8d480cffe0 0x7f8d48162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.525+0000 7f8d4e55e640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f8d480d3780 con 0x7f8d480cf690 2024-05-20T22:08:49.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.525+0000 7f8d4e55e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8d480d3a60 con 0x7f8d480cffe0 2024-05-20T22:08:49.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.525+0000 7f8d4e55e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d480d3d40 con 0x7f8d480d32a0 2024-05-20T22:08:49.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d477fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d480cf690 0x7f8d480cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d477fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d480cf690 0x7f8d480cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60176/0 (socket says 172.21.15.55:60176) 2024-05-20T22:08:49.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d477fe640 1 -- 172.21.15.55:0/2064614271 learned_addr learned my addr 172.21.15.55:0/2064614271 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:49.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d47fff640 1 --2- 172.21.15.55:0/2064614271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d480d32a0 0x7f8d4815f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d4cafb640 1 --2- 172.21.15.55:0/2064614271 >> v2:172.21.15.138:3300/0 conn(0x7f8d480cffe0 0x7f8d48162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d477fe640 1 -- 172.21.15.55:0/2064614271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d480d32a0 msgr2=0x7f8d4815f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:49.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d477fe640 1 --2- 172.21.15.55:0/2064614271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d480d32a0 0x7f8d4815f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d477fe640 1 -- 172.21.15.55:0/2064614271 >> v2:172.21.15.138:3300/0 conn(0x7f8d480cffe0 msgr2=0x7f8d48162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:49.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d477fe640 1 --2- 172.21.15.55:0/2064614271 >> v2:172.21.15.138:3300/0 conn(0x7f8d480cffe0 0x7f8d48162cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d477fe640 1 -- 172.21.15.55:0/2064614271 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d480d4020 con 0x7f8d480cf690 2024-05-20T22:08:49.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.526+0000 7f8d4cafb640 1 --2- 172.21.15.55:0/2064614271 >> v2:172.21.15.138:3300/0 conn(0x7f8d480cffe0 0x7f8d48162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:49.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.527+0000 7f8d477fe640 1 --2- 172.21.15.55:0/2064614271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480cf690 0x7f8d480cfaa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8d480cf450 tx=0x7f8d3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa495cd7e3f3dda server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:49.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.527+0000 7f8d46ffd640 1 -- 172.21.15.55:0/2064614271 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c037070 con 0x7f8d480cf690 2024-05-20T22:08:49.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.527+0000 7f8d46ffd640 1 -- 172.21.15.55:0/2064614271 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3c032440 con 0x7f8d480cf690 2024-05-20T22:08:49.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.527+0000 7f8d46ffd640 1 -- 172.21.15.55:0/2064614271 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c032810 con 0x7f8d480cf690 2024-05-20T22:08:49.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.528+0000 7f8d4e55e640 1 -- 172.21.15.55:0/2064614271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480cf690 msgr2=0x7f8d480cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.528+0000 7f8d4e55e640 1 --2- 172.21.15.55:0/2064614271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480cf690 0x7f8d480cfaa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8d480cf450 tx=0x7f8d3c02a1e0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.528+0000 7f8d4e55e640 1 -- 172.21.15.55:0/2064614271 shutdown_connections 2024-05-20T22:08:49.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.528+0000 7f8d4e55e640 1 --2- 172.21.15.55:0/2064614271 >> v2:172.21.15.138:3300/0 conn(0x7f8d480cffe0 0x7f8d48162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.528+0000 7f8d4e55e640 1 --2- 172.21.15.55:0/2064614271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480cf690 0x7f8d480cfaa0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.528+0000 7f8d4e55e640 1 --2- 172.21.15.55:0/2064614271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d480d32a0 0x7f8d4815f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.528+0000 7f8d4e55e640 1 -- 172.21.15.55:0/2064614271 >> 172.21.15.55:0/2064614271 conn(0x7f8d480b6de0 msgr2=0x7f8d480d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:49.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.528+0000 7f8d4e55e640 1 -- 172.21.15.55:0/2064614271 shutdown_connections 2024-05-20T22:08:49.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.528+0000 7f8d4e55e640 1 -- 172.21.15.55:0/2064614271 wait complete. 2024-05-20T22:08:49.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.530+0000 7f8d4e55e640 1 Processor -- start 2024-05-20T22:08:49.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.530+0000 7f8d4e55e640 1 -- start start 2024-05-20T22:08:49.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d4e55e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d480cf690 0x7f8d48153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d4e55e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d480cffe0 0x7f8d480c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d4e55e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480d32a0 0x7f8d480cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d4e55e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8d480d4820 con 0x7f8d480d32a0 2024-05-20T22:08:49.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d4e55e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d480d03f0 con 0x7f8d480cf690 2024-05-20T22:08:49.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d4e55e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d480cc500 con 0x7f8d480cffe0 2024-05-20T22:08:49.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d4cafb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480d32a0 0x7f8d480cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d4cafb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480d32a0 0x7f8d480cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60180/0 (socket says 172.21.15.55:60180) 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d4cafb640 1 -- 172.21.15.55:0/4157975037 learned_addr learned my addr 172.21.15.55:0/4157975037 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d47fff640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d480cf690 0x7f8d48153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.531+0000 7f8d477fe640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d480cffe0 0x7f8d480c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d4cafb640 1 -- 172.21.15.55:0/4157975037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d480cffe0 msgr2=0x7f8d480c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d4cafb640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d480cffe0 0x7f8d480c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d4cafb640 1 -- 172.21.15.55:0/4157975037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d480cf690 msgr2=0x7f8d48153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d4cafb640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d480cf690 0x7f8d48153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d4cafb640 1 -- 172.21.15.55:0/4157975037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d480c9f70 con 0x7f8d480d32a0 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d4cafb640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480d32a0 0x7f8d480cb010 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8d40007d20 tx=0x7f8d4000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d44ff9640 1 -- 172.21.15.55:0/4157975037 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4001a070 con 0x7f8d480d32a0 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d44ff9640 1 -- 172.21.15.55:0/4157975037 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d40015400 con 0x7f8d480d32a0 2024-05-20T22:08:49.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d4e55e640 1 -- 172.21.15.55:0/4157975037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d480ca250 con 0x7f8d480d32a0 2024-05-20T22:08:49.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d44ff9640 1 -- 172.21.15.55:0/4157975037 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d400157f0 con 0x7f8d480d32a0 2024-05-20T22:08:49.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.532+0000 7f8d4e55e640 1 -- 172.21.15.55:0/4157975037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d480ca810 con 0x7f8d480d32a0 2024-05-20T22:08:49.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.533+0000 7f8d4e55e640 1 -- 172.21.15.55:0/4157975037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d48069090 con 0x7f8d480d32a0 2024-05-20T22:08:49.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.534+0000 7f8d44ff9640 1 -- 172.21.15.55:0/4157975037 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d4001d070 con 0x7f8d480d32a0 2024-05-20T22:08:49.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.534+0000 7f8d44ff9640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d3406be30 0x7f8d3406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.534+0000 7f8d44ff9640 1 -- 172.21.15.55:0/4157975037 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8d4008b3d0 con 0x7f8d480d32a0 2024-05-20T22:08:49.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.534+0000 7f8d47fff640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d3406be30 0x7f8d3406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.535+0000 7f8d47fff640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d3406be30 0x7f8d3406e340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006fd0 tx=0x7f8d38008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:49.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.539+0000 7f8d44ff9640 1 -- 172.21.15.55:0/4157975037 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8d40010070 con 0x7f8d480d32a0 2024-05-20T22:08:49.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.726+0000 7f8d4e55e640 1 -- 172.21.15.55:0/4157975037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8d480cec80 con 0x7f8d480d32a0 2024-05-20T22:08:49.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.726+0000 7f8d44ff9640 1 -- 172.21.15.55:0/4157975037 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f8d400592a0 con 0x7f8d480d32a0 2024-05-20T22:08:49.727 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:49.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 -- 172.21.15.55:0/4157975037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d3406be30 msgr2=0x7f8d3406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d3406be30 0x7f8d3406e340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006fd0 tx=0x7f8d38008040 comp rx=0 tx=0).stop 2024-05-20T22:08:49.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 -- 172.21.15.55:0/4157975037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480d32a0 msgr2=0x7f8d480cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480d32a0 0x7f8d480cb010 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8d40007d20 tx=0x7f8d4000d4a0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 -- 172.21.15.55:0/4157975037 shutdown_connections 2024-05-20T22:08:49.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d3406be30 0x7f8d3406e340 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d480d32a0 0x7f8d480cb010 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d480cffe0 0x7f8d480c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 --2- 172.21.15.55:0/4157975037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d480cf690 0x7f8d48153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 -- 172.21.15.55:0/4157975037 >> 172.21.15.55:0/4157975037 conn(0x7f8d480b6de0 msgr2=0x7f8d480b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:49.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 -- 172.21.15.55:0/4157975037 shutdown_connections 2024-05-20T22:08:49.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.729+0000 7f8d4e55e640 1 -- 172.21.15.55:0/4157975037 wait complete. 2024-05-20T22:08:49.741 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:49.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.858+0000 7f62bd0e2640 1 Processor -- start 2024-05-20T22:08:49.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.858+0000 7f62bd0e2640 1 -- start start 2024-05-20T22:08:49.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62bd0e2640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f62b80b29e0 0x7f62b815f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62bd0e2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62b808bcb0 0x7f62b80cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62bd0e2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f62b808c0c0 0x7f62b80cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62bd0e2640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f62b80b2df0 con 0x7f62b80b29e0 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62bd0e2640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f62b808c690 con 0x7f62b808c0c0 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62bd0e2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f62b808c970 con 0x7f62b808bcb0 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62b6d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f62b80b29e0 0x7f62b815f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62b6d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f62b80b29e0 0x7f62b815f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60188/0 (socket says 172.21.15.55:60188) 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62b6d9d640 1 -- 172.21.15.55:0/317350073 learned_addr learned my addr 172.21.15.55:0/317350073 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62b759e640 1 --2- 172.21.15.55:0/317350073 >> v2:172.21.15.138:3300/0 conn(0x7f62b808c0c0 0x7f62b80cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.859+0000 7f62b659c640 1 --2- 172.21.15.55:0/317350073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62b808bcb0 0x7f62b80cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.860+0000 7f62b6d9d640 1 -- 172.21.15.55:0/317350073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62b808bcb0 msgr2=0x7f62b80cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.860+0000 7f62b6d9d640 1 --2- 172.21.15.55:0/317350073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62b808bcb0 0x7f62b80cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.860+0000 7f62b6d9d640 1 -- 172.21.15.55:0/317350073 >> v2:172.21.15.138:3300/0 conn(0x7f62b808c0c0 msgr2=0x7f62b80cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:49.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.860+0000 7f62b6d9d640 1 --2- 172.21.15.55:0/317350073 >> v2:172.21.15.138:3300/0 conn(0x7f62b808c0c0 0x7f62b80cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.860+0000 7f62b6d9d640 1 -- 172.21.15.55:0/317350073 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b80d0de0 con 0x7f62b80b29e0 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.860+0000 7f62b6d9d640 1 --2- 172.21.15.55:0/317350073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 0x7f62b815f0e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62b80b2790 tx=0x7f62a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef26f1160ea608b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62b5d9b640 1 -- 172.21.15.55:0/317350073 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8037070 con 0x7f62b80b29e0 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62b5d9b640 1 -- 172.21.15.55:0/317350073 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a8032440 con 0x7f62b80b29e0 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62b5d9b640 1 -- 172.21.15.55:0/317350073 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8032810 con 0x7f62b80b29e0 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62bd0e2640 1 -- 172.21.15.55:0/317350073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 msgr2=0x7f62b815f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62bd0e2640 1 --2- 172.21.15.55:0/317350073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 0x7f62b815f0e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62b80b2790 tx=0x7f62a802a1e0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62bd0e2640 1 -- 172.21.15.55:0/317350073 shutdown_connections 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62bd0e2640 1 --2- 172.21.15.55:0/317350073 >> v2:172.21.15.138:3300/0 conn(0x7f62b808c0c0 0x7f62b80cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62bd0e2640 1 --2- 172.21.15.55:0/317350073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62b808bcb0 0x7f62b80cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62bd0e2640 1 --2- 172.21.15.55:0/317350073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 0x7f62b815f0e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62bd0e2640 1 -- 172.21.15.55:0/317350073 >> 172.21.15.55:0/317350073 conn(0x7f62b80d2860 msgr2=0x7f62b80d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62bd0e2640 1 -- 172.21.15.55:0/317350073 shutdown_connections 2024-05-20T22:08:49.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.861+0000 7f62bd0e2640 1 -- 172.21.15.55:0/317350073 wait complete. 2024-05-20T22:08:49.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.863+0000 7f62bd0e2640 1 Processor -- start 2024-05-20T22:08:49.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.863+0000 7f62bd0e2640 1 -- start start 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62bd0e2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b808bcb0 0x7f62b80c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62bd0e2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62b808c0c0 0x7f62b80c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62bd0e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 0x7f62b81f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62bd0e2640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f62b808cc50 con 0x7f62b80b29e0 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62bd0e2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62b815f620 con 0x7f62b808bcb0 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62bd0e2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f62b80b80e0 con 0x7f62b808c0c0 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b759e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 0x7f62b81f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b759e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 0x7f62b81f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60194/0 (socket says 172.21.15.55:60194) 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b759e640 1 -- 172.21.15.55:0/1388502095 learned_addr learned my addr 172.21.15.55:0/1388502095 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b659c640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62b808c0c0 0x7f62b80c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b6d9d640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b808bcb0 0x7f62b80c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b759e640 1 -- 172.21.15.55:0/1388502095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62b808c0c0 msgr2=0x7f62b80c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b759e640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62b808c0c0 0x7f62b80c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b759e640 1 -- 172.21.15.55:0/1388502095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b808bcb0 msgr2=0x7f62b80c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b759e640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b808bcb0 0x7f62b80c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b759e640 1 -- 172.21.15.55:0/1388502095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b80b5c20 con 0x7f62b80b29e0 2024-05-20T22:08:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b6d9d640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b808bcb0 0x7f62b80c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.865+0000 7f62b759e640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 0x7f62b81f4fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009cb0 tx=0x7f62b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.866+0000 7f629f7fe640 1 -- 172.21.15.55:0/1388502095 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0018020 con 0x7f62b80b29e0 2024-05-20T22:08:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.866+0000 7f629f7fe640 1 -- 172.21.15.55:0/1388502095 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b0014480 con 0x7f62b80b29e0 2024-05-20T22:08:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.866+0000 7f62bd0e2640 1 -- 172.21.15.55:0/1388502095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b80b5f00 con 0x7f62b80b29e0 2024-05-20T22:08:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.866+0000 7f629f7fe640 1 -- 172.21.15.55:0/1388502095 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0014870 con 0x7f62b80b29e0 2024-05-20T22:08:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.866+0000 7f62bd0e2640 1 -- 172.21.15.55:0/1388502095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b814c910 con 0x7f62b80b29e0 2024-05-20T22:08:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.867+0000 7f62bd0e2640 1 -- 172.21.15.55:0/1388502095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b814cb70 con 0x7f62b80b29e0 2024-05-20T22:08:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.867+0000 7f629f7fe640 1 -- 172.21.15.55:0/1388502095 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62b001b020 con 0x7f62b80b29e0 2024-05-20T22:08:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.868+0000 7f629f7fe640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f62a406be30 0x7f62a406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.868+0000 7f629f7fe640 1 -- 172.21.15.55:0/1388502095 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f62b008bf10 con 0x7f62b80b29e0 2024-05-20T22:08:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.868+0000 7f62b6d9d640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f62a406be30 0x7f62a406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.869+0000 7f62b6d9d640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f62a406be30 0x7f62a406e340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f62a8034000 tx=0x7f62a802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:49.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:49.873+0000 7f629f7fe640 1 -- 172.21.15.55:0/1388502095 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f62b0022020 con 0x7f62b80b29e0 2024-05-20T22:08:50.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.058+0000 7f62bd0e2640 1 -- 172.21.15.55:0/1388502095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f62b80c6990 con 0x7f62b80b29e0 2024-05-20T22:08:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.059+0000 7f629f7fe640 1 -- 172.21.15.55:0/1388502095 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f62b0010040 con 0x7f62b80b29e0 2024-05-20T22:08:50.060 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 -- 172.21.15.55:0/1388502095 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f62a406be30 msgr2=0x7f62a406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f62a406be30 0x7f62a406e340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f62a8034000 tx=0x7f62a802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 -- 172.21.15.55:0/1388502095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 msgr2=0x7f62b81f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 0x7f62b81f4fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009cb0 tx=0x7f62b000c490 comp rx=0 tx=0).stop 2024-05-20T22:08:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 -- 172.21.15.55:0/1388502095 shutdown_connections 2024-05-20T22:08:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f62a406be30 0x7f62a406e340 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62b80b29e0 0x7f62b81f4fe0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62b808c0c0 0x7f62b80c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 --2- 172.21.15.55:0/1388502095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b808bcb0 0x7f62b80c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 -- 172.21.15.55:0/1388502095 >> 172.21.15.55:0/1388502095 conn(0x7f62b80d2860 msgr2=0x7f62b806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:50.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 -- 172.21.15.55:0/1388502095 shutdown_connections 2024-05-20T22:08:50.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.061+0000 7f62bd0e2640 1 -- 172.21.15.55:0/1388502095 wait complete. 2024-05-20T22:08:50.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:50.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.190+0000 7fb9afdce640 1 Processor -- start 2024-05-20T22:08:50.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.190+0000 7fb9afdce640 1 -- start start 2024-05-20T22:08:50.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.191+0000 7fb9afdce640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9a809ac90 0x7fb9a80d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.191+0000 7fb9afdce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 0x7fb9a80d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.191+0000 7fb9afdce640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a80b7f00 con 0x7fb9a80d12f0 2024-05-20T22:08:50.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.191+0000 7fb9afdce640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a80d1700 con 0x7fb9a809ac90 2024-05-20T22:08:50.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.191+0000 7fb9afdce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a80ce6d0 con 0x7fb9a809b0a0 2024-05-20T22:08:50.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.191+0000 7fb9ad369640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 0x7fb9a80d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.191+0000 7fb9adb6a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9a809ac90 0x7fb9a80d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.192+0000 7fb9ad369640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 0x7fb9a80d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:34122/0 (socket says 172.21.15.55:34122) 2024-05-20T22:08:50.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.192+0000 7fb9ad369640 1 -- 172.21.15.55:0/965424984 learned_addr learned my addr 172.21.15.55:0/965424984 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:50.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.192+0000 7fb9adb6a640 1 -- 172.21.15.55:0/965424984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 msgr2=0x7fb9a80d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:50.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.192+0000 7fb9adb6a640 1 --2- 172.21.15.55:0/965424984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 0x7fb9a80d0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.192+0000 7fb9adb6a640 1 -- 172.21.15.55:0/965424984 >> v1:172.21.15.138:6789/0 conn(0x7fb9a80d12f0 legacy=0x7fb9a8161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.192+0000 7fb9adb6a640 1 -- 172.21.15.55:0/965424984 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a80ce080 con 0x7fb9a809ac90 2024-05-20T22:08:50.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.192+0000 7fb9ad369640 1 --2- 172.21.15.55:0/965424984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 0x7fb9a80d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:50.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.193+0000 7fb9adb6a640 1 --2- 172.21.15.55:0/965424984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a809ac90 0x7fb9a80d0870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009d90 tx=0x7fb9a402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bdb0eb245d805bc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:50.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.193+0000 7fb9acb68640 1 -- 172.21.15.55:0/965424984 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4031600 con 0x7fb9a809ac90 2024-05-20T22:08:50.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.193+0000 7fb9acb68640 1 -- 172.21.15.55:0/965424984 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a40317a0 con 0x7fb9a809ac90 2024-05-20T22:08:50.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.193+0000 7fb9acb68640 1 -- 172.21.15.55:0/965424984 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4031b70 con 0x7fb9a809ac90 2024-05-20T22:08:50.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.194+0000 7fb9afdce640 1 -- 172.21.15.55:0/965424984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a809ac90 msgr2=0x7fb9a80d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.194+0000 7fb9afdce640 1 --2- 172.21.15.55:0/965424984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a809ac90 0x7fb9a80d0870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009d90 tx=0x7fb9a402a5d0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.194+0000 7fb9afdce640 1 -- 172.21.15.55:0/965424984 shutdown_connections 2024-05-20T22:08:50.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.194+0000 7fb9afdce640 1 --2- 172.21.15.55:0/965424984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 0x7fb9a80d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.194+0000 7fb9afdce640 1 --2- 172.21.15.55:0/965424984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a809ac90 0x7fb9a80d0870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.194+0000 7fb9afdce640 1 -- 172.21.15.55:0/965424984 >> 172.21.15.55:0/965424984 conn(0x7fb9a80b43f0 msgr2=0x7fb9a80b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:50.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.194+0000 7fb9afdce640 1 -- 172.21.15.55:0/965424984 shutdown_connections 2024-05-20T22:08:50.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.194+0000 7fb9afdce640 1 -- 172.21.15.55:0/965424984 wait complete. 2024-05-20T22:08:50.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.195+0000 7fb9afdce640 1 Processor -- start 2024-05-20T22:08:50.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.196+0000 7fb9afdce640 1 -- start start 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9afdce640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a809ac90 0x7fb9a806da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9afdce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 0x7fb9a806df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9afdce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a80d12f0 0x7fb9a80cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9afdce640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80b81c0 con 0x7fb9a809ac90 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9afdce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a808b990 con 0x7fb9a80d12f0 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9afdce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a809bb70 con 0x7fb9a809b0a0 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9adb6a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a809ac90 0x7fb9a806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9adb6a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a809ac90 0x7fb9a806da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60204/0 (socket says 172.21.15.55:60204) 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9adb6a640 1 -- 172.21.15.55:0/1768750928 learned_addr learned my addr 172.21.15.55:0/1768750928 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9ad369640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 0x7fb9a806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9ae36b640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a80d12f0 0x7fb9a80cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9adb6a640 1 -- 172.21.15.55:0/1768750928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 msgr2=0x7fb9a806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9adb6a640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 0x7fb9a806df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9adb6a640 1 -- 172.21.15.55:0/1768750928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a80d12f0 msgr2=0x7fb9a80cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9adb6a640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a80d12f0 0x7fb9a80cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.197+0000 7fb9adb6a640 1 -- 172.21.15.55:0/1768750928 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a806e7e0 con 0x7fb9a809ac90 2024-05-20T22:08:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.198+0000 7fb9adb6a640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a809ac90 0x7fb9a806da40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009eb0 tx=0x7fb9a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.198+0000 7fb99e7fc640 1 -- 172.21.15.55:0/1768750928 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a403f070 con 0x7fb9a809ac90 2024-05-20T22:08:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.198+0000 7fb99e7fc640 1 -- 172.21.15.55:0/1768750928 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a4030be0 con 0x7fb9a809ac90 2024-05-20T22:08:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.198+0000 7fb99e7fc640 1 -- 172.21.15.55:0/1768750928 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4031e60 con 0x7fb9a809ac90 2024-05-20T22:08:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.198+0000 7fb9afdce640 1 -- 172.21.15.55:0/1768750928 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a80ca340 con 0x7fb9a809ac90 2024-05-20T22:08:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.198+0000 7fb9afdce640 1 -- 172.21.15.55:0/1768750928 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a80ca900 con 0x7fb9a809ac90 2024-05-20T22:08:50.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.199+0000 7fb9afdce640 1 -- 172.21.15.55:0/1768750928 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a8069090 con 0x7fb9a809ac90 2024-05-20T22:08:50.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.200+0000 7fb99e7fc640 1 -- 172.21.15.55:0/1768750928 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9a403b410 con 0x7fb9a809ac90 2024-05-20T22:08:50.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.200+0000 7fb99e7fc640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb98c06be30 0x7fb98c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.200+0000 7fb99e7fc640 1 -- 172.21.15.55:0/1768750928 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb9a40ac7c0 con 0x7fb9a809ac90 2024-05-20T22:08:50.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.200+0000 7fb9ad369640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb98c06be30 0x7fb98c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.201+0000 7fb9ad369640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb98c06be30 0x7fb98c06e340 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb9a806ecd0 tx=0x7fb998009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:50.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.205+0000 7fb99e7fc640 1 -- 172.21.15.55:0/1768750928 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb9a407a690 con 0x7fb9a809ac90 2024-05-20T22:08:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.387+0000 7fb9afdce640 1 -- 172.21.15.55:0/1768750928 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb9a809b720 con 0x7fb9a809ac90 2024-05-20T22:08:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.387+0000 7fb99e7fc640 1 -- 172.21.15.55:0/1768750928 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb9a407e990 con 0x7fb9a809ac90 2024-05-20T22:08:50.388 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:50.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 -- 172.21.15.55:0/1768750928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb98c06be30 msgr2=0x7fb98c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb98c06be30 0x7fb98c06e340 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb9a806ecd0 tx=0x7fb998009240 comp rx=0 tx=0).stop 2024-05-20T22:08:50.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 -- 172.21.15.55:0/1768750928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a809ac90 msgr2=0x7fb9a806da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a809ac90 0x7fb9a806da40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009eb0 tx=0x7fb9a4004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 -- 172.21.15.55:0/1768750928 shutdown_connections 2024-05-20T22:08:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb98c06be30 0x7fb98c06e340 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a80d12f0 0x7fb9a80cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a809b0a0 0x7fb9a806df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 --2- 172.21.15.55:0/1768750928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a809ac90 0x7fb9a806da40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 -- 172.21.15.55:0/1768750928 >> 172.21.15.55:0/1768750928 conn(0x7fb9a80b43f0 msgr2=0x7fb9a80b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 -- 172.21.15.55:0/1768750928 shutdown_connections 2024-05-20T22:08:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.389+0000 7fb9afdce640 1 -- 172.21.15.55:0/1768750928 wait complete. 2024-05-20T22:08:50.402 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:08:50.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.517+0000 7f5b5e2b6640 1 Processor -- start 2024-05-20T22:08:50.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.517+0000 7f5b5e2b6640 1 -- start start 2024-05-20T22:08:50.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.518+0000 7f5b5e2b6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b580b4080 0x7f5b5815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.518+0000 7f5b5e2b6640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5b580d3290 0x7f5b580d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.518+0000 7f5b5e2b6640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b5806b500 con 0x7f5b580b35e0 2024-05-20T22:08:50.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.518+0000 7f5b5e2b6640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f5b58163960 con 0x7f5b580d3290 2024-05-20T22:08:50.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.518+0000 7f5b5e2b6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5b580b3e10 con 0x7f5b580b4080 2024-05-20T22:08:50.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.518+0000 7f5b5c853640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5b580d3290 0x7f5b580d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.518+0000 7f5b4ffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5b580b35e0 0x7f5b5815f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53280/0 (socket says 172.21.15.55:53280) 2024-05-20T22:08:50.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.519+0000 7f5b5c853640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5b580d3290 0x7f5b580d36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60206/0 (socket says 172.21.15.55:60206) 2024-05-20T22:08:50.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.519+0000 7f5b4ffff640 1 -- 172.21.15.55:0/1389063298 learned_addr learned my addr 172.21.15.55:0/1389063298 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:50.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.519+0000 7f5b57fff640 1 --2- 172.21.15.55:0/1389063298 >> v2:172.21.15.138:3300/0 conn(0x7f5b580b4080 0x7f5b5815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.519+0000 7f5b5c853640 1 -- 172.21.15.55:0/1389063298 >> v2:172.21.15.138:3300/0 conn(0x7f5b580b4080 msgr2=0x7f5b5815f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:50.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.519+0000 7f5b5c853640 1 --2- 172.21.15.55:0/1389063298 >> v2:172.21.15.138:3300/0 conn(0x7f5b580b4080 0x7f5b5815f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.519+0000 7f5b5c853640 1 -- 172.21.15.55:0/1389063298 >> v1:172.21.15.55:6789/0 conn(0x7f5b580b35e0 legacy=0x7f5b5815f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.519+0000 7f5b5c853640 1 -- 172.21.15.55:0/1389063298 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b580b3b30 con 0x7f5b580d3290 2024-05-20T22:08:50.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.519+0000 7f5b577fe640 1 -- 172.21.15.55:0/1389063298 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1958063637 0 0) 0x7f5b40001110 con 0x7f5b580b35e0 2024-05-20T22:08:50.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.519+0000 7f5b57fff640 1 --2- 172.21.15.55:0/1389063298 >> v2:172.21.15.138:3300/0 conn(0x7f5b580b4080 0x7f5b5815f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:50.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.520+0000 7f5b5c853640 1 --2- 172.21.15.55:0/1389063298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580d3290 0x7f5b580d36a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5b5808d7c0 tx=0x7f5b5002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67eaf6c4c6339523 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:50.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.520+0000 7f5b577fe640 1 -- 172.21.15.55:0/1389063298 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50007740 con 0x7f5b580d3290 2024-05-20T22:08:50.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.520+0000 7f5b577fe640 1 -- 172.21.15.55:0/1389063298 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b500078e0 con 0x7f5b580d3290 2024-05-20T22:08:50.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.520+0000 7f5b577fe640 1 -- 172.21.15.55:0/1389063298 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50007cb0 con 0x7f5b580d3290 2024-05-20T22:08:50.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.521+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/1389063298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580d3290 msgr2=0x7f5b580d36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.521+0000 7f5b5e2b6640 1 --2- 172.21.15.55:0/1389063298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580d3290 0x7f5b580d36a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5b5808d7c0 tx=0x7f5b5002a680 comp rx=0 tx=0).stop 2024-05-20T22:08:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.521+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/1389063298 shutdown_connections 2024-05-20T22:08:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.521+0000 7f5b5e2b6640 1 --2- 172.21.15.55:0/1389063298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580d3290 0x7f5b580d36a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.521+0000 7f5b5e2b6640 1 --2- 172.21.15.55:0/1389063298 >> v2:172.21.15.138:3300/0 conn(0x7f5b580b4080 0x7f5b5815f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.521+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/1389063298 >> 172.21.15.55:0/1389063298 conn(0x7f5b580d51b0 msgr2=0x7f5b580b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.521+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/1389063298 shutdown_connections 2024-05-20T22:08:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.521+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/1389063298 wait complete. 2024-05-20T22:08:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.522+0000 7f5b5e2b6640 1 Processor -- start 2024-05-20T22:08:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.522+0000 7f5b5e2b6640 1 -- start start 2024-05-20T22:08:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b5e2b6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b580b35e0 0x7f5b581543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b5e2b6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580b4080 0x7f5b580b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b5e2b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b580d3290 0x7f5b581f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b5e2b6640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581635f0 con 0x7f5b580b4080 2024-05-20T22:08:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b5e2b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581604f0 con 0x7f5b580d3290 2024-05-20T22:08:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b5e2b6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5815ff30 con 0x7f5b580b35e0 2024-05-20T22:08:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b4ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580b4080 0x7f5b580b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b4ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580b4080 0x7f5b580b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60216/0 (socket says 172.21.15.55:60216) 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b4ffff640 1 -- 172.21.15.55:0/2247060611 learned_addr learned my addr 172.21.15.55:0/2247060611 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b57fff640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b580b35e0 0x7f5b581543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b4ffff640 1 -- 172.21.15.55:0/2247060611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b580b35e0 msgr2=0x7f5b581543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b4ffff640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b580b35e0 0x7f5b581543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b4ffff640 1 -- 172.21.15.55:0/2247060611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b580d3290 msgr2=0x7f5b581f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b5c853640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b580d3290 0x7f5b581f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b4ffff640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b580d3290 0x7f5b581f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b4ffff640 1 -- 172.21.15.55:0/2247060611 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b580c9c40 con 0x7f5b580b4080 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.524+0000 7f5b4ffff640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580b4080 0x7f5b580b67c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000c460 tx=0x7f5b4000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.525+0000 7f5b557fa640 1 -- 172.21.15.55:0/2247060611 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40018070 con 0x7f5b580b4080 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.525+0000 7f5b557fa640 1 -- 172.21.15.55:0/2247060611 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b40007500 con 0x7f5b580b4080 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.525+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/2247060611 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b580b4630 con 0x7f5b580b4080 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.525+0000 7f5b557fa640 1 -- 172.21.15.55:0/2247060611 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40004a20 con 0x7f5b580b4080 2024-05-20T22:08:50.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.525+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/2247060611 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b580b4bf0 con 0x7f5b580b4080 2024-05-20T22:08:50.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.526+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/2247060611 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b58069090 con 0x7f5b580b4080 2024-05-20T22:08:50.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.526+0000 7f5b557fa640 1 -- 172.21.15.55:0/2247060611 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b4001b070 con 0x7f5b580b4080 2024-05-20T22:08:50.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.527+0000 7f5b557fa640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5b3806be30 0x7f5b3806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:50.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.527+0000 7f5b557fa640 1 -- 172.21.15.55:0/2247060611 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5b4008a5b0 con 0x7f5b580b4080 2024-05-20T22:08:50.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.527+0000 7f5b57fff640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5b3806be30 0x7f5b3806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:50.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.527+0000 7f5b57fff640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5b3806be30 0x7f5b3806e340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b48006fd0 tx=0x7f5b48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:50.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.531+0000 7f5b557fa640 1 -- 172.21.15.55:0/2247060611 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5b5806b500 con 0x7f5b580b4080 2024-05-20T22:08:50.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.728+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/2247060611 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5b580ca320 con 0x7f5b580b4080 2024-05-20T22:08:50.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.728+0000 7f5b557fa640 1 -- 172.21.15.55:0/2247060611 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f5b40058ac0 con 0x7f5b580b4080 2024-05-20T22:08:50.729 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-20T22:08:50.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/2247060611 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5b3806be30 msgr2=0x7f5b3806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5b3806be30 0x7f5b3806e340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b48006fd0 tx=0x7f5b48008040 comp rx=0 tx=0).stop 2024-05-20T22:08:50.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/2247060611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580b4080 msgr2=0x7f5b580b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:50.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580b4080 0x7f5b580b67c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000c460 tx=0x7f5b4000c920 comp rx=0 tx=0).stop 2024-05-20T22:08:50.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/2247060611 shutdown_connections 2024-05-20T22:08:50.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5b3806be30 0x7f5b3806e340 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b580d3290 0x7f5b581f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b580b4080 0x7f5b580b67c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 --2- 172.21.15.55:0/2247060611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b580b35e0 0x7f5b581543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:50.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/2247060611 >> 172.21.15.55:0/2247060611 conn(0x7f5b580d51b0 msgr2=0x7f5b58157080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:50.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/2247060611 shutdown_connections 2024-05-20T22:08:50.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:50.730+0000 7f5b5e2b6640 1 -- 172.21.15.55:0/2247060611 wait complete. 2024-05-20T22:08:50.742 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-20T22:08:50.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:50.743 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-20T22:08:50.774 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:08:50.774 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:08:50.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:50.774 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-20T22:08:50.877 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:08:50.878 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:50.895 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:08:50.895 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:50.952 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:51.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.105+0000 7fa355aaf640 1 Processor -- start 2024-05-20T22:08:51.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.105+0000 7fa355aaf640 1 -- start start 2024-05-20T22:08:51.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.106+0000 7fa355aaf640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa35015f100 0x7fa3500b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.106+0000 7fa355aaf640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa35006c790 0x7fa3500cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.106+0000 7fa355aaf640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3500d3dd0 con 0x7fa3500b3b30 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.106+0000 7fa355aaf640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa3500d40b0 con 0x7fa35015f100 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.106+0000 7fa355aaf640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa3500d4390 con 0x7fa35006c790 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.106+0000 7fa347fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa35015f100 0x7fa3500b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.106+0000 7fa347fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa35015f100 0x7fa3500b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60226/0 (socket says 172.21.15.55:60226) 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.106+0000 7fa34f7fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa3500b3b30 0x7fa35006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53288/0 (socket says 172.21.15.55:53288) 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.106+0000 7fa347fff640 1 -- 172.21.15.55:0/2715715524 learned_addr learned my addr 172.21.15.55:0/2715715524 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.107+0000 7fa347fff640 1 -- 172.21.15.55:0/2715715524 >> v2:172.21.15.138:3300/0 conn(0x7fa35006c790 msgr2=0x7fa3500cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.107+0000 7fa34ffff640 1 --2- 172.21.15.55:0/2715715524 >> v2:172.21.15.138:3300/0 conn(0x7fa35006c790 0x7fa3500cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.107+0000 7fa347fff640 1 --2- 172.21.15.55:0/2715715524 >> v2:172.21.15.138:3300/0 conn(0x7fa35006c790 0x7fa3500cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.107+0000 7fa347fff640 1 -- 172.21.15.55:0/2715715524 >> v1:172.21.15.55:6789/0 conn(0x7fa3500b3b30 legacy=0x7fa35006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.107+0000 7fa347fff640 1 -- 172.21.15.55:0/2715715524 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35006b5a0 con 0x7fa35015f100 2024-05-20T22:08:51.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.107+0000 7fa34effd640 1 -- 172.21.15.55:0/2715715524 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3890402851 0 0) 0x7fa3500d3dd0 con 0x7fa3500b3b30 2024-05-20T22:08:51.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.108+0000 7fa347fff640 1 --2- 172.21.15.55:0/2715715524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 0x7fa3500b7fe0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa3500d12f0 tx=0x7fa33802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae74e5d0bf66dfe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:51.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.108+0000 7fa34effd640 1 -- 172.21.15.55:0/2715715524 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338037070 con 0x7fa35015f100 2024-05-20T22:08:51.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.108+0000 7fa34effd640 1 -- 172.21.15.55:0/2715715524 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa338032440 con 0x7fa35015f100 2024-05-20T22:08:51.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.108+0000 7fa34effd640 1 -- 172.21.15.55:0/2715715524 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338032810 con 0x7fa35015f100 2024-05-20T22:08:51.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.108+0000 7fa355aaf640 1 -- 172.21.15.55:0/2715715524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 msgr2=0x7fa3500b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.108+0000 7fa355aaf640 1 --2- 172.21.15.55:0/2715715524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 0x7fa3500b7fe0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa3500d12f0 tx=0x7fa33802a270 comp rx=0 tx=0).stop 2024-05-20T22:08:51.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.109+0000 7fa355aaf640 1 -- 172.21.15.55:0/2715715524 shutdown_connections 2024-05-20T22:08:51.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.109+0000 7fa355aaf640 1 --2- 172.21.15.55:0/2715715524 >> v2:172.21.15.138:3300/0 conn(0x7fa35006c790 0x7fa3500cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.109+0000 7fa355aaf640 1 --2- 172.21.15.55:0/2715715524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 0x7fa3500b7fe0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.109+0000 7fa355aaf640 1 -- 172.21.15.55:0/2715715524 >> 172.21.15.55:0/2715715524 conn(0x7fa3500d51d0 msgr2=0x7fa3500b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:51.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.109+0000 7fa355aaf640 1 -- 172.21.15.55:0/2715715524 shutdown_connections 2024-05-20T22:08:51.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.109+0000 7fa355aaf640 1 -- 172.21.15.55:0/2715715524 wait complete. 2024-05-20T22:08:51.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.110+0000 7fa355aaf640 1 Processor -- start 2024-05-20T22:08:51.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.110+0000 7fa355aaf640 1 -- start start 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa355aaf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa35006c790 0x7fa3501f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa355aaf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3500b3b30 0x7fa35015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa355aaf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 0x7fa3501531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa355aaf640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa35006cba0 con 0x7fa35015f100 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa355aaf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa35006bb60 con 0x7fa35006c790 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa355aaf640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa35006cd20 con 0x7fa3500b3b30 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa34ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 0x7fa3501531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa34ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 0x7fa3501531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60232/0 (socket says 172.21.15.55:60232) 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa34ffff640 1 -- 172.21.15.55:0/2850382925 learned_addr learned my addr 172.21.15.55:0/2850382925 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa34f7fe640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa35006c790 0x7fa3501f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa347fff640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3500b3b30 0x7fa35015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa34ffff640 1 -- 172.21.15.55:0/2850382925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3500b3b30 msgr2=0x7fa35015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa34ffff640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3500b3b30 0x7fa35015a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa34ffff640 1 -- 172.21.15.55:0/2850382925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa35006c790 msgr2=0x7fa3501f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa34ffff640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa35006c790 0x7fa3501f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa34ffff640 1 -- 172.21.15.55:0/2850382925 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3501536e0 con 0x7fa35015f100 2024-05-20T22:08:51.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.112+0000 7fa34ffff640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 0x7fa3501531a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa34800cc60 tx=0x7fa34800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:51.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.113+0000 7fa34cff9640 1 -- 172.21.15.55:0/2850382925 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348003990 con 0x7fa35015f100 2024-05-20T22:08:51.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.113+0000 7fa34cff9640 1 -- 172.21.15.55:0/2850382925 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348003b30 con 0x7fa35015f100 2024-05-20T22:08:51.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.113+0000 7fa355aaf640 1 -- 172.21.15.55:0/2850382925 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3501f96a0 con 0x7fa35015f100 2024-05-20T22:08:51.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.113+0000 7fa34cff9640 1 -- 172.21.15.55:0/2850382925 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348004020 con 0x7fa35015f100 2024-05-20T22:08:51.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.113+0000 7fa355aaf640 1 -- 172.21.15.55:0/2850382925 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3501f9c60 con 0x7fa35015f100 2024-05-20T22:08:51.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.114+0000 7fa355aaf640 1 -- 172.21.15.55:0/2850382925 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3500ba8a0 con 0x7fa35015f100 2024-05-20T22:08:51.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.114+0000 7fa34cff9640 1 -- 172.21.15.55:0/2850382925 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3480041c0 con 0x7fa35015f100 2024-05-20T22:08:51.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.115+0000 7fa34cff9640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa33006be30 0x7fa33006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.115+0000 7fa34f7fe640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa33006be30 0x7fa33006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.115+0000 7fa34cff9640 1 -- 172.21.15.55:0/2850382925 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa34808b820 con 0x7fa35015f100 2024-05-20T22:08:51.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.115+0000 7fa34f7fe640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa33006be30 0x7fa33006e340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa340000f60 tx=0x7fa340007c00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:51.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.120+0000 7fa34cff9640 1 -- 172.21.15.55:0/2850382925 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa3500ba8a0 con 0x7fa35015f100 2024-05-20T22:08:51.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.303+0000 7fa355aaf640 1 -- 172.21.15.55:0/2850382925 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa350069090 con 0x7fa35015f100 2024-05-20T22:08:51.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.304+0000 7fa34cff9640 1 -- 172.21.15.55:0/2850382925 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7fa348014030 con 0x7fa35015f100 2024-05-20T22:08:51.306 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:51.306 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.306+0000 7fa355aaf640 1 -- 172.21.15.55:0/2850382925 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa33006be30 msgr2=0x7fa33006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.306+0000 7fa355aaf640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa33006be30 0x7fa33006e340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa340000f60 tx=0x7fa340007c00 comp rx=0 tx=0).stop 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.306+0000 7fa355aaf640 1 -- 172.21.15.55:0/2850382925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 msgr2=0x7fa3501531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.306+0000 7fa355aaf640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 0x7fa3501531a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa34800cc60 tx=0x7fa34800a5c0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.306+0000 7fa355aaf640 1 -- 172.21.15.55:0/2850382925 shutdown_connections 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.307+0000 7fa355aaf640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa33006be30 0x7fa33006e340 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.307+0000 7fa355aaf640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa35015f100 0x7fa3501531a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.307+0000 7fa355aaf640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3500b3b30 0x7fa35015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.307+0000 7fa355aaf640 1 --2- 172.21.15.55:0/2850382925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa35006c790 0x7fa3501f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.307+0000 7fa355aaf640 1 -- 172.21.15.55:0/2850382925 >> 172.21.15.55:0/2850382925 conn(0x7fa3500d51d0 msgr2=0x7fa35015e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.307+0000 7fa355aaf640 1 -- 172.21.15.55:0/2850382925 shutdown_connections 2024-05-20T22:08:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.307+0000 7fa355aaf640 1 -- 172.21.15.55:0/2850382925 wait complete. 2024-05-20T22:08:51.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:51.320 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:51.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.437+0000 7fbf55406640 1 Processor -- start 2024-05-20T22:08:51.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.437+0000 7fbf55406640 1 -- start start 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.438+0000 7fbf55406640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf500b2920 0x7fbf500b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.438+0000 7fbf55406640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf500ce2d0 con 0x7fbf5006be00 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.438+0000 7fbf55406640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf500ce5b0 con 0x7fbf500d1790 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.438+0000 7fbf55406640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbf500ceee0 con 0x7fbf500b2920 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.438+0000 7fbf4f7fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbf5006be00 0x7fbf5006c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53302/0 (socket says 172.21.15.55:53302) 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.438+0000 7fbf4f7fe640 1 -- 172.21.15.55:0/2577699408 learned_addr learned my addr 172.21.15.55:0/2577699408 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.438+0000 7fbf4effd640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf500b2920 0x7fbf500b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 750240369 0 0) 0x7fbf500ce2d0 con 0x7fbf5006be00 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf3c003520 con 0x7fbf5006be00 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 474474019 0 0) 0x7fbf500ce5b0 con 0x7fbf500d1790 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf500ce2d0 con 0x7fbf500d1790 2024-05-20T22:08:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2495792918 0 0) 0x7fbf3c003520 con 0x7fbf5006be00 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf500b2920 msgr2=0x7fbf500b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf500b2920 0x7fbf500b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 >> v1:172.21.15.138:6789/0 conn(0x7fbf500d1790 legacy=0x7fbf500d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf500ce890 con 0x7fbf5006be00 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fbf480019f0 con 0x7fbf5006be00 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 >> v1:172.21.15.55:6789/0 conn(0x7fbf5006be00 legacy=0x7fbf5006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf3c003b70 0x7fbf3c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf3c004800 0x7fbf3c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf500b2920 0x7fbf3c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbf500ceee0 con 0x7fbf3c004800 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c00bb80 con 0x7fbf3c003b70 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c00bd80 con 0x7fbf500b2920 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf467fc640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf3c004800 0x7fbf3c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4effd640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf3c003b70 0x7fbf3c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf4f7fe640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf500b2920 0x7fbf3c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf467fc640 1 -- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf500b2920 msgr2=0x7fbf3c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:51.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf467fc640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf500b2920 0x7fbf3c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf467fc640 1 -- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf3c003b70 msgr2=0x7fbf3c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:51.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf467fc640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf3c003b70 0x7fbf3c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.439+0000 7fbf467fc640 1 -- 172.21.15.55:0/2577699408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c003520 con 0x7fbf3c004800 2024-05-20T22:08:51.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.440+0000 7fbf467fc640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf3c004800 0x7fbf3c004c10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbf380018a0 tx=0x7fbf3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebb5804e034c0de server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:51.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.440+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf380014f0 con 0x7fbf3c004800 2024-05-20T22:08:51.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.440+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf380052d0 con 0x7fbf3c004800 2024-05-20T22:08:51.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.440+0000 7fbf4e7fc640 1 -- 172.21.15.55:0/2577699408 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3802ac60 con 0x7fbf3c004800 2024-05-20T22:08:51.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.441+0000 7fbf55406640 1 -- 172.21.15.55:0/2577699408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf3c004800 msgr2=0x7fbf3c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.441+0000 7fbf55406640 1 --2- 172.21.15.55:0/2577699408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf3c004800 0x7fbf3c004c10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbf380018a0 tx=0x7fbf3c005830 comp rx=0 tx=0).stop 2024-05-20T22:08:51.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.441+0000 7fbf4f7fe640 1 -- 172.21.15.55:0/2577699408 reap_dead start 2024-05-20T22:08:51.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.441+0000 7fbf55406640 1 -- 172.21.15.55:0/2577699408 shutdown_connections 2024-05-20T22:08:51.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.441+0000 7fbf55406640 1 -- 172.21.15.55:0/2577699408 >> 172.21.15.55:0/2577699408 conn(0x7fbf500737c0 msgr2=0x7fbf500b6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:51.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.441+0000 7fbf55406640 1 -- 172.21.15.55:0/2577699408 shutdown_connections 2024-05-20T22:08:51.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.441+0000 7fbf55406640 1 -- 172.21.15.55:0/2577699408 wait complete. 2024-05-20T22:08:51.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.443+0000 7fbf55406640 1 Processor -- start 2024-05-20T22:08:51.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.443+0000 7fbf55406640 1 -- start start 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.444+0000 7fbf55406640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf3c004800 0x7fbf50150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.444+0000 7fbf55406640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf501f68c0 0x7fbf50153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.444+0000 7fbf55406640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf501507a0 0x7fbf501f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.444+0000 7fbf55406640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbf500ce050 con 0x7fbf501f68c0 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.444+0000 7fbf55406640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf500cded0 con 0x7fbf3c004800 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.444+0000 7fbf55406640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5015d100 con 0x7fbf501507a0 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.444+0000 7fbf467fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf501f68c0 0x7fbf50153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf467fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf501f68c0 0x7fbf50153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60256/0 (socket says 172.21.15.55:60256) 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf467fc640 1 -- 172.21.15.55:0/842088186 learned_addr learned my addr 172.21.15.55:0/842088186 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf4f7fe640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf501507a0 0x7fbf501f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf4effd640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf3c004800 0x7fbf50150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf467fc640 1 -- 172.21.15.55:0/842088186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf501507a0 msgr2=0x7fbf501f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf467fc640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf501507a0 0x7fbf501f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf467fc640 1 -- 172.21.15.55:0/842088186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf3c004800 msgr2=0x7fbf50150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf467fc640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf3c004800 0x7fbf50150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf467fc640 1 -- 172.21.15.55:0/842088186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf501f32d0 con 0x7fbf501f68c0 2024-05-20T22:08:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf467fc640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf501f68c0 0x7fbf50153050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005040 tx=0x7fbf380012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf47fff640 1 -- 172.21.15.55:0/842088186 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3803f020 con 0x7fbf501f68c0 2024-05-20T22:08:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf47fff640 1 -- 172.21.15.55:0/842088186 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf38006e50 con 0x7fbf501f68c0 2024-05-20T22:08:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf55406640 1 -- 172.21.15.55:0/842088186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf501f70a0 con 0x7fbf501f68c0 2024-05-20T22:08:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf47fff640 1 -- 172.21.15.55:0/842088186 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38037040 con 0x7fbf501f68c0 2024-05-20T22:08:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.445+0000 7fbf55406640 1 -- 172.21.15.55:0/842088186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf501f7660 con 0x7fbf501f68c0 2024-05-20T22:08:51.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.446+0000 7fbf55406640 1 -- 172.21.15.55:0/842088186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf501f6df0 con 0x7fbf501f68c0 2024-05-20T22:08:51.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.447+0000 7fbf47fff640 1 -- 172.21.15.55:0/842088186 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf38042020 con 0x7fbf501f68c0 2024-05-20T22:08:51.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.448+0000 7fbf47fff640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbf3006be60 0x7fbf3006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.448+0000 7fbf47fff640 1 -- 172.21.15.55:0/842088186 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbf380aaff0 con 0x7fbf501f68c0 2024-05-20T22:08:51.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.448+0000 7fbf4effd640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbf3006be60 0x7fbf3006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.448+0000 7fbf4effd640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbf3006be60 0x7fbf3006e370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009160 tx=0x7fbf40008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:51.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.452+0000 7fbf47fff640 1 -- 172.21.15.55:0/842088186 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbf38049030 con 0x7fbf501f68c0 2024-05-20T22:08:51.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.634+0000 7fbf55406640 1 -- 172.21.15.55:0/842088186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf50069090 con 0x7fbf501f68c0 2024-05-20T22:08:51.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.634+0000 7fbf47fff640 1 -- 172.21.15.55:0/842088186 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fbf501f32d0 con 0x7fbf501f68c0 2024-05-20T22:08:51.635 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:51.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 -- 172.21.15.55:0/842088186 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbf3006be60 msgr2=0x7fbf3006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbf3006be60 0x7fbf3006e370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009160 tx=0x7fbf40008040 comp rx=0 tx=0).stop 2024-05-20T22:08:51.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 -- 172.21.15.55:0/842088186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf501f68c0 msgr2=0x7fbf50153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf501f68c0 0x7fbf50153050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005040 tx=0x7fbf380012d0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 -- 172.21.15.55:0/842088186 shutdown_connections 2024-05-20T22:08:51.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbf3006be60 0x7fbf3006e370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf501507a0 0x7fbf501f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf501f68c0 0x7fbf50153050 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 --2- 172.21.15.55:0/842088186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf3c004800 0x7fbf50150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 -- 172.21.15.55:0/842088186 >> 172.21.15.55:0/842088186 conn(0x7fbf500737c0 msgr2=0x7fbf500cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:51.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 -- 172.21.15.55:0/842088186 shutdown_connections 2024-05-20T22:08:51.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.637+0000 7fbf55406640 1 -- 172.21.15.55:0/842088186 wait complete. 2024-05-20T22:08:51.649 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.767+0000 7f55ee861640 1 Processor -- start 2024-05-20T22:08:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.767+0000 7f55ee861640 1 -- start start 2024-05-20T22:08:51.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.768+0000 7f55ee861640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f55e80b1ef0 0x7f55e80d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.768+0000 7f55ee861640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e80b40d0 con 0x7f55e80d1540 2024-05-20T22:08:51.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.768+0000 7f55ee861640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e80b5b10 con 0x7f55e80b7d90 2024-05-20T22:08:51.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.768+0000 7f55ee861640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f55e80b6160 con 0x7f55e80b1ef0 2024-05-20T22:08:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.768+0000 7f55e77fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f55e80b1ef0 0x7f55e80d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.768+0000 7f55ecdfe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f55e80d1540 0x7f55e80b65f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53306/0 (socket says 172.21.15.55:53306) 2024-05-20T22:08:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.768+0000 7f55ecdfe640 1 -- 172.21.15.55:0/1875967188 learned_addr learned my addr 172.21.15.55:0/1875967188 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.769+0000 7f55e77fe640 1 -- 172.21.15.55:0/1875967188 >> v1:172.21.15.138:6789/0 conn(0x7f55e80b7d90 legacy=0x7f55e80b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.769+0000 7f55e77fe640 1 -- 172.21.15.55:0/1875967188 >> v1:172.21.15.55:6789/0 conn(0x7f55e80d1540 legacy=0x7f55e80b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.769+0000 7f55e6ffd640 1 -- 172.21.15.55:0/1875967188 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4249321999 0 0) 0x7f55e80b40d0 con 0x7f55e80d1540 2024-05-20T22:08:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.769+0000 7f55e77fe640 1 -- 172.21.15.55:0/1875967188 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e80b5df0 con 0x7f55e80b1ef0 2024-05-20T22:08:51.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.769+0000 7f55e77fe640 1 --2- 172.21.15.55:0/1875967188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80b1ef0 0x7f55e80d1000 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f55d00097a0 tx=0x7f55d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f952ecc401248e89 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:51.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.770+0000 7f55e6ffd640 1 -- 172.21.15.55:0/1875967188 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0037070 con 0x7f55e80b1ef0 2024-05-20T22:08:51.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.770+0000 7f55e6ffd640 1 -- 172.21.15.55:0/1875967188 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d0032440 con 0x7f55e80b1ef0 2024-05-20T22:08:51.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.770+0000 7f55e6ffd640 1 -- 172.21.15.55:0/1875967188 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0032810 con 0x7f55e80b1ef0 2024-05-20T22:08:51.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.770+0000 7f55ee861640 1 -- 172.21.15.55:0/1875967188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80b1ef0 msgr2=0x7f55e80d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.770+0000 7f55ee861640 1 --2- 172.21.15.55:0/1875967188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80b1ef0 0x7f55e80d1000 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f55d00097a0 tx=0x7f55d002a1e0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.770+0000 7f55ee861640 1 -- 172.21.15.55:0/1875967188 shutdown_connections 2024-05-20T22:08:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.770+0000 7f55ee861640 1 --2- 172.21.15.55:0/1875967188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80b1ef0 0x7f55e80d1000 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.770+0000 7f55ee861640 1 -- 172.21.15.55:0/1875967188 >> 172.21.15.55:0/1875967188 conn(0x7f55e809be10 msgr2=0x7f55e80d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.770+0000 7f55ee861640 1 -- 172.21.15.55:0/1875967188 shutdown_connections 2024-05-20T22:08:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.770+0000 7f55ee861640 1 -- 172.21.15.55:0/1875967188 wait complete. 2024-05-20T22:08:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.772+0000 7f55ee861640 1 Processor -- start 2024-05-20T22:08:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.772+0000 7f55ee861640 1 -- start start 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ee861640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e80b1ef0 0x7f55e8152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ee861640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e80b7d90 0x7f55e8152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ee861640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80d1540 0x7f55e809db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ee861640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f55e80b6e80 con 0x7f55e80d1540 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ee861640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55e80b6d00 con 0x7f55e80b7d90 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ee861640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f55e80b7000 con 0x7f55e80b1ef0 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ecdfe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80d1540 0x7f55e809db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ecdfe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80d1540 0x7f55e809db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60264/0 (socket says 172.21.15.55:60264) 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ecdfe640 1 -- 172.21.15.55:0/345961802 learned_addr learned my addr 172.21.15.55:0/345961802 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55e7fff640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e80b1ef0 0x7f55e8152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55e77fe640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e80b7d90 0x7f55e8152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ecdfe640 1 -- 172.21.15.55:0/345961802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e80b1ef0 msgr2=0x7f55e8152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ecdfe640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e80b1ef0 0x7f55e8152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ecdfe640 1 -- 172.21.15.55:0/345961802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e80b7d90 msgr2=0x7f55e8152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ecdfe640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e80b7d90 0x7f55e8152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ecdfe640 1 -- 172.21.15.55:0/345961802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e809cad0 con 0x7f55e80d1540 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55e7fff640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e80b1ef0 0x7f55e8152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55e77fe640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e80b7d90 0x7f55e8152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ecdfe640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80d1540 0x7f55e809db70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f55e000c700 tx=0x7f55e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55e4ff9640 1 -- 172.21.15.55:0/345961802 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0018070 con 0x7f55e80d1540 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55e4ff9640 1 -- 172.21.15.55:0/345961802 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e000ce10 con 0x7f55e80d1540 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ee861640 1 -- 172.21.15.55:0/345961802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e809cdb0 con 0x7f55e80d1540 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55e4ff9640 1 -- 172.21.15.55:0/345961802 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e000a860 con 0x7f55e80d1540 2024-05-20T22:08:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.774+0000 7f55ee861640 1 -- 172.21.15.55:0/345961802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e809d370 con 0x7f55e80d1540 2024-05-20T22:08:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.776+0000 7f55ee861640 1 -- 172.21.15.55:0/345961802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e8069090 con 0x7f55e80d1540 2024-05-20T22:08:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.776+0000 7f55e4ff9640 1 -- 172.21.15.55:0/345961802 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f55e001b070 con 0x7f55e80d1540 2024-05-20T22:08:51.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.777+0000 7f55e4ff9640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c806be30 0x7f55c806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:51.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.777+0000 7f55e4ff9640 1 -- 172.21.15.55:0/345961802 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f55e008a910 con 0x7f55e80d1540 2024-05-20T22:08:51.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.777+0000 7f55e7fff640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c806be30 0x7f55c806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:51.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.777+0000 7f55e7fff640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c806be30 0x7f55c806e340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f55d8006fd0 tx=0x7f55d8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:51.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.781+0000 7f55e4ff9640 1 -- 172.21.15.55:0/345961802 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f55e0058d90 con 0x7f55e80d1540 2024-05-20T22:08:51.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.964+0000 7f55ee861640 1 -- 172.21.15.55:0/345961802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55e8071a10 con 0x7f55e80d1540 2024-05-20T22:08:51.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.965+0000 7f55e4ff9640 1 -- 172.21.15.55:0/345961802 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f55e0058690 con 0x7f55e80d1540 2024-05-20T22:08:51.966 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 -- 172.21.15.55:0/345961802 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c806be30 msgr2=0x7f55c806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c806be30 0x7f55c806e340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f55d8006fd0 tx=0x7f55d8008040 comp rx=0 tx=0).stop 2024-05-20T22:08:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 -- 172.21.15.55:0/345961802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80d1540 msgr2=0x7f55e809db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80d1540 0x7f55e809db70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f55e000c700 tx=0x7f55e000cbc0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 -- 172.21.15.55:0/345961802 shutdown_connections 2024-05-20T22:08:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c806be30 0x7f55c806e340 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e80d1540 0x7f55e809db70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e80b7d90 0x7f55e8152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 --2- 172.21.15.55:0/345961802 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e80b1ef0 0x7f55e8152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 -- 172.21.15.55:0/345961802 >> 172.21.15.55:0/345961802 conn(0x7f55e809be10 msgr2=0x7f55e8070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:51.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 -- 172.21.15.55:0/345961802 shutdown_connections 2024-05-20T22:08:51.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:51.967+0000 7f55ee861640 1 -- 172.21.15.55:0/345961802 wait complete. 2024-05-20T22:08:51.979 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:52.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.096+0000 7fedac1aa640 1 Processor -- start 2024-05-20T22:08:52.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.097+0000 7fedac1aa640 1 -- start start 2024-05-20T22:08:52.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7fedac1aa640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feda40d32a0 0x7feda415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7fedac1aa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feda40cf690 0x7feda40cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7fedac1aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feda40cffe0 0x7feda4162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7fedac1aa640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7feda40d3780 con 0x7feda40d32a0 2024-05-20T22:08:52.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7fedac1aa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feda40d3a60 con 0x7feda40cf690 2024-05-20T22:08:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7fedac1aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feda40d3d40 con 0x7feda40cffe0 2024-05-20T22:08:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7feda9f46640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feda40d32a0 0x7feda415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7feda9f46640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feda40d32a0 0x7feda415f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60270/0 (socket says 172.21.15.55:60270) 2024-05-20T22:08:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7feda9f46640 1 -- 172.21.15.55:0/2958235597 learned_addr learned my addr 172.21.15.55:0/2958235597 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7feda9f46640 1 -- 172.21.15.55:0/2958235597 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feda40cffe0 msgr2=0x7feda4162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7feda9f46640 1 --2- 172.21.15.55:0/2958235597 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feda40cffe0 0x7feda4162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7feda9f46640 1 -- 172.21.15.55:0/2958235597 >> v2:172.21.15.138:3300/0 conn(0x7feda40cf690 msgr2=0x7feda40cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7feda9f46640 1 --2- 172.21.15.55:0/2958235597 >> v2:172.21.15.138:3300/0 conn(0x7feda40cf690 0x7feda40cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.098+0000 7feda9f46640 1 -- 172.21.15.55:0/2958235597 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda40d4020 con 0x7feda40d32a0 2024-05-20T22:08:52.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.099+0000 7feda9f46640 1 --2- 172.21.15.55:0/2958235597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40d32a0 0x7feda415f150 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feda408e150 tx=0x7fed9402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce4fc74c763a7c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:52.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.099+0000 7feda8f44640 1 -- 172.21.15.55:0/2958235597 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94036070 con 0x7feda40d32a0 2024-05-20T22:08:52.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.099+0000 7feda8f44640 1 -- 172.21.15.55:0/2958235597 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed94031600 con 0x7feda40d32a0 2024-05-20T22:08:52.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.099+0000 7feda8f44640 1 -- 172.21.15.55:0/2958235597 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed940319d0 con 0x7feda40d32a0 2024-05-20T22:08:52.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.100+0000 7fedac1aa640 1 -- 172.21.15.55:0/2958235597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40d32a0 msgr2=0x7feda415f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.100+0000 7fedac1aa640 1 --2- 172.21.15.55:0/2958235597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40d32a0 0x7feda415f150 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feda408e150 tx=0x7fed9402a6c0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.100+0000 7fedac1aa640 1 -- 172.21.15.55:0/2958235597 shutdown_connections 2024-05-20T22:08:52.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.100+0000 7fedac1aa640 1 --2- 172.21.15.55:0/2958235597 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feda40cffe0 0x7feda4162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.100+0000 7fedac1aa640 1 --2- 172.21.15.55:0/2958235597 >> v2:172.21.15.138:3300/0 conn(0x7feda40cf690 0x7feda40cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.100+0000 7fedac1aa640 1 --2- 172.21.15.55:0/2958235597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40d32a0 0x7feda415f150 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.100+0000 7fedac1aa640 1 -- 172.21.15.55:0/2958235597 >> 172.21.15.55:0/2958235597 conn(0x7feda40b6de0 msgr2=0x7feda40d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:52.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.100+0000 7fedac1aa640 1 -- 172.21.15.55:0/2958235597 shutdown_connections 2024-05-20T22:08:52.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.100+0000 7fedac1aa640 1 -- 172.21.15.55:0/2958235597 wait complete. 2024-05-20T22:08:52.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.102+0000 7fedac1aa640 1 Processor -- start 2024-05-20T22:08:52.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.102+0000 7fedac1aa640 1 -- start start 2024-05-20T22:08:52.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.103+0000 7fedac1aa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40cf690 0x7feda40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.103+0000 7fedac1aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feda40cffe0 0x7feda41f9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.103+0000 7fedac1aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feda40d32a0 0x7feda41f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.103+0000 7fedac1aa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7feda40d4820 con 0x7feda40cf690 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.103+0000 7fedac1aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feda40d03f0 con 0x7feda40d32a0 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.103+0000 7fedac1aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feda4160440 con 0x7feda40cffe0 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.103+0000 7feda9f46640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40cf690 0x7feda40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.103+0000 7feda9f46640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40cf690 0x7feda40ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60280/0 (socket says 172.21.15.55:60280) 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.103+0000 7feda9f46640 1 -- 172.21.15.55:0/3911699364 learned_addr learned my addr 172.21.15.55:0/3911699364 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda9745640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feda40cffe0 0x7feda41f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7fedaa747640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feda40d32a0 0x7feda41f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda9f46640 1 -- 172.21.15.55:0/3911699364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feda40cffe0 msgr2=0x7feda41f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda9f46640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feda40cffe0 0x7feda41f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda9f46640 1 -- 172.21.15.55:0/3911699364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feda40d32a0 msgr2=0x7feda41f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda9f46640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feda40d32a0 0x7feda41f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda9f46640 1 -- 172.21.15.55:0/3911699364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda40cac40 con 0x7feda40cf690 2024-05-20T22:08:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda9745640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feda40cffe0 0x7feda41f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:52.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7fedaa747640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feda40d32a0 0x7feda41f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:52.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda9f46640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40cf690 0x7feda40ce740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fed94000f80 tx=0x7fed94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:52.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda27fc640 1 -- 172.21.15.55:0/3911699364 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94041020 con 0x7feda40cf690 2024-05-20T22:08:52.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda27fc640 1 -- 172.21.15.55:0/3911699364 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed94007820 con 0x7feda40cf690 2024-05-20T22:08:52.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7fedac1aa640 1 -- 172.21.15.55:0/3911699364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda40caf20 con 0x7feda40cf690 2024-05-20T22:08:52.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7feda27fc640 1 -- 172.21.15.55:0/3911699364 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94036040 con 0x7feda40cf690 2024-05-20T22:08:52.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.104+0000 7fedac1aa640 1 -- 172.21.15.55:0/3911699364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda41f8f80 con 0x7feda40cf690 2024-05-20T22:08:52.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.105+0000 7fedac1aa640 1 -- 172.21.15.55:0/3911699364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda4069090 con 0x7feda40cf690 2024-05-20T22:08:52.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.106+0000 7feda27fc640 1 -- 172.21.15.55:0/3911699364 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed94044020 con 0x7feda40cf690 2024-05-20T22:08:52.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.106+0000 7feda27fc640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed9006be60 0x7fed9006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.106+0000 7feda27fc640 1 -- 172.21.15.55:0/3911699364 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fed940ac0d0 con 0x7feda40cf690 2024-05-20T22:08:52.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.107+0000 7feda9745640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed9006be60 0x7fed9006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.107+0000 7feda9745640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed9006be60 0x7fed9006e370 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feda41628d0 tx=0x7fed98008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.111+0000 7feda27fc640 1 -- 172.21.15.55:0/3911699364 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fed9404b030 con 0x7feda40cf690 2024-05-20T22:08:52.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.295+0000 7fedac1aa640 1 -- 172.21.15.55:0/3911699364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feda40d2e50 con 0x7feda40cf690 2024-05-20T22:08:52.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.296+0000 7feda27fc640 1 -- 172.21.15.55:0/3911699364 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fed9407a5f0 con 0x7feda40cf690 2024-05-20T22:08:52.297 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 -- 172.21.15.55:0/3911699364 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed9006be60 msgr2=0x7fed9006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed9006be60 0x7fed9006e370 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feda41628d0 tx=0x7fed98008040 comp rx=0 tx=0).stop 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 -- 172.21.15.55:0/3911699364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40cf690 msgr2=0x7feda40ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40cf690 0x7feda40ce740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fed94000f80 tx=0x7fed94004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 -- 172.21.15.55:0/3911699364 shutdown_connections 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fed9006be60 0x7fed9006e370 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feda40d32a0 0x7feda41f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feda40cffe0 0x7feda41f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 --2- 172.21.15.55:0/3911699364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda40cf690 0x7feda40ce740 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 -- 172.21.15.55:0/3911699364 >> 172.21.15.55:0/3911699364 conn(0x7feda40b6de0 msgr2=0x7feda40ba100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:52.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 -- 172.21.15.55:0/3911699364 shutdown_connections 2024-05-20T22:08:52.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.298+0000 7fedac1aa640 1 -- 172.21.15.55:0/3911699364 wait complete. 2024-05-20T22:08:52.310 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:52.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.426+0000 7f623b36b640 1 Processor -- start 2024-05-20T22:08:52.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.427+0000 7f623b36b640 1 -- start start 2024-05-20T22:08:52.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.428+0000 7f623b36b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f623415f150 0x7f62340cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.428+0000 7f623b36b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62340d3780 con 0x7f623406b500 2024-05-20T22:08:52.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.428+0000 7f623b36b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62340d3a60 con 0x7f62340d32a0 2024-05-20T22:08:52.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.428+0000 7f623b36b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f62340d3d40 con 0x7f623415f150 2024-05-20T22:08:52.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.428+0000 7f6238906640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f623415f150 0x7f62340cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.428+0000 7f6238906640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f623415f150 0x7f62340cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60290/0 (socket says 172.21.15.55:60290) 2024-05-20T22:08:52.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.428+0000 7f6238906640 1 -- 172.21.15.55:0/115187535 learned_addr learned my addr 172.21.15.55:0/115187535 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:52.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.428+0000 7f6239908640 1 --1- 172.21.15.55:0/115187535 >> v1:172.21.15.55:6789/0 conn(0x7f623406b500 0x7f623406b910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53318/0 (socket says 172.21.15.55:53318) 2024-05-20T22:08:52.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.429+0000 7f6238906640 1 -- 172.21.15.55:0/115187535 >> v1:172.21.15.138:6789/0 conn(0x7f62340d32a0 legacy=0x7f62340cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:52.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.429+0000 7f6238906640 1 -- 172.21.15.55:0/115187535 >> v1:172.21.15.55:6789/0 conn(0x7f623406b500 legacy=0x7f623406b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.429+0000 7f622bfff640 1 -- 172.21.15.55:0/115187535 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 725570067 0 0) 0x7f62340d3780 con 0x7f623406b500 2024-05-20T22:08:52.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.429+0000 7f6238906640 1 -- 172.21.15.55:0/115187535 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62340d4020 con 0x7f623415f150 2024-05-20T22:08:52.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.429+0000 7f6238906640 1 --2- 172.21.15.55:0/115187535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623415f150 0x7f62340cfda0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6224009870 tx=0x7f622402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3545633c67d75eee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.430+0000 7f622bfff640 1 -- 172.21.15.55:0/115187535 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224037070 con 0x7f623415f150 2024-05-20T22:08:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.430+0000 7f622bfff640 1 -- 172.21.15.55:0/115187535 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6224032440 con 0x7f623415f150 2024-05-20T22:08:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.430+0000 7f622bfff640 1 -- 172.21.15.55:0/115187535 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224032810 con 0x7f623415f150 2024-05-20T22:08:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.430+0000 7f623b36b640 1 -- 172.21.15.55:0/115187535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623415f150 msgr2=0x7f62340cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.430+0000 7f623b36b640 1 --2- 172.21.15.55:0/115187535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623415f150 0x7f62340cfda0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6224009870 tx=0x7f622402a2a0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.430+0000 7f623b36b640 1 -- 172.21.15.55:0/115187535 shutdown_connections 2024-05-20T22:08:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.430+0000 7f623b36b640 1 --2- 172.21.15.55:0/115187535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623415f150 0x7f62340cfda0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.430+0000 7f623b36b640 1 -- 172.21.15.55:0/115187535 >> 172.21.15.55:0/115187535 conn(0x7f62340b6de0 msgr2=0x7f62340d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.430+0000 7f623b36b640 1 -- 172.21.15.55:0/115187535 shutdown_connections 2024-05-20T22:08:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.430+0000 7f623b36b640 1 -- 172.21.15.55:0/115187535 wait complete. 2024-05-20T22:08:52.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.432+0000 7f623b36b640 1 Processor -- start 2024-05-20T22:08:52.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.432+0000 7f623b36b640 1 -- start start 2024-05-20T22:08:52.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f623b36b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623406b500 0x7f62340c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f623b36b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62340d32a0 0x7f6234152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f623b36b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623415f150 0x7f62340b9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f623b36b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f62340d4820 con 0x7f623406b500 2024-05-20T22:08:52.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f623b36b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f623415f560 con 0x7f623415f150 2024-05-20T22:08:52.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f623b36b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6234160de0 con 0x7f62340d32a0 2024-05-20T22:08:52.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6239107640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623406b500 0x7f62340c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6239107640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623406b500 0x7f62340c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60306/0 (socket says 172.21.15.55:60306) 2024-05-20T22:08:52.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6239107640 1 -- 172.21.15.55:0/2019651346 learned_addr learned my addr 172.21.15.55:0/2019651346 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6238906640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62340d32a0 0x7f6234152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6239908640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623415f150 0x7f62340b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6239107640 1 -- 172.21.15.55:0/2019651346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62340d32a0 msgr2=0x7f6234152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6239107640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62340d32a0 0x7f6234152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6239107640 1 -- 172.21.15.55:0/2019651346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623415f150 msgr2=0x7f62340b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6239107640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623415f150 0x7f62340b9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6239107640 1 -- 172.21.15.55:0/2019651346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62340caa90 con 0x7f623406b500 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6239107640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623406b500 0x7f62340c9370 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f623000b420 tx=0x7f623000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6229ffb640 1 -- 172.21.15.55:0/2019651346 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230011830 con 0x7f623406b500 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6229ffb640 1 -- 172.21.15.55:0/2019651346 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6230016070 con 0x7f623406b500 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f6229ffb640 1 -- 172.21.15.55:0/2019651346 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230011c10 con 0x7f623406b500 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.434+0000 7f623b36b640 1 -- 172.21.15.55:0/2019651346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62340cad70 con 0x7f623406b500 2024-05-20T22:08:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.435+0000 7f623b36b640 1 -- 172.21.15.55:0/2019651346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62340cb330 con 0x7f623406b500 2024-05-20T22:08:52.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.436+0000 7f623b36b640 1 -- 172.21.15.55:0/2019651346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6234069090 con 0x7f623406b500 2024-05-20T22:08:52.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.436+0000 7f6229ffb640 1 -- 172.21.15.55:0/2019651346 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f623001b460 con 0x7f623406b500 2024-05-20T22:08:52.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.437+0000 7f6229ffb640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f621806be30 0x7f621806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.437+0000 7f6229ffb640 1 -- 172.21.15.55:0/2019651346 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f623008ac00 con 0x7f623406b500 2024-05-20T22:08:52.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.437+0000 7f6238906640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f621806be30 0x7f621806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.437+0000 7f6238906640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f621806be30 0x7f621806e340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6224034000 tx=0x7f622402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:52.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.441+0000 7f6229ffb640 1 -- 172.21.15.55:0/2019651346 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6230091080 con 0x7f623406b500 2024-05-20T22:08:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.624+0000 7f623b36b640 1 -- 172.21.15.55:0/2019651346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f623415f800 con 0x7f623406b500 2024-05-20T22:08:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.625+0000 7f6229ffb640 1 -- 172.21.15.55:0/2019651346 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f6230020020 con 0x7f623406b500 2024-05-20T22:08:52.626 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.626+0000 7f623b36b640 1 -- 172.21.15.55:0/2019651346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f621806be30 msgr2=0x7f621806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.626+0000 7f623b36b640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f621806be30 0x7f621806e340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6224034000 tx=0x7f622402e000 comp rx=0 tx=0).stop 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.627+0000 7f623b36b640 1 -- 172.21.15.55:0/2019651346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623406b500 msgr2=0x7f62340c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.627+0000 7f623b36b640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623406b500 0x7f62340c9370 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f623000b420 tx=0x7f623000b8e0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.627+0000 7f623b36b640 1 -- 172.21.15.55:0/2019651346 shutdown_connections 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.627+0000 7f623b36b640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f621806be30 0x7f621806e340 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.627+0000 7f623b36b640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623415f150 0x7f62340b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.627+0000 7f623b36b640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f62340d32a0 0x7f6234152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.627+0000 7f623b36b640 1 --2- 172.21.15.55:0/2019651346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f623406b500 0x7f62340c9370 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.627+0000 7f623b36b640 1 -- 172.21.15.55:0/2019651346 >> 172.21.15.55:0/2019651346 conn(0x7f62340b6de0 msgr2=0x7f62340b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.627+0000 7f623b36b640 1 -- 172.21.15.55:0/2019651346 shutdown_connections 2024-05-20T22:08:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.627+0000 7f623b36b640 1 -- 172.21.15.55:0/2019651346 wait complete. 2024-05-20T22:08:52.639 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:08:52.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.754+0000 7f98a9c7a640 1 Processor -- start 2024-05-20T22:08:52.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.754+0000 7f98a9c7a640 1 -- start start 2024-05-20T22:08:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.755+0000 7f98a9c7a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f98a415f150 0x7f98a40cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.755+0000 7f98a9c7a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a406b500 0x7f98a406b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.755+0000 7f98a9c7a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a40d3780 con 0x7f98a40d32a0 2024-05-20T22:08:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.755+0000 7f98a9c7a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40d3a60 con 0x7f98a415f150 2024-05-20T22:08:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.755+0000 7f98a9c7a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98a40d3d40 con 0x7f98a406b500 2024-05-20T22:08:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a37fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f98a40d32a0 0x7f98a40cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53322/0 (socket says 172.21.15.55:53322) 2024-05-20T22:08:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a37fe640 1 -- 172.21.15.55:0/1701093107 learned_addr learned my addr 172.21.15.55:0/1701093107 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a3fff640 1 --2- 172.21.15.55:0/1701093107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a406b500 0x7f98a406b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a2ffd640 1 --2- 172.21.15.55:0/1701093107 >> v2:172.21.15.138:3300/0 conn(0x7f98a415f150 0x7f98a40cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a27fc640 1 -- 172.21.15.55:0/1701093107 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1301643009 0 0) 0x7f98a40d3780 con 0x7f98a40d32a0 2024-05-20T22:08:52.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a27fc640 1 -- 172.21.15.55:0/1701093107 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f988c003520 con 0x7f98a40d32a0 2024-05-20T22:08:52.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a27fc640 1 -- 172.21.15.55:0/1701093107 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 540284516 0 0) 0x7f988c003520 con 0x7f98a40d32a0 2024-05-20T22:08:52.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a3fff640 1 -- 172.21.15.55:0/1701093107 >> v2:172.21.15.138:3300/0 conn(0x7f98a415f150 msgr2=0x7f98a40cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:52.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a3fff640 1 --2- 172.21.15.55:0/1701093107 >> v2:172.21.15.138:3300/0 conn(0x7f98a415f150 0x7f98a40cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a3fff640 1 -- 172.21.15.55:0/1701093107 >> v1:172.21.15.55:6789/0 conn(0x7f98a40d32a0 legacy=0x7f98a40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a3fff640 1 -- 172.21.15.55:0/1701093107 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40d4020 con 0x7f98a406b500 2024-05-20T22:08:52.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.756+0000 7f98a2ffd640 1 --2- 172.21.15.55:0/1701093107 >> v2:172.21.15.138:3300/0 conn(0x7f98a415f150 0x7f98a40cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:52.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.757+0000 7f98a3fff640 1 --2- 172.21.15.55:0/1701093107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a406b500 0x7f98a406b910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f989c009d90 tx=0x7f989c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6dc40e5f0e32cd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:52.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.757+0000 7f98a27fc640 1 -- 172.21.15.55:0/1701093107 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894001670 con 0x7f98a406b500 2024-05-20T22:08:52.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.757+0000 7f98a27fc640 1 -- 172.21.15.55:0/1701093107 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989c007540 con 0x7f98a406b500 2024-05-20T22:08:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.757+0000 7f98a27fc640 1 -- 172.21.15.55:0/1701093107 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c007910 con 0x7f98a406b500 2024-05-20T22:08:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.758+0000 7f98a9c7a640 1 -- 172.21.15.55:0/1701093107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a406b500 msgr2=0x7f98a406b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.758+0000 7f98a9c7a640 1 --2- 172.21.15.55:0/1701093107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a406b500 0x7f98a406b910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f989c009d90 tx=0x7f989c02a5f0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.758+0000 7f98a9c7a640 1 -- 172.21.15.55:0/1701093107 shutdown_connections 2024-05-20T22:08:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.758+0000 7f98a9c7a640 1 --2- 172.21.15.55:0/1701093107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a406b500 0x7f98a406b910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.758+0000 7f98a9c7a640 1 --2- 172.21.15.55:0/1701093107 >> v2:172.21.15.138:3300/0 conn(0x7f98a415f150 0x7f98a40cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.758+0000 7f98a9c7a640 1 -- 172.21.15.55:0/1701093107 >> 172.21.15.55:0/1701093107 conn(0x7f98a40b6de0 msgr2=0x7f98a40d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.758+0000 7f98a9c7a640 1 -- 172.21.15.55:0/1701093107 shutdown_connections 2024-05-20T22:08:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.758+0000 7f98a9c7a640 1 -- 172.21.15.55:0/1701093107 wait complete. 2024-05-20T22:08:52.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.760+0000 7f98a9c7a640 1 Processor -- start 2024-05-20T22:08:52.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.760+0000 7f98a9c7a640 1 -- start start 2024-05-20T22:08:52.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.761+0000 7f98a9c7a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a406b500 0x7f98a4153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.761+0000 7f98a9c7a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a40d32a0 0x7f98a415ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.761+0000 7f98a9c7a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a415f150 0x7f98a40c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.761+0000 7f98a9c7a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f98a40d4820 con 0x7f98a406b500 2024-05-20T22:08:52.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.761+0000 7f98a9c7a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98a415f560 con 0x7f98a40d32a0 2024-05-20T22:08:52.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.761+0000 7f98a9c7a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98a40c9f70 con 0x7f98a415f150 2024-05-20T22:08:52.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.761+0000 7f98a37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a406b500 0x7f98a4153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.761+0000 7f98a37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a406b500 0x7f98a4153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60320/0 (socket says 172.21.15.55:60320) 2024-05-20T22:08:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.761+0000 7f98a37fe640 1 -- 172.21.15.55:0/3623394374 learned_addr learned my addr 172.21.15.55:0/3623394374 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f98a3fff640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a415f150 0x7f98a40c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f98a37fe640 1 -- 172.21.15.55:0/3623394374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a415f150 msgr2=0x7f98a40c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f98a2ffd640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a40d32a0 0x7f98a415ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f98a37fe640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a415f150 0x7f98a40c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f98a37fe640 1 -- 172.21.15.55:0/3623394374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a40d32a0 msgr2=0x7f98a415ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f98a37fe640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a40d32a0 0x7f98a415ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f98a37fe640 1 -- 172.21.15.55:0/3623394374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40ccc90 con 0x7f98a406b500 2024-05-20T22:08:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f98a37fe640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a406b500 0x7f98a4153840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f989400c700 tx=0x7f989400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:52.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f988bfff640 1 -- 172.21.15.55:0/3623394374 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894017070 con 0x7f98a406b500 2024-05-20T22:08:52.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f988bfff640 1 -- 172.21.15.55:0/3623394374 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9894005ad0 con 0x7f98a406b500 2024-05-20T22:08:52.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f988bfff640 1 -- 172.21.15.55:0/3623394374 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894007070 con 0x7f98a406b500 2024-05-20T22:08:52.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f98a9c7a640 1 -- 172.21.15.55:0/3623394374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a40ccf70 con 0x7f98a406b500 2024-05-20T22:08:52.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.762+0000 7f98a9c7a640 1 -- 172.21.15.55:0/3623394374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a40cd530 con 0x7f98a406b500 2024-05-20T22:08:52.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.763+0000 7f98a9c7a640 1 -- 172.21.15.55:0/3623394374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a4069090 con 0x7f98a406b500 2024-05-20T22:08:52.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.764+0000 7f988bfff640 1 -- 172.21.15.55:0/3623394374 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f989401a070 con 0x7f98a406b500 2024-05-20T22:08:52.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.764+0000 7f988bfff640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989006be60 0x7f989006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:52.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.765+0000 7f98a2ffd640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989006be60 0x7f989006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:52.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.765+0000 7f988bfff640 1 -- 172.21.15.55:0/3623394374 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f989408a5d0 con 0x7f98a406b500 2024-05-20T22:08:52.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.765+0000 7f98a2ffd640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989006be60 0x7f989006e370 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f98a40ce620 tx=0x7f9898008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:52.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.769+0000 7f988bfff640 1 -- 172.21.15.55:0/3623394374 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9894058530 con 0x7f98a406b500 2024-05-20T22:08:52.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.965+0000 7f98a9c7a640 1 -- 172.21.15.55:0/3623394374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f98a40d2e50 con 0x7f98a406b500 2024-05-20T22:08:52.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.965+0000 7f988bfff640 1 -- 172.21.15.55:0/3623394374 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f989405c830 con 0x7f98a406b500 2024-05-20T22:08:52.967 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.967+0000 7f98a9c7a640 1 -- 172.21.15.55:0/3623394374 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989006be60 msgr2=0x7f989006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.967+0000 7f98a9c7a640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989006be60 0x7f989006e370 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f98a40ce620 tx=0x7f9898008040 comp rx=0 tx=0).stop 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.967+0000 7f98a9c7a640 1 -- 172.21.15.55:0/3623394374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a406b500 msgr2=0x7f98a4153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.967+0000 7f98a9c7a640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a406b500 0x7f98a4153840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f989400c700 tx=0x7f989400cbc0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.967+0000 7f98a9c7a640 1 -- 172.21.15.55:0/3623394374 shutdown_connections 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.967+0000 7f98a9c7a640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989006be60 0x7f989006e370 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.967+0000 7f98a9c7a640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a415f150 0x7f98a40c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.967+0000 7f98a9c7a640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a40d32a0 0x7f98a415ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.967+0000 7f98a9c7a640 1 --2- 172.21.15.55:0/3623394374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a406b500 0x7f98a4153840 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.967+0000 7f98a9c7a640 1 -- 172.21.15.55:0/3623394374 >> 172.21.15.55:0/3623394374 conn(0x7f98a40b6de0 msgr2=0x7f98a406b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.968+0000 7f98a9c7a640 1 -- 172.21.15.55:0/3623394374 shutdown_connections 2024-05-20T22:08:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:52.968+0000 7f98a9c7a640 1 -- 172.21.15.55:0/3623394374 wait complete. 2024-05-20T22:08:52.980 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:08:52.980 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:52.997 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:53.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.152+0000 7fd43e538640 1 Processor -- start 2024-05-20T22:08:53.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.152+0000 7fd43e538640 1 -- start start 2024-05-20T22:08:53.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.153+0000 7fd43e538640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4380d4220 0x7fd43806c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.153+0000 7fd43e538640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4380b9f80 0x7fd4380d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.153+0000 7fd43e538640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd43806ce60 con 0x7fd43806ca50 2024-05-20T22:08:53.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.153+0000 7fd43e538640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd4380b4320 con 0x7fd4380d4220 2024-05-20T22:08:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.153+0000 7fd43e538640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd4380b7320 con 0x7fd4380b9f80 2024-05-20T22:08:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.154+0000 7fd43cad5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4380b9f80 0x7fd4380d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.154+0000 7fd437fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4380d4220 0x7fd43806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.154+0000 7fd43cad5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4380b9f80 0x7fd4380d2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:34226/0 (socket says 172.21.15.55:34226) 2024-05-20T22:08:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.154+0000 7fd43cad5640 1 -- 172.21.15.55:0/3761078026 learned_addr learned my addr 172.21.15.55:0/3761078026 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.154+0000 7fd437fff640 1 -- 172.21.15.55:0/3761078026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4380b9f80 msgr2=0x7fd4380d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.154+0000 7fd437fff640 1 --2- 172.21.15.55:0/3761078026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4380b9f80 0x7fd4380d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.154+0000 7fd437fff640 1 -- 172.21.15.55:0/3761078026 >> v1:172.21.15.138:6789/0 conn(0x7fd43806ca50 legacy=0x7fd4380d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.154+0000 7fd437fff640 1 -- 172.21.15.55:0/3761078026 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4380d0580 con 0x7fd4380d4220 2024-05-20T22:08:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.154+0000 7fd436ffd640 1 -- 172.21.15.55:0/3761078026 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120164142 0 0) 0x7fd42c001110 con 0x7fd43806ca50 2024-05-20T22:08:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.154+0000 7fd43cad5640 1 --2- 172.21.15.55:0/3761078026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4380b9f80 0x7fd4380d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.155+0000 7fd437fff640 1 --2- 172.21.15.55:0/3761078026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4380d4220 0x7fd43806c510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd4280097a0 tx=0x7fd42802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae729e39141e382 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.155+0000 7fd436ffd640 1 -- 172.21.15.55:0/3761078026 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428037070 con 0x7fd4380d4220 2024-05-20T22:08:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.155+0000 7fd436ffd640 1 -- 172.21.15.55:0/3761078026 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd428032440 con 0x7fd4380d4220 2024-05-20T22:08:53.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.155+0000 7fd436ffd640 1 -- 172.21.15.55:0/3761078026 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428032810 con 0x7fd4380d4220 2024-05-20T22:08:53.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.156+0000 7fd43e538640 1 -- 172.21.15.55:0/3761078026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4380d4220 msgr2=0x7fd43806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.156+0000 7fd43e538640 1 --2- 172.21.15.55:0/3761078026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4380d4220 0x7fd43806c510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd4280097a0 tx=0x7fd42802a180 comp rx=0 tx=0).stop 2024-05-20T22:08:53.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.156+0000 7fd43e538640 1 -- 172.21.15.55:0/3761078026 shutdown_connections 2024-05-20T22:08:53.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.156+0000 7fd43e538640 1 --2- 172.21.15.55:0/3761078026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4380b9f80 0x7fd4380d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.156+0000 7fd43e538640 1 --2- 172.21.15.55:0/3761078026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4380d4220 0x7fd43806c510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.156+0000 7fd43e538640 1 -- 172.21.15.55:0/3761078026 >> 172.21.15.55:0/3761078026 conn(0x7fd4380b6720 msgr2=0x7fd4380b6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.156+0000 7fd43e538640 1 -- 172.21.15.55:0/3761078026 shutdown_connections 2024-05-20T22:08:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.156+0000 7fd43e538640 1 -- 172.21.15.55:0/3761078026 wait complete. 2024-05-20T22:08:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.158+0000 7fd43e538640 1 Processor -- start 2024-05-20T22:08:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.158+0000 7fd43e538640 1 -- start start 2024-05-20T22:08:53.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd43e538640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd43806ca50 0x7fd43815cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd43e538640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4380b9f80 0x7fd4381f9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd43e538640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4380d4220 0x7fd4381f9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd43e538640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd4380b8a50 con 0x7fd4380b9f80 2024-05-20T22:08:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd43e538640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4380b8770 con 0x7fd4380d4220 2024-05-20T22:08:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd43e538640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd4380b8420 con 0x7fd43806ca50 2024-05-20T22:08:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd4377fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4380b9f80 0x7fd4381f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd4377fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4380b9f80 0x7fd4381f9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60330/0 (socket says 172.21.15.55:60330) 2024-05-20T22:08:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd4377fe640 1 -- 172.21.15.55:0/2892987549 learned_addr learned my addr 172.21.15.55:0/2892987549 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd437fff640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd43806ca50 0x7fd43815cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.159+0000 7fd43cad5640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4380d4220 0x7fd4381f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd4377fe640 1 -- 172.21.15.55:0/2892987549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd43806ca50 msgr2=0x7fd43815cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd4377fe640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd43806ca50 0x7fd43815cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd4377fe640 1 -- 172.21.15.55:0/2892987549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4380d4220 msgr2=0x7fd4381f9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd4377fe640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4380d4220 0x7fd4381f9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd4377fe640 1 -- 172.21.15.55:0/2892987549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4381eb2e0 con 0x7fd4380b9f80 2024-05-20T22:08:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd4377fe640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4380b9f80 0x7fd4381f9440 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00c440 tx=0x7fd42c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd434ff9640 1 -- 172.21.15.55:0/2892987549 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c017070 con 0x7fd4380b9f80 2024-05-20T22:08:53.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd434ff9640 1 -- 172.21.15.55:0/2892987549 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42c005480 con 0x7fd4380b9f80 2024-05-20T22:08:53.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd43e538640 1 -- 172.21.15.55:0/2892987549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4381ea240 con 0x7fd4380b9f80 2024-05-20T22:08:53.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd434ff9640 1 -- 172.21.15.55:0/2892987549 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c003830 con 0x7fd4380b9f80 2024-05-20T22:08:53.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.160+0000 7fd43e538640 1 -- 172.21.15.55:0/2892987549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4381ea800 con 0x7fd4380b9f80 2024-05-20T22:08:53.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.161+0000 7fd43e538640 1 -- 172.21.15.55:0/2892987549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd438069090 con 0x7fd4380b9f80 2024-05-20T22:08:53.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.162+0000 7fd434ff9640 1 -- 172.21.15.55:0/2892987549 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd42c01a070 con 0x7fd4380b9f80 2024-05-20T22:08:53.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.162+0000 7fd434ff9640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd42406be60 0x7fd42406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.162+0000 7fd434ff9640 1 -- 172.21.15.55:0/2892987549 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd42c08a510 con 0x7fd4380b9f80 2024-05-20T22:08:53.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.162+0000 7fd437fff640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd42406be60 0x7fd42406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.163+0000 7fd437fff640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd42406be60 0x7fd42406e370 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd428034000 tx=0x7fd42802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:53.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.167+0000 7fd434ff9640 1 -- 172.21.15.55:0/2892987549 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd43806ce60 con 0x7fd4380b9f80 2024-05-20T22:08:53.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.351+0000 7fd43e538640 1 -- 172.21.15.55:0/2892987549 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4380cfa00 con 0x7fd4380b9f80 2024-05-20T22:08:53.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.352+0000 7fd434ff9640 1 -- 172.21.15.55:0/2892987549 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7fd42c0583e0 con 0x7fd4380b9f80 2024-05-20T22:08:53.353 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:53.354 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:53.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 -- 172.21.15.55:0/2892987549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd42406be60 msgr2=0x7fd42406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd42406be60 0x7fd42406e370 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd428034000 tx=0x7fd42802e000 comp rx=0 tx=0).stop 2024-05-20T22:08:53.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 -- 172.21.15.55:0/2892987549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4380b9f80 msgr2=0x7fd4381f9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4380b9f80 0x7fd4381f9440 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00c440 tx=0x7fd42c00c900 comp rx=0 tx=0).stop 2024-05-20T22:08:53.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 -- 172.21.15.55:0/2892987549 shutdown_connections 2024-05-20T22:08:53.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd42406be60 0x7fd42406e370 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4380d4220 0x7fd4381f9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4380b9f80 0x7fd4381f9440 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 --2- 172.21.15.55:0/2892987549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd43806ca50 0x7fd43815cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 -- 172.21.15.55:0/2892987549 >> 172.21.15.55:0/2892987549 conn(0x7fd4380b6720 msgr2=0x7fd4380b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:53.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 -- 172.21.15.55:0/2892987549 shutdown_connections 2024-05-20T22:08:53.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.354+0000 7fd43e538640 1 -- 172.21.15.55:0/2892987549 wait complete. 2024-05-20T22:08:53.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:53.368 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:53.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.485+0000 7f5ced05a640 1 Processor -- start 2024-05-20T22:08:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.485+0000 7f5ced05a640 1 -- start start 2024-05-20T22:08:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.486+0000 7f5ced05a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5ce80cf680 0x7f5ce815f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.486+0000 7f5ced05a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ce806cb70 0x7f5ce80b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.486+0000 7f5ced05a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce806b6a0 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.486+0000 7f5ced05a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f5ce806b980 con 0x7f5ce806cb70 2024-05-20T22:08:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.486+0000 7f5ced05a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5ce806bc60 con 0x7f5ce80cf680 2024-05-20T22:08:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.487+0000 7f5ce759e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ce806cb70 0x7f5ce80b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.487+0000 7f5ce759e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ce806cb70 0x7f5ce80b83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60338/0 (socket says 172.21.15.55:60338) 2024-05-20T22:08:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.487+0000 7f5ce759e640 1 -- 172.21.15.55:0/1315770480 learned_addr learned my addr 172.21.15.55:0/1315770480 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:53.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.487+0000 7f5ce6d9d640 1 --2- 172.21.15.55:0/1315770480 >> v2:172.21.15.138:3300/0 conn(0x7f5ce80cf680 0x7f5ce815f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.487+0000 7f5ce759e640 1 -- 172.21.15.55:0/1315770480 >> v2:172.21.15.138:3300/0 conn(0x7f5ce80cf680 msgr2=0x7f5ce815f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:53.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.487+0000 7f5ce759e640 1 --2- 172.21.15.55:0/1315770480 >> v2:172.21.15.138:3300/0 conn(0x7f5ce80cf680 0x7f5ce815f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.487+0000 7f5ce759e640 1 -- 172.21.15.55:0/1315770480 >> v1:172.21.15.138:6789/0 conn(0x7f5ce80b7fb0 legacy=0x7f5ce806c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:53.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.487+0000 7f5ce759e640 1 -- 172.21.15.55:0/1315770480 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce806bf40 con 0x7f5ce806cb70 2024-05-20T22:08:53.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.487+0000 7f5ce6d9d640 1 --2- 172.21.15.55:0/1315770480 >> v2:172.21.15.138:3300/0 conn(0x7f5ce80cf680 0x7f5ce815f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:53.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.488+0000 7f5ce759e640 1 --2- 172.21.15.55:0/1315770480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce806cb70 0x7f5ce80b83c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80b6f50 tx=0x7f5ce002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5ce0e60df1c696 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:53.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.488+0000 7f5ce5d9b640 1 -- 172.21.15.55:0/1315770480 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0037070 con 0x7f5ce806cb70 2024-05-20T22:08:53.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.488+0000 7f5ce5d9b640 1 -- 172.21.15.55:0/1315770480 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce0032440 con 0x7f5ce806cb70 2024-05-20T22:08:53.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.488+0000 7f5ce5d9b640 1 -- 172.21.15.55:0/1315770480 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0032810 con 0x7f5ce806cb70 2024-05-20T22:08:53.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.488+0000 7f5ced05a640 1 -- 172.21.15.55:0/1315770480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce806cb70 msgr2=0x7f5ce80b83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.489+0000 7f5ced05a640 1 --2- 172.21.15.55:0/1315770480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce806cb70 0x7f5ce80b83c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80b6f50 tx=0x7f5ce002a1a0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.489+0000 7f5ced05a640 1 -- 172.21.15.55:0/1315770480 shutdown_connections 2024-05-20T22:08:53.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.489+0000 7f5ced05a640 1 --2- 172.21.15.55:0/1315770480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce806cb70 0x7f5ce80b83c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.489+0000 7f5ced05a640 1 --2- 172.21.15.55:0/1315770480 >> v2:172.21.15.138:3300/0 conn(0x7f5ce80cf680 0x7f5ce815f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.489+0000 7f5ced05a640 1 -- 172.21.15.55:0/1315770480 >> 172.21.15.55:0/1315770480 conn(0x7f5ce80d5450 msgr2=0x7f5ce80b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:53.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.489+0000 7f5ced05a640 1 -- 172.21.15.55:0/1315770480 shutdown_connections 2024-05-20T22:08:53.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.489+0000 7f5ced05a640 1 -- 172.21.15.55:0/1315770480 wait complete. 2024-05-20T22:08:53.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.491+0000 7f5ced05a640 1 Processor -- start 2024-05-20T22:08:53.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.491+0000 7f5ced05a640 1 -- start start 2024-05-20T22:08:53.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ced05a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce806cb70 0x7f5ce8155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ced05a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce80b7fb0 0x7f5ce8153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ced05a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce80cf680 0x7f5ce81f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ced05a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce80d3740 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ced05a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce80d35c0 con 0x7f5ce80cf680 2024-05-20T22:08:53.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ced05a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce80d38c0 con 0x7f5ce806cb70 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ce659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce80b7fb0 0x7f5ce8153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ce659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce80b7fb0 0x7f5ce8153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60352/0 (socket says 172.21.15.55:60352) 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ce659c640 1 -- 172.21.15.55:0/1496982758 learned_addr learned my addr 172.21.15.55:0/1496982758 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ce6d9d640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce806cb70 0x7f5ce8155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ce759e640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce80cf680 0x7f5ce81f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.492+0000 7f5ce659c640 1 -- 172.21.15.55:0/1496982758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce806cb70 msgr2=0x7f5ce8155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ce659c640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce806cb70 0x7f5ce8155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ce659c640 1 -- 172.21.15.55:0/1496982758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce80cf680 msgr2=0x7f5ce81f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ce659c640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce80cf680 0x7f5ce81f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ce659c640 1 -- 172.21.15.55:0/1496982758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce81547e0 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ce6d9d640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce806cb70 0x7f5ce8155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ce759e640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce80cf680 0x7f5ce81f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ce659c640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce80b7fb0 0x7f5ce8153d20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00b9f0 tx=0x7f5cdc00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ccf7fe640 1 -- 172.21.15.55:0/1496982758 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00cbd0 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ccf7fe640 1 -- 172.21.15.55:0/1496982758 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cdc00cd70 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ced05a640 1 -- 172.21.15.55:0/1496982758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce8154ac0 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ccf7fe640 1 -- 172.21.15.55:0/1496982758 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc003bb0 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.493+0000 7f5ced05a640 1 -- 172.21.15.55:0/1496982758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce8155080 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.494+0000 7f5ced05a640 1 -- 172.21.15.55:0/1496982758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce8154550 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.495+0000 7f5ccf7fe640 1 -- 172.21.15.55:0/1496982758 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5cdc014440 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.495+0000 7f5ccf7fe640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5cd406be60 0x7f5cd406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.495+0000 7f5ccf7fe640 1 -- 172.21.15.55:0/1496982758 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5cdc08b100 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.495+0000 7f5ce6d9d640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5cd406be60 0x7f5cd406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.496+0000 7f5ce6d9d640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5cd406be60 0x7f5cd406e370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800a990 tx=0x7f5cd8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:53.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.500+0000 7f5ccf7fe640 1 -- 172.21.15.55:0/1496982758 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5cdc0596b0 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.682+0000 7f5ced05a640 1 -- 172.21.15.55:0/1496982758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ce8069090 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.682+0000 7f5ccf7fe640 1 -- 172.21.15.55:0/1496982758 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f5cdc058fb0 con 0x7f5ce80b7fb0 2024-05-20T22:08:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 -- 172.21.15.55:0/1496982758 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5cd406be60 msgr2=0x7f5cd406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5cd406be60 0x7f5cd406e370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800a990 tx=0x7f5cd8008040 comp rx=0 tx=0).stop 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 -- 172.21.15.55:0/1496982758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce80b7fb0 msgr2=0x7f5ce8153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce80b7fb0 0x7f5ce8153d20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00b9f0 tx=0x7f5cdc00beb0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 -- 172.21.15.55:0/1496982758 shutdown_connections 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5cd406be60 0x7f5cd406e370 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ce80cf680 0x7f5ce81f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ce80b7fb0 0x7f5ce8153d20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 --2- 172.21.15.55:0/1496982758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ce806cb70 0x7f5ce8155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 -- 172.21.15.55:0/1496982758 >> 172.21.15.55:0/1496982758 conn(0x7f5ce80d5450 msgr2=0x7f5ce80b5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 -- 172.21.15.55:0/1496982758 shutdown_connections 2024-05-20T22:08:53.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.685+0000 7f5ced05a640 1 -- 172.21.15.55:0/1496982758 wait complete. 2024-05-20T22:08:53.698 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:53.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.816+0000 7f39c50f0640 1 Processor -- start 2024-05-20T22:08:53.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.816+0000 7f39c50f0640 1 -- start start 2024-05-20T22:08:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.817+0000 7f39c50f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c015f150 0x7f39c00cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.817+0000 7f39c50f0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39c006b500 0x7f39c006b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.817+0000 7f39c50f0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39c00d3780 con 0x7f39c00d32a0 2024-05-20T22:08:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.817+0000 7f39c50f0640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f39c00d3a60 con 0x7f39c006b500 2024-05-20T22:08:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.817+0000 7f39c50f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39c00d3d40 con 0x7f39c015f150 2024-05-20T22:08:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39bf59e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39c006b500 0x7f39c006b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39be59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c015f150 0x7f39c00cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39bf59e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39c006b500 0x7f39c006b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:49480/0 (socket says 172.21.15.55:49480) 2024-05-20T22:08:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39bed9d640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f39c00d32a0 0x7f39c00cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:52256/0 (socket says 172.21.15.55:52256) 2024-05-20T22:08:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39be59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c015f150 0x7f39c00cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:34244/0 (socket says 172.21.15.55:34244) 2024-05-20T22:08:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39bf59e640 1 -- 172.21.15.55:0/1723320987 learned_addr learned my addr 172.21.15.55:0/1723320987 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39bf59e640 1 -- 172.21.15.55:0/1723320987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c015f150 msgr2=0x7f39c00cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39bf59e640 1 --2- 172.21.15.55:0/1723320987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c015f150 0x7f39c00cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39bf59e640 1 -- 172.21.15.55:0/1723320987 >> v1:172.21.15.138:6789/0 conn(0x7f39c00d32a0 legacy=0x7f39c00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39bf59e640 1 -- 172.21.15.55:0/1723320987 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c00d4020 con 0x7f39c006b500 2024-05-20T22:08:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39bdd9b640 1 -- 172.21.15.55:0/1723320987 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 824369832 0 0) 0x7f39c00d3780 con 0x7f39c00d32a0 2024-05-20T22:08:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.818+0000 7f39be59c640 1 --2- 172.21.15.55:0/1723320987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c015f150 0x7f39c00cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.819+0000 7f39bf59e640 1 --2- 172.21.15.55:0/1723320987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c006b500 0x7f39c006b910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009e50 tx=0x7f39b802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c3e5b6e72d66f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.819+0000 7f39bdd9b640 1 -- 172.21.15.55:0/1723320987 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8036070 con 0x7f39c006b500 2024-05-20T22:08:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.819+0000 7f39bdd9b640 1 -- 172.21.15.55:0/1723320987 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b8031600 con 0x7f39c006b500 2024-05-20T22:08:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.819+0000 7f39bdd9b640 1 -- 172.21.15.55:0/1723320987 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b80319d0 con 0x7f39c006b500 2024-05-20T22:08:53.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.820+0000 7f39c50f0640 1 -- 172.21.15.55:0/1723320987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c006b500 msgr2=0x7f39c006b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.820+0000 7f39c50f0640 1 --2- 172.21.15.55:0/1723320987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c006b500 0x7f39c006b910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009e50 tx=0x7f39b802a710 comp rx=0 tx=0).stop 2024-05-20T22:08:53.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.820+0000 7f39c50f0640 1 -- 172.21.15.55:0/1723320987 shutdown_connections 2024-05-20T22:08:53.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.820+0000 7f39c50f0640 1 --2- 172.21.15.55:0/1723320987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c006b500 0x7f39c006b910 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.820+0000 7f39c50f0640 1 --2- 172.21.15.55:0/1723320987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c015f150 0x7f39c00cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.820+0000 7f39c50f0640 1 -- 172.21.15.55:0/1723320987 >> 172.21.15.55:0/1723320987 conn(0x7f39c00b6de0 msgr2=0x7f39c00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:53.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.820+0000 7f39c50f0640 1 -- 172.21.15.55:0/1723320987 shutdown_connections 2024-05-20T22:08:53.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.820+0000 7f39c50f0640 1 -- 172.21.15.55:0/1723320987 wait complete. 2024-05-20T22:08:53.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.822+0000 7f39c50f0640 1 Processor -- start 2024-05-20T22:08:53.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.822+0000 7f39c50f0640 1 -- start start 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.823+0000 7f39c50f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c006b500 0x7f39c01614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.823+0000 7f39c50f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c00d32a0 0x7f39c014b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.823+0000 7f39c50f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39c015f150 0x7f39c00b9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.823+0000 7f39c50f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f39c00d4820 con 0x7f39c015f150 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.823+0000 7f39c50f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39c015f560 con 0x7f39c006b500 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.823+0000 7f39c50f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39c00cd6f0 con 0x7f39c00d32a0 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39bf59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39c015f150 0x7f39c00b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39bf59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39c015f150 0x7f39c00b9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60360/0 (socket says 172.21.15.55:60360) 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39bf59e640 1 -- 172.21.15.55:0/3923774249 learned_addr learned my addr 172.21.15.55:0/3923774249 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39be59c640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c00d32a0 0x7f39c014b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39bed9d640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c006b500 0x7f39c01614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39bf59e640 1 -- 172.21.15.55:0/3923774249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c00d32a0 msgr2=0x7f39c014b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39bf59e640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c00d32a0 0x7f39c014b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39bf59e640 1 -- 172.21.15.55:0/3923774249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c006b500 msgr2=0x7f39c01614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39bf59e640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c006b500 0x7f39c01614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39bf59e640 1 -- 172.21.15.55:0/3923774249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c01620b0 con 0x7f39c015f150 2024-05-20T22:08:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39bf59e640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39c015f150 0x7f39c00b9850 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f39b8034040 tx=0x7f39b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39a77fe640 1 -- 172.21.15.55:0/3923774249 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8041020 con 0x7f39c015f150 2024-05-20T22:08:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39a77fe640 1 -- 172.21.15.55:0/3923774249 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b8037040 con 0x7f39c015f150 2024-05-20T22:08:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39c50f0640 1 -- 172.21.15.55:0/3923774249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c0162390 con 0x7f39c015f150 2024-05-20T22:08:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.824+0000 7f39a77fe640 1 -- 172.21.15.55:0/3923774249 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8036050 con 0x7f39c015f150 2024-05-20T22:08:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.825+0000 7f39c50f0640 1 -- 172.21.15.55:0/3923774249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c00cbe90 con 0x7f39c015f150 2024-05-20T22:08:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.826+0000 7f39c50f0640 1 -- 172.21.15.55:0/3923774249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c0161e20 con 0x7f39c015f150 2024-05-20T22:08:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.826+0000 7f39a77fe640 1 -- 172.21.15.55:0/3923774249 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f39b8007c50 con 0x7f39c015f150 2024-05-20T22:08:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.826+0000 7f39a77fe640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f39ac06be30 0x7f39ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.827+0000 7f39a77fe640 1 -- 172.21.15.55:0/3923774249 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f39b80ab870 con 0x7f39c015f150 2024-05-20T22:08:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.827+0000 7f39bed9d640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f39ac06be30 0x7f39ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.827+0000 7f39bed9d640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f39ac06be30 0x7f39ac06e340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39b0007fd0 tx=0x7f39b0007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:53.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:53.831+0000 7f39a77fe640 1 -- 172.21.15.55:0/3923774249 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f39b804b060 con 0x7f39c015f150 2024-05-20T22:08:54.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.014+0000 7f39c50f0640 1 -- 172.21.15.55:0/3923774249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f39c0069090 con 0x7f39c015f150 2024-05-20T22:08:54.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.014+0000 7f39a77fe640 1 -- 172.21.15.55:0/3923774249 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f39b80797d0 con 0x7f39c015f150 2024-05-20T22:08:54.016 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:54.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 -- 172.21.15.55:0/3923774249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f39ac06be30 msgr2=0x7f39ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f39ac06be30 0x7f39ac06e340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39b0007fd0 tx=0x7f39b0007de0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 -- 172.21.15.55:0/3923774249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39c015f150 msgr2=0x7f39c00b9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39c015f150 0x7f39c00b9850 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f39b8034040 tx=0x7f39b8004a40 comp rx=0 tx=0).stop 2024-05-20T22:08:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 -- 172.21.15.55:0/3923774249 shutdown_connections 2024-05-20T22:08:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f39ac06be30 0x7f39ac06e340 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39c015f150 0x7f39c00b9850 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39c00d32a0 0x7f39c014b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 --2- 172.21.15.55:0/3923774249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c006b500 0x7f39c01614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 -- 172.21.15.55:0/3923774249 >> 172.21.15.55:0/3923774249 conn(0x7f39c00b6de0 msgr2=0x7f39c00b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 -- 172.21.15.55:0/3923774249 shutdown_connections 2024-05-20T22:08:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.016+0000 7f39c50f0640 1 -- 172.21.15.55:0/3923774249 wait complete. 2024-05-20T22:08:54.030 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:54.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.146+0000 7fa6a6745640 1 Processor -- start 2024-05-20T22:08:54.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.146+0000 7fa6a6745640 1 -- start start 2024-05-20T22:08:54.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.147+0000 7fa6a6745640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6a00b29e0 0x7fa6a015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.147+0000 7fa6a6745640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a008c690 con 0x7fa6a00cf7f0 2024-05-20T22:08:54.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.147+0000 7fa6a6745640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a008c970 con 0x7fa6a008bcb0 2024-05-20T22:08:54.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.147+0000 7fa6a6745640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a00b2df0 con 0x7fa6a00b29e0 2024-05-20T22:08:54.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.147+0000 7fa69ffff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6a00b29e0 0x7fa6a015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.147+0000 7fa6a4ce2640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa6a00cf7f0 0x7fa6a00d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53326/0 (socket says 172.21.15.55:53326) 2024-05-20T22:08:54.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.147+0000 7fa6a4ce2640 1 -- 172.21.15.55:0/1699957527 learned_addr learned my addr 172.21.15.55:0/1699957527 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:54.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.148+0000 7fa69ffff640 1 -- 172.21.15.55:0/1699957527 >> v1:172.21.15.138:6789/0 conn(0x7fa6a008bcb0 legacy=0x7fa6a00cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:54.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.148+0000 7fa69ffff640 1 -- 172.21.15.55:0/1699957527 >> v1:172.21.15.55:6789/0 conn(0x7fa6a00cf7f0 legacy=0x7fa6a00d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.148+0000 7fa69effd640 1 -- 172.21.15.55:0/1699957527 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3504154129 0 0) 0x7fa6a008c690 con 0x7fa6a00cf7f0 2024-05-20T22:08:54.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.148+0000 7fa69ffff640 1 -- 172.21.15.55:0/1699957527 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a00d0de0 con 0x7fa6a00b29e0 2024-05-20T22:08:54.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.149+0000 7fa69ffff640 1 --2- 172.21.15.55:0/1699957527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a00b29e0 0x7fa6a015f0e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa6900097a0 tx=0x7fa69002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad53bdd7f45d276b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.149+0000 7fa69effd640 1 -- 172.21.15.55:0/1699957527 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690037070 con 0x7fa6a00b29e0 2024-05-20T22:08:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.149+0000 7fa69effd640 1 -- 172.21.15.55:0/1699957527 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa690032440 con 0x7fa6a00b29e0 2024-05-20T22:08:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.149+0000 7fa69effd640 1 -- 172.21.15.55:0/1699957527 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690032810 con 0x7fa6a00b29e0 2024-05-20T22:08:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.149+0000 7fa6a6745640 1 -- 172.21.15.55:0/1699957527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a00b29e0 msgr2=0x7fa6a015f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.149+0000 7fa6a6745640 1 --2- 172.21.15.55:0/1699957527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a00b29e0 0x7fa6a015f0e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa6900097a0 tx=0x7fa69002a1e0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.149+0000 7fa6a6745640 1 -- 172.21.15.55:0/1699957527 shutdown_connections 2024-05-20T22:08:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.149+0000 7fa6a6745640 1 --2- 172.21.15.55:0/1699957527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a00b29e0 0x7fa6a015f0e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.149+0000 7fa6a6745640 1 -- 172.21.15.55:0/1699957527 >> 172.21.15.55:0/1699957527 conn(0x7fa6a00d2860 msgr2=0x7fa6a00d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.150+0000 7fa6a6745640 1 -- 172.21.15.55:0/1699957527 shutdown_connections 2024-05-20T22:08:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.150+0000 7fa6a6745640 1 -- 172.21.15.55:0/1699957527 wait complete. 2024-05-20T22:08:54.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.151+0000 7fa6a6745640 1 Processor -- start 2024-05-20T22:08:54.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.151+0000 7fa6a6745640 1 -- start start 2024-05-20T22:08:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa6a6745640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a008bcb0 0x7fa6a0161c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa6a6745640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00b29e0 0x7fa6a015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa6a6745640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6a015e830 0x7fa6a0070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa6a6745640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a008cc50 con 0x7fa6a008bcb0 2024-05-20T22:08:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa6a6745640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a015f620 con 0x7fa6a015e830 2024-05-20T22:08:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa6a6745640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a00b80e0 con 0x7fa6a00b29e0 2024-05-20T22:08:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a008bcb0 0x7fa6a0161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a008bcb0 0x7fa6a0161c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60378/0 (socket says 172.21.15.55:60378) 2024-05-20T22:08:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69ffff640 1 -- 172.21.15.55:0/1229707256 learned_addr learned my addr 172.21.15.55:0/1229707256 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69f7fe640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00b29e0 0x7fa6a015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa6a4ce2640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6a015e830 0x7fa6a0070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69ffff640 1 -- 172.21.15.55:0/1229707256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00b29e0 msgr2=0x7fa6a015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69ffff640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00b29e0 0x7fa6a015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69ffff640 1 -- 172.21.15.55:0/1229707256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6a015e830 msgr2=0x7fa6a0070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69ffff640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6a015e830 0x7fa6a0070fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69ffff640 1 -- 172.21.15.55:0/1229707256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a00af810 con 0x7fa6a008bcb0 2024-05-20T22:08:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69f7fe640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00b29e0 0x7fa6a015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:54.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.153+0000 7fa69ffff640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a008bcb0 0x7fa6a0161c60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa690035040 tx=0x7fa6900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:54.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.154+0000 7fa69cff9640 1 -- 172.21.15.55:0/1229707256 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690037040 con 0x7fa6a008bcb0 2024-05-20T22:08:54.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.154+0000 7fa69cff9640 1 -- 172.21.15.55:0/1229707256 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa690004250 con 0x7fa6a008bcb0 2024-05-20T22:08:54.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.154+0000 7fa6a6745640 1 -- 172.21.15.55:0/1229707256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a00afaf0 con 0x7fa6a008bcb0 2024-05-20T22:08:54.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.154+0000 7fa69cff9640 1 -- 172.21.15.55:0/1229707256 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690032e20 con 0x7fa6a008bcb0 2024-05-20T22:08:54.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.154+0000 7fa6a6745640 1 -- 172.21.15.55:0/1229707256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a00b00b0 con 0x7fa6a008bcb0 2024-05-20T22:08:54.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.155+0000 7fa6a6745640 1 -- 172.21.15.55:0/1229707256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a0069090 con 0x7fa6a008bcb0 2024-05-20T22:08:54.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.155+0000 7fa69cff9640 1 -- 172.21.15.55:0/1229707256 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa690030770 con 0x7fa6a008bcb0 2024-05-20T22:08:54.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.156+0000 7fa69cff9640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa68c06be60 0x7fa68c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.156+0000 7fa69cff9640 1 -- 172.21.15.55:0/1229707256 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa6900ac470 con 0x7fa6a008bcb0 2024-05-20T22:08:54.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.156+0000 7fa69f7fe640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa68c06be60 0x7fa68c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.156+0000 7fa69f7fe640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa68c06be60 0x7fa68c06e370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00726e0 tx=0x7fa694008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:54.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.160+0000 7fa69cff9640 1 -- 172.21.15.55:0/1229707256 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa69004c070 con 0x7fa6a008bcb0 2024-05-20T22:08:54.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.344+0000 7fa6a6745640 1 -- 172.21.15.55:0/1229707256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6a00b27c0 con 0x7fa6a008bcb0 2024-05-20T22:08:54.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.344+0000 7fa69cff9640 1 -- 172.21.15.55:0/1229707256 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fa69007a990 con 0x7fa6a008bcb0 2024-05-20T22:08:54.346 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:54.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 -- 172.21.15.55:0/1229707256 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa68c06be60 msgr2=0x7fa68c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa68c06be60 0x7fa68c06e370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00726e0 tx=0x7fa694008040 comp rx=0 tx=0).stop 2024-05-20T22:08:54.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 -- 172.21.15.55:0/1229707256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a008bcb0 msgr2=0x7fa6a0161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a008bcb0 0x7fa6a0161c60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa690035040 tx=0x7fa6900046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 -- 172.21.15.55:0/1229707256 shutdown_connections 2024-05-20T22:08:54.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa68c06be60 0x7fa68c06e370 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6a015e830 0x7fa6a0070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6a00b29e0 0x7fa6a015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 --2- 172.21.15.55:0/1229707256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6a008bcb0 0x7fa6a0161c60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 -- 172.21.15.55:0/1229707256 >> 172.21.15.55:0/1229707256 conn(0x7fa6a00d2860 msgr2=0x7fa6a0159660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:54.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 -- 172.21.15.55:0/1229707256 shutdown_connections 2024-05-20T22:08:54.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.346+0000 7fa6a6745640 1 -- 172.21.15.55:0/1229707256 wait complete. 2024-05-20T22:08:54.360 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:54.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.476+0000 7feece406640 1 Processor -- start 2024-05-20T22:08:54.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.476+0000 7feece406640 1 -- start start 2024-05-20T22:08:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.477+0000 7feece406640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec80b4080 0x7feec815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.477+0000 7feece406640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feec80d3290 0x7feec80d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.477+0000 7feece406640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec806b500 con 0x7feec80b35e0 2024-05-20T22:08:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.477+0000 7feece406640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feec8163960 con 0x7feec80d3290 2024-05-20T22:08:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.477+0000 7feece406640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feec80b3e10 con 0x7feec80b4080 2024-05-20T22:08:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.477+0000 7feec77fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7feec80b35e0 0x7feec815f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53330/0 (socket says 172.21.15.55:53330) 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.477+0000 7feec77fe640 1 -- 172.21.15.55:0/3945320362 learned_addr learned my addr 172.21.15.55:0/3945320362 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feecc9a3640 1 --2- 172.21.15.55:0/3945320362 >> v2:172.21.15.138:3300/0 conn(0x7feec80d3290 0x7feec80d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec7fff640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec80b4080 0x7feec815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2257791798 0 0) 0x7feebc001110 con 0x7feec80b35e0 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeb0003520 con 0x7feec80b35e0 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2837080752 0 0) 0x7feeb0003520 con 0x7feec80b35e0 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec80b4080 msgr2=0x7feec815f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec80b4080 0x7feec815f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 >> v2:172.21.15.138:3300/0 conn(0x7feec80d3290 msgr2=0x7feec80d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 --2- 172.21.15.55:0/3945320362 >> v2:172.21.15.138:3300/0 conn(0x7feec80d3290 0x7feec80d36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec80b3b30 con 0x7feec80b35e0 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7feebc001840 con 0x7feec80b35e0 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 >> v1:172.21.15.55:6789/0 conn(0x7feec80b35e0 legacy=0x7feec815f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00039b0 0x7feeb0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec80b4080 0x7feeb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb0004d40 0x7feeb0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7feec8163960 con 0x7feeb00039b0 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feec80b3e10 con 0x7feeb0004d40 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feeb000b9c0 con 0x7feec80b4080 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec7fff640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00039b0 0x7feeb0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feec77fe640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec80b4080 0x7feeb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.478+0000 7feecc9a3640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb0004d40 0x7feeb0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.479+0000 7feec7fff640 1 -- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec80b4080 msgr2=0x7feeb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.479+0000 7feec7fff640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec80b4080 0x7feeb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.479+0000 7feec7fff640 1 -- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb0004d40 msgr2=0x7feeb0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:54.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.479+0000 7feec7fff640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb0004d40 0x7feeb0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.479+0000 7feec7fff640 1 -- 172.21.15.55:0/3945320362 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb0003520 con 0x7feeb00039b0 2024-05-20T22:08:54.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.479+0000 7feec7fff640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00039b0 0x7feeb0003e20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009a90 tx=0x7feeb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3387cb916311e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:54.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.479+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8036070 con 0x7feeb00039b0 2024-05-20T22:08:54.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.480+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8031400 con 0x7feeb00039b0 2024-05-20T22:08:54.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.480+0000 7feec6ffd640 1 -- 172.21.15.55:0/3945320362 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8009690 con 0x7feeb00039b0 2024-05-20T22:08:54.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.480+0000 7feece406640 1 -- 172.21.15.55:0/3945320362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00039b0 msgr2=0x7feeb0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.480+0000 7feece406640 1 --2- 172.21.15.55:0/3945320362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00039b0 0x7feeb0003e20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009a90 tx=0x7feeb0005830 comp rx=0 tx=0).stop 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.480+0000 7feecc9a3640 1 -- 172.21.15.55:0/3945320362 reap_dead start 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.480+0000 7feece406640 1 -- 172.21.15.55:0/3945320362 shutdown_connections 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.480+0000 7feece406640 1 -- 172.21.15.55:0/3945320362 >> 172.21.15.55:0/3945320362 conn(0x7feec80d51b0 msgr2=0x7feec80b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.480+0000 7feece406640 1 -- 172.21.15.55:0/3945320362 shutdown_connections 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.480+0000 7feece406640 1 -- 172.21.15.55:0/3945320362 wait complete. 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.482+0000 7feece406640 1 Processor -- start 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.482+0000 7feece406640 1 -- start start 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.483+0000 7feece406640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb00039b0 0x7feec815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feece406640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feec81f49f0 0x7feec81f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feece406640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec81f5340 0x7feec81f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feece406640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7feec81635f0 con 0x7feec81f49f0 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feece406640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feec81604f0 con 0x7feeb00039b0 2024-05-20T22:08:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feece406640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feec815ff30 con 0x7feec81f5340 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feec81f49f0 0x7feec81f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feec81f49f0 0x7feec81f4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60388/0 (socket says 172.21.15.55:60388) 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec77fe640 1 -- 172.21.15.55:0/795465968 learned_addr learned my addr 172.21.15.55:0/795465968 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec7fff640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb00039b0 0x7feec815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feecc9a3640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec81f5340 0x7feec81f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec77fe640 1 -- 172.21.15.55:0/795465968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec81f5340 msgr2=0x7feec81f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec77fe640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec81f5340 0x7feec81f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec77fe640 1 -- 172.21.15.55:0/795465968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb00039b0 msgr2=0x7feec815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec77fe640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb00039b0 0x7feec815b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec77fe640 1 -- 172.21.15.55:0/795465968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec8162070 con 0x7feec81f49f0 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec77fe640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feec81f49f0 0x7feec81f4e00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feebc007370 tx=0x7feebc007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec4ff9640 1 -- 172.21.15.55:0/795465968 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc00f9c0 con 0x7feec81f49f0 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec4ff9640 1 -- 172.21.15.55:0/795465968 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc018070 con 0x7feec81f49f0 2024-05-20T22:08:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feece406640 1 -- 172.21.15.55:0/795465968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec815ead0 con 0x7feec81f49f0 2024-05-20T22:08:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feec4ff9640 1 -- 172.21.15.55:0/795465968 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc00fd60 con 0x7feec81f49f0 2024-05-20T22:08:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.484+0000 7feece406640 1 -- 172.21.15.55:0/795465968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec815e510 con 0x7feec81f49f0 2024-05-20T22:08:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.486+0000 7feece406640 1 -- 172.21.15.55:0/795465968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec8069090 con 0x7feec81f49f0 2024-05-20T22:08:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.486+0000 7feec4ff9640 1 -- 172.21.15.55:0/795465968 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feebc01c410 con 0x7feec81f49f0 2024-05-20T22:08:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.486+0000 7feec4ff9640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be60 0x7feeb406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.487+0000 7feec4ff9640 1 -- 172.21.15.55:0/795465968 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7feebc08ad60 con 0x7feec81f49f0 2024-05-20T22:08:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.487+0000 7feec7fff640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be60 0x7feeb406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.487+0000 7feec7fff640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be60 0x7feeb406e370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feeb8033000 tx=0x7feeb8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:54.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.491+0000 7feec4ff9640 1 -- 172.21.15.55:0/795465968 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7feec806b500 con 0x7feec81f49f0 2024-05-20T22:08:54.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.676+0000 7feece406640 1 -- 172.21.15.55:0/795465968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7feec80d2ec0 con 0x7feec81f49f0 2024-05-20T22:08:54.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.677+0000 7feec4ff9640 1 -- 172.21.15.55:0/795465968 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7feebc058cc0 con 0x7feec81f49f0 2024-05-20T22:08:54.679 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:54.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 -- 172.21.15.55:0/795465968 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be60 msgr2=0x7feeb406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be60 0x7feeb406e370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feeb8033000 tx=0x7feeb8008030 comp rx=0 tx=0).stop 2024-05-20T22:08:54.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 -- 172.21.15.55:0/795465968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feec81f49f0 msgr2=0x7feec81f4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feec81f49f0 0x7feec81f4e00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feebc007370 tx=0x7feebc007830 comp rx=0 tx=0).stop 2024-05-20T22:08:54.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 -- 172.21.15.55:0/795465968 shutdown_connections 2024-05-20T22:08:54.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feeb406be60 0x7feeb406e370 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feec81f5340 0x7feec81f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feec81f49f0 0x7feec81f4e00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 --2- 172.21.15.55:0/795465968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb00039b0 0x7feec815b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 -- 172.21.15.55:0/795465968 >> 172.21.15.55:0/795465968 conn(0x7feec80d51b0 msgr2=0x7feec80b7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:54.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 -- 172.21.15.55:0/795465968 shutdown_connections 2024-05-20T22:08:54.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.679+0000 7feece406640 1 -- 172.21.15.55:0/795465968 wait complete. 2024-05-20T22:08:54.693 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:08:54.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.806+0000 7fbcfa337640 1 Processor -- start 2024-05-20T22:08:54.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.806+0000 7fbcfa337640 1 -- start start 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.808+0000 7fbcfa337640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbcf415f150 0x7fbcf40cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.808+0000 7fbcfa337640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf40d3780 con 0x7fbcf406b500 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.808+0000 7fbcfa337640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf40d3a60 con 0x7fbcf40d32a0 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.808+0000 7fbcfa337640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf40d3d40 con 0x7fbcf415f150 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.808+0000 7fbcf88d4640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbcf406b500 0x7fbcf406b910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53342/0 (socket says 172.21.15.55:53342) 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.808+0000 7fbcf88d4640 1 -- 172.21.15.55:0/2731634570 learned_addr learned my addr 172.21.15.55:0/2731634570 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.808+0000 7fbcf37fe640 1 --2- 172.21.15.55:0/2731634570 >> v2:172.21.15.138:3300/0 conn(0x7fbcf415f150 0x7fbcf40cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.808+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1715075482 0 0) 0x7fbcf40d3780 con 0x7fbcf406b500 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.808+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcdc003520 con 0x7fbcf406b500 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 382026694 0 0) 0x7fbcdc003520 con 0x7fbcf406b500 2024-05-20T22:08:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 >> v2:172.21.15.138:3300/0 conn(0x7fbcf415f150 msgr2=0x7fbcf40cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 --2- 172.21.15.55:0/2731634570 >> v2:172.21.15.138:3300/0 conn(0x7fbcf415f150 0x7fbcf40cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 >> v1:172.21.15.138:6789/0 conn(0x7fbcf40d32a0 legacy=0x7fbcf40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf40d4020 con 0x7fbcf406b500 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fbcec0019f0 con 0x7fbcf406b500 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 >> v1:172.21.15.55:6789/0 conn(0x7fbcf406b500 legacy=0x7fbcf406b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcdc0039b0 0x7fbcdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcdc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcdc005150 0x7fbcdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf40d3d40 con 0x7fbcdc004800 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc00c0c0 con 0x7fbcdc0039b0 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbcdc00c2c0 con 0x7fbcdc005150 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf37fe640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcdc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf3fff640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcdc0039b0 0x7fbcdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf88d4640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcdc005150 0x7fbcdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf37fe640 1 -- 172.21.15.55:0/2731634570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcdc005150 msgr2=0x7fbcdc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf37fe640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcdc005150 0x7fbcdc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf37fe640 1 -- 172.21.15.55:0/2731634570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcdc0039b0 msgr2=0x7fbcdc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf37fe640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcdc0039b0 0x7fbcdc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf37fe640 1 -- 172.21.15.55:0/2731634570 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc003520 con 0x7fbcdc004800 2024-05-20T22:08:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.809+0000 7fbcf88d4640 1 -- 172.21.15.55:0/2731634570 reap_dead start 2024-05-20T22:08:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.810+0000 7fbcf37fe640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcdc004c10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009ac0 tx=0x7fbcdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57772a5ec3223cc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.810+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8036070 con 0x7fbcdc004800 2024-05-20T22:08:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.810+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce8031400 con 0x7fbcdc004800 2024-05-20T22:08:54.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.810+0000 7fbcf2ffd640 1 -- 172.21.15.55:0/2731634570 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8009690 con 0x7fbcdc004800 2024-05-20T22:08:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.811+0000 7fbcfa337640 1 -- 172.21.15.55:0/2731634570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 msgr2=0x7fbcdc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.811+0000 7fbcfa337640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcdc004c10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009ac0 tx=0x7fbcdc005830 comp rx=0 tx=0).stop 2024-05-20T22:08:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.811+0000 7fbcfa337640 1 -- 172.21.15.55:0/2731634570 shutdown_connections 2024-05-20T22:08:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.811+0000 7fbcfa337640 1 --2- 172.21.15.55:0/2731634570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcdc004c10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.811+0000 7fbcfa337640 1 -- 172.21.15.55:0/2731634570 >> 172.21.15.55:0/2731634570 conn(0x7fbcf40b6de0 msgr2=0x7fbcf40d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.811+0000 7fbcfa337640 1 -- 172.21.15.55:0/2731634570 shutdown_connections 2024-05-20T22:08:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.811+0000 7fbcfa337640 1 -- 172.21.15.55:0/2731634570 wait complete. 2024-05-20T22:08:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.813+0000 7fbcfa337640 1 Processor -- start 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.813+0000 7fbcfa337640 1 -- start start 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.814+0000 7fbcfa337640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcf40cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.814+0000 7fbcfa337640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcf4153100 0x7fbcf41f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.814+0000 7fbcfa337640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcf4153510 0x7fbcf41f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.814+0000 7fbcfa337640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf40d4820 con 0x7fbcdc004800 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.814+0000 7fbcfa337640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf415f560 con 0x7fbcf4153510 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.814+0000 7fbcfa337640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf415f980 con 0x7fbcf4153100 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.814+0000 7fbcf3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcf40cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.814+0000 7fbcf3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcf40cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60406/0 (socket says 172.21.15.55:60406) 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.814+0000 7fbcf3fff640 1 -- 172.21.15.55:0/2350858494 learned_addr learned my addr 172.21.15.55:0/2350858494 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf88d4640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcf4153510 0x7fbcf41f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf37fe640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcf4153100 0x7fbcf41f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf3fff640 1 -- 172.21.15.55:0/2350858494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcf4153100 msgr2=0x7fbcf41f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf3fff640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcf4153100 0x7fbcf41f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf3fff640 1 -- 172.21.15.55:0/2350858494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcf4153510 msgr2=0x7fbcf41f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf3fff640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcf4153510 0x7fbcf41f92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf3fff640 1 -- 172.21.15.55:0/2350858494 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf415f770 con 0x7fbcdc004800 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf3fff640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcf40cb180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbce400f730 tx=0x7fbce400fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf0ff9640 1 -- 172.21.15.55:0/2350858494 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce401a070 con 0x7fbcdc004800 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf0ff9640 1 -- 172.21.15.55:0/2350858494 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce400fe40 con 0x7fbcdc004800 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcfa337640 1 -- 172.21.15.55:0/2350858494 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf40cd490 con 0x7fbcdc004800 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcf0ff9640 1 -- 172.21.15.55:0/2350858494 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce4007050 con 0x7fbcdc004800 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.815+0000 7fbcfa337640 1 -- 172.21.15.55:0/2350858494 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf4155250 con 0x7fbcdc004800 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.816+0000 7fbcfa337640 1 -- 172.21.15.55:0/2350858494 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf40b7930 con 0x7fbcdc004800 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.817+0000 7fbcf0ff9640 1 -- 172.21.15.55:0/2350858494 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbce401d070 con 0x7fbcdc004800 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.817+0000 7fbcf0ff9640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbce006be60 0x7fbce006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.817+0000 7fbcf0ff9640 1 -- 172.21.15.55:0/2350858494 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbce408aa00 con 0x7fbcdc004800 2024-05-20T22:08:54.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.817+0000 7fbcf37fe640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbce006be60 0x7fbce006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:54.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.818+0000 7fbcf37fe640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbce006be60 0x7fbce006e370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbce8033000 tx=0x7fbce8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:54.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:54.822+0000 7fbcf0ff9640 1 -- 172.21.15.55:0/2350858494 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbcf40b7930 con 0x7fbcdc004800 2024-05-20T22:08:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.017+0000 7fbcfa337640 1 -- 172.21.15.55:0/2350858494 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbcf4155040 con 0x7fbcdc004800 2024-05-20T22:08:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.017+0000 7fbcf0ff9640 1 -- 172.21.15.55:0/2350858494 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fbce4058f20 con 0x7fbcdc004800 2024-05-20T22:08:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-20T22:08:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 -- 172.21.15.55:0/2350858494 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbce006be60 msgr2=0x7fbce006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbce006be60 0x7fbce006e370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbce8033000 tx=0x7fbce8008030 comp rx=0 tx=0).stop 2024-05-20T22:08:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 -- 172.21.15.55:0/2350858494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 msgr2=0x7fbcf40cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcf40cb180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbce400f730 tx=0x7fbce400fbf0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 -- 172.21.15.55:0/2350858494 shutdown_connections 2024-05-20T22:08:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbce006be60 0x7fbce006e370 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcf4153510 0x7fbcf41f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcf4153100 0x7fbcf41f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 --2- 172.21.15.55:0/2350858494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbcdc004800 0x7fbcf40cb180 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 -- 172.21.15.55:0/2350858494 >> 172.21.15.55:0/2350858494 conn(0x7fbcf40b6de0 msgr2=0x7fbcf40ba0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 -- 172.21.15.55:0/2350858494 shutdown_connections 2024-05-20T22:08:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.019+0000 7fbcfa337640 1 -- 172.21.15.55:0/2350858494 wait complete. 2024-05-20T22:08:55.033 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi055.front.sepia.ceph.com... 2024-05-20T22:08:55.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:55.033 DEBUG:teuthology.orchestra.run.smithi055:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-20T22:08:55.058 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:08:55.059 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:08:55.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:55.059 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-20T22:08:55.116 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:08:55.116 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:55.136 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:08:55.136 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:55.196 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:55.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.308+0000 7f53e8ce9640 1 Processor -- start 2024-05-20T22:08:55.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.309+0000 7f53e8ce9640 1 -- start start 2024-05-20T22:08:55.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.310+0000 7f53e8ce9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f53e40b8190 0x7f53e406d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.310+0000 7f53e8ce9640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f53e4163ce0 0x7f53e40ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.310+0000 7f53e8ce9640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e406bf60 con 0x7f53e406d5c0 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.310+0000 7f53e8ce9640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f53e409bca0 con 0x7f53e4163ce0 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.310+0000 7f53e8ce9640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f53e40d1d20 con 0x7f53e40b8190 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.310+0000 7f53e2d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f53e4163ce0 0x7f53e40ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.310+0000 7f53e1d9b640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f53e406d5c0 0x7f53e41635d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53346/0 (socket says 172.21.15.55:53346) 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.310+0000 7f53e1d9b640 1 -- 172.21.15.55:0/2359751679 learned_addr learned my addr 172.21.15.55:0/2359751679 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.310+0000 7f53e2d9d640 1 --2- 172.21.15.55:0/2359751679 >> v2:172.21.15.55:3300/0 conn(0x7f53e4163ce0 0x7f53e40ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60416/0 (socket says 172.21.15.55:60416) 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.311+0000 7f53e259c640 1 --2- 172.21.15.55:0/2359751679 >> v2:172.21.15.138:3300/0 conn(0x7f53e40b8190 0x7f53e406d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.311+0000 7f53e2d9d640 1 -- 172.21.15.55:0/2359751679 >> v2:172.21.15.138:3300/0 conn(0x7f53e40b8190 msgr2=0x7f53e406d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.311+0000 7f53e2d9d640 1 --2- 172.21.15.55:0/2359751679 >> v2:172.21.15.138:3300/0 conn(0x7f53e40b8190 0x7f53e406d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.311+0000 7f53e2d9d640 1 -- 172.21.15.55:0/2359751679 >> v1:172.21.15.55:6789/0 conn(0x7f53e406d5c0 legacy=0x7f53e41635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.311+0000 7f53e2d9d640 1 -- 172.21.15.55:0/2359751679 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e409c130 con 0x7f53e4163ce0 2024-05-20T22:08:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.311+0000 7f53e159a640 1 -- 172.21.15.55:0/2359751679 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 110004062 0 0) 0x7f53e406bf60 con 0x7f53e406d5c0 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.311+0000 7f53e2d9d640 1 --2- 172.21.15.55:0/2359751679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e4163ce0 0x7f53e40ccf60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f53e408b8d0 tx=0x7f53dc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2877d35183c676de server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.312+0000 7f53e159a640 1 -- 172.21.15.55:0/2359751679 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc007740 con 0x7f53e4163ce0 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.312+0000 7f53e159a640 1 -- 172.21.15.55:0/2359751679 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc0078e0 con 0x7f53e4163ce0 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.312+0000 7f53e159a640 1 -- 172.21.15.55:0/2359751679 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc007cb0 con 0x7f53e4163ce0 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.312+0000 7f53e8ce9640 1 -- 172.21.15.55:0/2359751679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e4163ce0 msgr2=0x7f53e40ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.312+0000 7f53e8ce9640 1 --2- 172.21.15.55:0/2359751679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e4163ce0 0x7f53e40ccf60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f53e408b8d0 tx=0x7f53dc02a680 comp rx=0 tx=0).stop 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.312+0000 7f53e8ce9640 1 -- 172.21.15.55:0/2359751679 shutdown_connections 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.312+0000 7f53e8ce9640 1 --2- 172.21.15.55:0/2359751679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e4163ce0 0x7f53e40ccf60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.312+0000 7f53e8ce9640 1 --2- 172.21.15.55:0/2359751679 >> v2:172.21.15.138:3300/0 conn(0x7f53e40b8190 0x7f53e406d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.312+0000 7f53e8ce9640 1 -- 172.21.15.55:0/2359751679 >> 172.21.15.55:0/2359751679 conn(0x7f53e40cf390 msgr2=0x7f53e40cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.313+0000 7f53e8ce9640 1 -- 172.21.15.55:0/2359751679 shutdown_connections 2024-05-20T22:08:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.313+0000 7f53e8ce9640 1 -- 172.21.15.55:0/2359751679 wait complete. 2024-05-20T22:08:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.314+0000 7f53e8ce9640 1 Processor -- start 2024-05-20T22:08:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.315+0000 7f53e8ce9640 1 -- start start 2024-05-20T22:08:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e8ce9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e406d5c0 0x7f53e41f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e8ce9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40b8190 0x7f53e409f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e8ce9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e4163ce0 0x7f53e40b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e8ce9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f53e40b1bb0 con 0x7f53e40b8190 2024-05-20T22:08:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e8ce9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53e40d22a0 con 0x7f53e4163ce0 2024-05-20T22:08:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e8ce9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f53e40b1d30 con 0x7f53e406d5c0 2024-05-20T22:08:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40b8190 0x7f53e409f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40b8190 0x7f53e409f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60422/0 (socket says 172.21.15.55:60422) 2024-05-20T22:08:55.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e1d9b640 1 -- 172.21.15.55:0/809570822 learned_addr learned my addr 172.21.15.55:0/809570822 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:55.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e259c640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e406d5c0 0x7f53e41f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e2d9d640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e4163ce0 0x7f53e40b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e1d9b640 1 -- 172.21.15.55:0/809570822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e406d5c0 msgr2=0x7f53e41f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e1d9b640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e406d5c0 0x7f53e41f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e1d9b640 1 -- 172.21.15.55:0/809570822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e4163ce0 msgr2=0x7f53e40b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e1d9b640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e4163ce0 0x7f53e40b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e1d9b640 1 -- 172.21.15.55:0/809570822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e41578d0 con 0x7f53e40b8190 2024-05-20T22:08:55.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.316+0000 7f53e1d9b640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40b8190 0x7f53e409f800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c410 tx=0x7f53d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:55.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.317+0000 7f53d2ffd640 1 -- 172.21.15.55:0/809570822 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d80074e0 con 0x7f53e40b8190 2024-05-20T22:08:55.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.317+0000 7f53d2ffd640 1 -- 172.21.15.55:0/809570822 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d80047e0 con 0x7f53e40b8190 2024-05-20T22:08:55.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.317+0000 7f53e8ce9640 1 -- 172.21.15.55:0/809570822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e409dbd0 con 0x7f53e40b8190 2024-05-20T22:08:55.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.317+0000 7f53d2ffd640 1 -- 172.21.15.55:0/809570822 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8004bd0 con 0x7f53e40b8190 2024-05-20T22:08:55.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.317+0000 7f53e8ce9640 1 -- 172.21.15.55:0/809570822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e409e190 con 0x7f53e40b8190 2024-05-20T22:08:55.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.318+0000 7f53e8ce9640 1 -- 172.21.15.55:0/809570822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e4069090 con 0x7f53e40b8190 2024-05-20T22:08:55.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.318+0000 7f53d2ffd640 1 -- 172.21.15.55:0/809570822 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53d8011070 con 0x7f53e40b8190 2024-05-20T22:08:55.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.319+0000 7f53d2ffd640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53c406be30 0x7f53c406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.319+0000 7f53d2ffd640 1 -- 172.21.15.55:0/809570822 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f53d808af20 con 0x7f53e40b8190 2024-05-20T22:08:55.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.319+0000 7f53e259c640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53c406be30 0x7f53c406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.319+0000 7f53e259c640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53c406be30 0x7f53c406e340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f53e40d0b00 tx=0x7f53d4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:55.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.324+0000 7f53d2ffd640 1 -- 172.21.15.55:0/809570822 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f53d800e280 con 0x7f53e40b8190 2024-05-20T22:08:55.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.509+0000 7f53e8ce9640 1 -- 172.21.15.55:0/809570822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53e40b2050 con 0x7f53e40b8190 2024-05-20T22:08:55.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.510+0000 7f53d2ffd640 1 -- 172.21.15.55:0/809570822 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7f53d8059440 con 0x7f53e40b8190 2024-05-20T22:08:55.512 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:55.513 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 -- 172.21.15.55:0/809570822 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53c406be30 msgr2=0x7f53c406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53c406be30 0x7f53c406e340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f53e40d0b00 tx=0x7f53d4008040 comp rx=0 tx=0).stop 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 -- 172.21.15.55:0/809570822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40b8190 msgr2=0x7f53e409f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40b8190 0x7f53e409f800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c410 tx=0x7f53d800c8d0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 -- 172.21.15.55:0/809570822 shutdown_connections 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53c406be30 0x7f53c406e340 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e4163ce0 0x7f53e40b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40b8190 0x7f53e409f800 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 --2- 172.21.15.55:0/809570822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e406d5c0 0x7f53e41f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 -- 172.21.15.55:0/809570822 >> 172.21.15.55:0/809570822 conn(0x7f53e40cf390 msgr2=0x7f53e40708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 -- 172.21.15.55:0/809570822 shutdown_connections 2024-05-20T22:08:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.512+0000 7f53e8ce9640 1 -- 172.21.15.55:0/809570822 wait complete. 2024-05-20T22:08:55.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:55.527 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.643+0000 7f53fe97e640 1 Processor -- start 2024-05-20T22:08:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.643+0000 7f53fe97e640 1 -- start start 2024-05-20T22:08:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53fe97e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f53f8167ff0 0x7f53f816a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53fe97e640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f80d04b0 con 0x7f53f815f150 2024-05-20T22:08:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53fe97e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f80d3780 con 0x7f53f80d32a0 2024-05-20T22:08:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53fe97e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f53f806c510 con 0x7f53f8167ff0 2024-05-20T22:08:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53effff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f53f815f150 0x7f53f80cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53360/0 (socket says 172.21.15.55:53360) 2024-05-20T22:08:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53effff640 1 -- 172.21.15.55:0/1669424722 learned_addr learned my addr 172.21.15.55:0/1669424722 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53fcf1b640 1 --2- 172.21.15.55:0/1669424722 >> v2:172.21.15.138:3300/0 conn(0x7f53f8167ff0 0x7f53f816a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3047059162 0 0) 0x7f53f80d04b0 con 0x7f53f815f150 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53e4003520 con 0x7f53f815f150 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 422512827 0 0) 0x7f53f80d3780 con 0x7f53f80d32a0 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53f80d04b0 con 0x7f53f80d32a0 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 80476190 0 0) 0x7f53e4003520 con 0x7f53f815f150 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 >> v2:172.21.15.138:3300/0 conn(0x7f53f8167ff0 msgr2=0x7f53f816a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53f77fe640 1 --2- 172.21.15.55:0/1669424722 >> v2:172.21.15.138:3300/0 conn(0x7f53f8167ff0 0x7f53f816a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.645+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 >> v1:172.21.15.138:6789/0 conn(0x7f53f80d32a0 legacy=0x7f53f80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f80d3a60 con 0x7f53f815f150 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f53e0001670 con 0x7f53f815f150 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 >> v1:172.21.15.55:6789/0 conn(0x7f53f815f150 legacy=0x7f53f80cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53f77fe640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e4003b40 0x7f53e4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53f77fe640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e4004800 0x7f53e4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53f77fe640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40051a0 0x7f53e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f53f806c510 con 0x7f53e40051a0 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53e400c140 con 0x7f53e4003b40 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f53e400c340 con 0x7f53e4004800 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53fcf1b640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40051a0 0x7f53e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53f7fff640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e4003b40 0x7f53e4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53effff640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e4004800 0x7f53e4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53fcf1b640 1 -- 172.21.15.55:0/1669424722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e4004800 msgr2=0x7f53e4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53fcf1b640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e4004800 0x7f53e4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53fcf1b640 1 -- 172.21.15.55:0/1669424722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e4003b40 msgr2=0x7f53e4003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53fcf1b640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e4003b40 0x7f53e4003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53fcf1b640 1 -- 172.21.15.55:0/1669424722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e4003520 con 0x7f53e40051a0 2024-05-20T22:08:55.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.646+0000 7f53effff640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e4004800 0x7f53e4004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:55.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.647+0000 7f53fcf1b640 1 -- 172.21.15.55:0/1669424722 reap_dead start 2024-05-20T22:08:55.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.647+0000 7f53fcf1b640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40051a0 0x7f53e400b9c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f53e40057e0 tx=0x7f53e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1894e0f14d90f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:55.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.647+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0009c10 con 0x7f53e40051a0 2024-05-20T22:08:55.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.647+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f0003b90 con 0x7f53e40051a0 2024-05-20T22:08:55.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.647+0000 7f53f77fe640 1 -- 172.21.15.55:0/1669424722 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0004020 con 0x7f53e40051a0 2024-05-20T22:08:55.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.648+0000 7f53fe97e640 1 -- 172.21.15.55:0/1669424722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40051a0 msgr2=0x7f53e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.648+0000 7f53fe97e640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40051a0 0x7f53e400b9c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f53e40057e0 tx=0x7f53e4005830 comp rx=0 tx=0).stop 2024-05-20T22:08:55.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.648+0000 7f53fe97e640 1 -- 172.21.15.55:0/1669424722 shutdown_connections 2024-05-20T22:08:55.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.648+0000 7f53fe97e640 1 --2- 172.21.15.55:0/1669424722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53e40051a0 0x7f53e400b9c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.648+0000 7f53fe97e640 1 -- 172.21.15.55:0/1669424722 >> 172.21.15.55:0/1669424722 conn(0x7f53f80b6de0 msgr2=0x7f53f80d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:55.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.648+0000 7f53fe97e640 1 -- 172.21.15.55:0/1669424722 shutdown_connections 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.648+0000 7f53fe97e640 1 -- 172.21.15.55:0/1669424722 wait complete. 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.650+0000 7f53fe97e640 1 Processor -- start 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.650+0000 7f53fe97e640 1 -- start start 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53fe97e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e40051a0 0x7f53f80cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53fe97e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f8151f40 0x7f53f80b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53fe97e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f80b9010 0x7f53f80b9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53fe97e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f53f80d4300 con 0x7f53f8151f40 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53fe97e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53f80d4020 con 0x7f53f80b9010 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53fe97e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f53f815f560 con 0x7f53e40051a0 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53effff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f8151f40 0x7f53f80b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53effff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f8151f40 0x7f53f80b8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60446/0 (socket says 172.21.15.55:60446) 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53effff640 1 -- 172.21.15.55:0/2987387179 learned_addr learned my addr 172.21.15.55:0/2987387179 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53f7fff640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e40051a0 0x7f53f80cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53fcf1b640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f80b9010 0x7f53f80b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53effff640 1 -- 172.21.15.55:0/2987387179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e40051a0 msgr2=0x7f53f80cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53effff640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e40051a0 0x7f53f80cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53effff640 1 -- 172.21.15.55:0/2987387179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f80b9010 msgr2=0x7f53f80b9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53effff640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f80b9010 0x7f53f80b9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53effff640 1 -- 172.21.15.55:0/2987387179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f80b9960 con 0x7f53f8151f40 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.651+0000 7f53fcf1b640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f80b9010 0x7f53f80b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.652+0000 7f53effff640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f8151f40 0x7f53f80b8ad0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e780 tx=0x7f53e000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.652+0000 7f53f57fa640 1 -- 172.21.15.55:0/2987387179 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e000c660 con 0x7f53f8151f40 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.652+0000 7f53f57fa640 1 -- 172.21.15.55:0/2987387179 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e0018070 con 0x7f53f8151f40 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.652+0000 7f53f57fa640 1 -- 172.21.15.55:0/2987387179 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e000ca00 con 0x7f53f8151f40 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.652+0000 7f53fe97e640 1 -- 172.21.15.55:0/2987387179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f80b9e10 con 0x7f53f8151f40 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.652+0000 7f53fe97e640 1 -- 172.21.15.55:0/2987387179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f80ce4c0 con 0x7f53f8151f40 2024-05-20T22:08:55.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.653+0000 7f53fe97e640 1 -- 172.21.15.55:0/2987387179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f80b49d0 con 0x7f53f8151f40 2024-05-20T22:08:55.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.653+0000 7f53f57fa640 1 -- 172.21.15.55:0/2987387179 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53e000cba0 con 0x7f53f8151f40 2024-05-20T22:08:55.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.654+0000 7f53f57fa640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53d806be30 0x7f53d806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.654+0000 7f53f57fa640 1 -- 172.21.15.55:0/2987387179 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f53e008b0b0 con 0x7f53f8151f40 2024-05-20T22:08:55.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.654+0000 7f53f7fff640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53d806be30 0x7f53d806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.655+0000 7f53f7fff640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53d806be30 0x7f53d806e340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53e8005530 tx=0x7f53e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:55.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.659+0000 7f53f57fa640 1 -- 172.21.15.55:0/2987387179 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f53f80b49d0 con 0x7f53f8151f40 2024-05-20T22:08:55.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.842+0000 7f53fe97e640 1 -- 172.21.15.55:0/2987387179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f53f8069090 con 0x7f53f8151f40 2024-05-20T22:08:55.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.843+0000 7f53f57fa640 1 -- 172.21.15.55:0/2987387179 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f53e00595d0 con 0x7f53f8151f40 2024-05-20T22:08:55.845 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:55.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 -- 172.21.15.55:0/2987387179 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53d806be30 msgr2=0x7f53d806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53d806be30 0x7f53d806e340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53e8005530 tx=0x7f53e8009040 comp rx=0 tx=0).stop 2024-05-20T22:08:55.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 -- 172.21.15.55:0/2987387179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f8151f40 msgr2=0x7f53f80b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f8151f40 0x7f53f80b8ad0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e780 tx=0x7f53e000ec40 comp rx=0 tx=0).stop 2024-05-20T22:08:55.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 -- 172.21.15.55:0/2987387179 shutdown_connections 2024-05-20T22:08:55.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f53d806be30 0x7f53d806e340 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53f80b9010 0x7f53f80b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f8151f40 0x7f53f80b8ad0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 --2- 172.21.15.55:0/2987387179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f53e40051a0 0x7f53f80cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 -- 172.21.15.55:0/2987387179 >> 172.21.15.55:0/2987387179 conn(0x7f53f80b6de0 msgr2=0x7f53f806c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 -- 172.21.15.55:0/2987387179 shutdown_connections 2024-05-20T22:08:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.845+0000 7f53fe97e640 1 -- 172.21.15.55:0/2987387179 wait complete. 2024-05-20T22:08:55.858 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:55.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.975+0000 7fa579338640 1 Processor -- start 2024-05-20T22:08:55.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.975+0000 7fa579338640 1 -- start start 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa579338640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5740ce230 0x7fa5740b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa579338640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf3f0 0x7fa57409a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa579338640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5740b3180 con 0x7fa5740cf800 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa579338640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa5740ce730 con 0x7fa5740ce230 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa579338640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa5740cea10 con 0x7fa5740cf3f0 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa572ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5740ce230 0x7fa5740b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa56a7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf3f0 0x7fa57409a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa56a7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf3f0 0x7fa57409a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:34336/0 (socket says 172.21.15.55:34336) 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa572ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5740ce230 0x7fa5740b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:49618/0 (socket says 172.21.15.55:49618) 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa56a7fc640 1 -- 172.21.15.55:0/1629163760 learned_addr learned my addr 172.21.15.55:0/1629163760 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:55.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.976+0000 7fa5737fe640 1 --1- 172.21.15.55:0/1629163760 >> v1:172.21.15.138:6789/0 conn(0x7fa5740cf800 0x7fa57415fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:52286/0 (socket says 172.21.15.55:52286) 2024-05-20T22:08:55.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.977+0000 7fa56a7fc640 1 -- 172.21.15.55:0/1629163760 >> v2:172.21.15.138:3300/0 conn(0x7fa5740ce230 msgr2=0x7fa5740b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:55.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.977+0000 7fa56a7fc640 1 --2- 172.21.15.55:0/1629163760 >> v2:172.21.15.138:3300/0 conn(0x7fa5740ce230 0x7fa5740b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.977+0000 7fa56a7fc640 1 -- 172.21.15.55:0/1629163760 >> v1:172.21.15.138:6789/0 conn(0x7fa5740cf800 legacy=0x7fa57415fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.977+0000 7fa56a7fc640 1 -- 172.21.15.55:0/1629163760 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5740cecf0 con 0x7fa5740cf3f0 2024-05-20T22:08:55.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.977+0000 7fa572ffd640 1 --2- 172.21.15.55:0/1629163760 >> v2:172.21.15.138:3300/0 conn(0x7fa5740ce230 0x7fa5740b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:55.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.978+0000 7fa56a7fc640 1 --2- 172.21.15.55:0/1629163760 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf3f0 0x7fa57409a6c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009870 tx=0x7fa55c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78281210bcd5af69 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:55.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.978+0000 7fa5727fc640 1 -- 172.21.15.55:0/1629163760 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c037070 con 0x7fa5740cf3f0 2024-05-20T22:08:55.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.978+0000 7fa5727fc640 1 -- 172.21.15.55:0/1629163760 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c032440 con 0x7fa5740cf3f0 2024-05-20T22:08:55.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.978+0000 7fa5727fc640 1 -- 172.21.15.55:0/1629163760 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c032810 con 0x7fa5740cf3f0 2024-05-20T22:08:55.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.978+0000 7fa579338640 1 -- 172.21.15.55:0/1629163760 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf3f0 msgr2=0x7fa57409a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.978+0000 7fa579338640 1 --2- 172.21.15.55:0/1629163760 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf3f0 0x7fa57409a6c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009870 tx=0x7fa55c02a250 comp rx=0 tx=0).stop 2024-05-20T22:08:55.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.979+0000 7fa579338640 1 -- 172.21.15.55:0/1629163760 shutdown_connections 2024-05-20T22:08:55.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.979+0000 7fa579338640 1 --2- 172.21.15.55:0/1629163760 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf3f0 0x7fa57409a6c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.979+0000 7fa579338640 1 --2- 172.21.15.55:0/1629163760 >> v2:172.21.15.138:3300/0 conn(0x7fa5740ce230 0x7fa5740b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.979+0000 7fa579338640 1 -- 172.21.15.55:0/1629163760 >> 172.21.15.55:0/1629163760 conn(0x7fa5740737c0 msgr2=0x7fa5740b6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:55.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.979+0000 7fa579338640 1 -- 172.21.15.55:0/1629163760 shutdown_connections 2024-05-20T22:08:55.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.979+0000 7fa579338640 1 -- 172.21.15.55:0/1629163760 wait complete. 2024-05-20T22:08:55.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.980+0000 7fa579338640 1 Processor -- start 2024-05-20T22:08:55.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.981+0000 7fa579338640 1 -- start start 2024-05-20T22:08:55.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa579338640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5740ce230 0x7fa57414bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa579338640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5740cf3f0 0x7fa5741f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa579338640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf800 0x7fa57406bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa579338640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa57415c200 con 0x7fa5740cf3f0 2024-05-20T22:08:55.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa579338640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa57415f950 con 0x7fa5740ce230 2024-05-20T22:08:55.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa579338640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa57406e7f0 con 0x7fa5740cf800 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa56a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5740cf3f0 0x7fa5741f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa56a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5740cf3f0 0x7fa5741f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60456/0 (socket says 172.21.15.55:60456) 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa56a7fc640 1 -- 172.21.15.55:0/584246862 learned_addr learned my addr 172.21.15.55:0/584246862 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa572ffd640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5740ce230 0x7fa57414bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa5737fe640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf800 0x7fa57406bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa56a7fc640 1 -- 172.21.15.55:0/584246862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf800 msgr2=0x7fa57406bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa56a7fc640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf800 0x7fa57406bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa56a7fc640 1 -- 172.21.15.55:0/584246862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5740ce230 msgr2=0x7fa57414bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa56a7fc640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5740ce230 0x7fa57414bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.982+0000 7fa56a7fc640 1 -- 172.21.15.55:0/584246862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa574157a00 con 0x7fa5740cf3f0 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.983+0000 7fa56a7fc640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5740cf3f0 0x7fa5741f52c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009870 tx=0x7fa55c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:55.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.983+0000 7fa56bfff640 1 -- 172.21.15.55:0/584246862 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c037040 con 0x7fa5740cf3f0 2024-05-20T22:08:55.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.983+0000 7fa56bfff640 1 -- 172.21.15.55:0/584246862 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c042070 con 0x7fa5740cf3f0 2024-05-20T22:08:55.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.983+0000 7fa56bfff640 1 -- 172.21.15.55:0/584246862 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c030880 con 0x7fa5740cf3f0 2024-05-20T22:08:55.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.983+0000 7fa579338640 1 -- 172.21.15.55:0/584246862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa574157cc0 con 0x7fa5740cf3f0 2024-05-20T22:08:55.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.983+0000 7fa579338640 1 -- 172.21.15.55:0/584246862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa574158280 con 0x7fa5740cf3f0 2024-05-20T22:08:55.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.984+0000 7fa579338640 1 -- 172.21.15.55:0/584246862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5741558f0 con 0x7fa5740cf3f0 2024-05-20T22:08:55.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.984+0000 7fa56bfff640 1 -- 172.21.15.55:0/584246862 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa55c030a20 con 0x7fa5740cf3f0 2024-05-20T22:08:55.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.985+0000 7fa56bfff640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa55406be60 0x7fa55406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:55.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.985+0000 7fa56bfff640 1 -- 172.21.15.55:0/584246862 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa55c0ab930 con 0x7fa5740cf3f0 2024-05-20T22:08:55.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.985+0000 7fa572ffd640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa55406be60 0x7fa55406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:55.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.985+0000 7fa572ffd640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa55406be60 0x7fa55406e370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa564009560 tx=0x7fa564009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:55.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:55.990+0000 7fa56bfff640 1 -- 172.21.15.55:0/584246862 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa5741558f0 con 0x7fa5740cf3f0 2024-05-20T22:08:56.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.172+0000 7fa579338640 1 -- 172.21.15.55:0/584246862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa574069090 con 0x7fa5740cf3f0 2024-05-20T22:08:56.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.173+0000 7fa56bfff640 1 -- 172.21.15.55:0/584246862 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fa55c079800 con 0x7fa5740cf3f0 2024-05-20T22:08:56.176 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:56.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 -- 172.21.15.55:0/584246862 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa55406be60 msgr2=0x7fa55406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa55406be60 0x7fa55406e370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa564009560 tx=0x7fa564009190 comp rx=0 tx=0).stop 2024-05-20T22:08:56.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 -- 172.21.15.55:0/584246862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5740cf3f0 msgr2=0x7fa5741f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5740cf3f0 0x7fa5741f52c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009870 tx=0x7fa55c0046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 -- 172.21.15.55:0/584246862 shutdown_connections 2024-05-20T22:08:56.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa55406be60 0x7fa55406e370 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5740cf800 0x7fa57406bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5740cf3f0 0x7fa5741f52c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 --2- 172.21.15.55:0/584246862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5740ce230 0x7fa57414bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 -- 172.21.15.55:0/584246862 >> 172.21.15.55:0/584246862 conn(0x7fa5740737c0 msgr2=0x7fa57406cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:56.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 -- 172.21.15.55:0/584246862 shutdown_connections 2024-05-20T22:08:56.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.175+0000 7fa579338640 1 -- 172.21.15.55:0/584246862 wait complete. 2024-05-20T22:08:56.188 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:56.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.303+0000 7febd3537640 1 Processor -- start 2024-05-20T22:08:56.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.303+0000 7febd3537640 1 -- start start 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febd3537640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc15e550 0x7febcc15e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febd3537640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7febcc06bfe0 0x7febcc06c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febd3537640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febcc157870 con 0x7febcc160a30 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febd3537640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7febcc06c930 con 0x7febcc06bfe0 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febd3537640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7febcc073310 con 0x7febcc15e550 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febd1ad4640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7febcc160a30 0x7febcc160e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53374/0 (socket says 172.21.15.55:53374) 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febd1ad4640 1 -- 172.21.15.55:0/1635767590 learned_addr learned my addr 172.21.15.55:0/1635767590 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febd12d3640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc15e550 0x7febcc15e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febd0ad2640 1 --2- 172.21.15.55:0/1635767590 >> v2:172.21.15.138:3300/0 conn(0x7febcc06bfe0 0x7febcc06c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15043893 0 0) 0x7febcc157870 con 0x7febcc160a30 2024-05-20T22:08:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.305+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febb4003520 con 0x7febcc160a30 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2960303740 0 0) 0x7febb4003520 con 0x7febcc160a30 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc15e550 msgr2=0x7febcc15e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc15e550 0x7febcc15e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 >> v2:172.21.15.138:3300/0 conn(0x7febcc06bfe0 msgr2=0x7febcc06c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 --2- 172.21.15.55:0/1635767590 >> v2:172.21.15.138:3300/0 conn(0x7febcc06bfe0 0x7febcc06c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febcc0735f0 con 0x7febcc160a30 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7febc4001690 con 0x7febcc160a30 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 >> v1:172.21.15.55:6789/0 conn(0x7febcc160a30 legacy=0x7febcc160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febb40039b0 0x7febb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 0x7febb4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc15e550 0x7febb4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7febcc06c930 con 0x7febb4004800 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febcc073310 con 0x7febb40039b0 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7febb400b9c0 con 0x7febcc15e550 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febd0ad2640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 0x7febb4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febd12d3640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febb40039b0 0x7febb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febd1ad4640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc15e550 0x7febb4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febd0ad2640 1 -- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc15e550 msgr2=0x7febb4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febd0ad2640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc15e550 0x7febb4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febd0ad2640 1 -- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febb40039b0 msgr2=0x7febb4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febd0ad2640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febb40039b0 0x7febb4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.306+0000 7febd0ad2640 1 -- 172.21.15.55:0/1635767590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb4003520 con 0x7febb4004800 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.307+0000 7febd0ad2640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 0x7febb4004c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7febc0009ac0 tx=0x7febb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79f28e785f4198d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.307+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0036070 con 0x7febb4004800 2024-05-20T22:08:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.307+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc0031400 con 0x7febb4004800 2024-05-20T22:08:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.307+0000 7febcbfff640 1 -- 172.21.15.55:0/1635767590 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0009690 con 0x7febb4004800 2024-05-20T22:08:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.308+0000 7febd3537640 1 -- 172.21.15.55:0/1635767590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 msgr2=0x7febb4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.308+0000 7febd3537640 1 --2- 172.21.15.55:0/1635767590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 0x7febb4004c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7febc0009ac0 tx=0x7febb4005830 comp rx=0 tx=0).stop 2024-05-20T22:08:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.308+0000 7febd1ad4640 1 -- 172.21.15.55:0/1635767590 reap_dead start 2024-05-20T22:08:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.308+0000 7febd3537640 1 -- 172.21.15.55:0/1635767590 shutdown_connections 2024-05-20T22:08:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.308+0000 7febd3537640 1 -- 172.21.15.55:0/1635767590 >> 172.21.15.55:0/1635767590 conn(0x7febcc0b3980 msgr2=0x7febcc0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.308+0000 7febd3537640 1 -- 172.21.15.55:0/1635767590 shutdown_connections 2024-05-20T22:08:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.308+0000 7febd3537640 1 -- 172.21.15.55:0/1635767590 wait complete. 2024-05-20T22:08:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.310+0000 7febd3537640 1 Processor -- start 2024-05-20T22:08:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.310+0000 7febd3537640 1 -- start start 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd3537640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 0x7febcc13c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd3537640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc148330 0x7febcc0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd3537640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febcc0b08a0 0x7febcc154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd3537640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7febcc06bac0 con 0x7febb4004800 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd3537640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febcc06b7e0 con 0x7febcc0b08a0 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd3537640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7febcc15e960 con 0x7febcc148330 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd12d3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 0x7febcc13c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd12d3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 0x7febcc13c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60472/0 (socket says 172.21.15.55:60472) 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd12d3640 1 -- 172.21.15.55:0/3251119654 learned_addr learned my addr 172.21.15.55:0/3251119654 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd1ad4640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febcc0b08a0 0x7febcc154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.311+0000 7febd0ad2640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc148330 0x7febcc0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febd12d3640 1 -- 172.21.15.55:0/3251119654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc148330 msgr2=0x7febcc0b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febd12d3640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc148330 0x7febcc0b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febd12d3640 1 -- 172.21.15.55:0/3251119654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febcc0b08a0 msgr2=0x7febcc154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febd12d3640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febcc0b08a0 0x7febcc154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febd12d3640 1 -- 172.21.15.55:0/3251119654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febcc0b0cb0 con 0x7febb4004800 2024-05-20T22:08:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febd12d3640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 0x7febcc13c140 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7febbc011d00 tx=0x7febbc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febc9ffb640 1 -- 172.21.15.55:0/3251119654 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc00fea0 con 0x7febb4004800 2024-05-20T22:08:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febc9ffb640 1 -- 172.21.15.55:0/3251119654 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc015630 con 0x7febb4004800 2024-05-20T22:08:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febd3537640 1 -- 172.21.15.55:0/3251119654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febcc13c680 con 0x7febb4004800 2024-05-20T22:08:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febc9ffb640 1 -- 172.21.15.55:0/3251119654 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc015a20 con 0x7febb4004800 2024-05-20T22:08:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.312+0000 7febd3537640 1 -- 172.21.15.55:0/3251119654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febcc0b2d80 con 0x7febb4004800 2024-05-20T22:08:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.313+0000 7febd3537640 1 -- 172.21.15.55:0/3251119654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febcc069090 con 0x7febb4004800 2024-05-20T22:08:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.314+0000 7febc9ffb640 1 -- 172.21.15.55:0/3251119654 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febbc015bc0 con 0x7febb4004800 2024-05-20T22:08:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.315+0000 7febc9ffb640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7febb806be30 0x7febb806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.315+0000 7febc9ffb640 1 -- 172.21.15.55:0/3251119654 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7febbc08c170 con 0x7febb4004800 2024-05-20T22:08:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.315+0000 7febd0ad2640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7febb806be30 0x7febb806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.315+0000 7febd0ad2640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7febb806be30 0x7febb806e340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7febc0033000 tx=0x7febc0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:56.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.319+0000 7febc9ffb640 1 -- 172.21.15.55:0/3251119654 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7febcc0b0cb0 con 0x7febb4004800 2024-05-20T22:08:56.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.502+0000 7febd3537640 1 -- 172.21.15.55:0/3251119654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7febcc14c170 con 0x7febb4004800 2024-05-20T22:08:56.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.502+0000 7febc9ffb640 1 -- 172.21.15.55:0/3251119654 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7febbc019030 con 0x7febb4004800 2024-05-20T22:08:56.504 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:56.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 -- 172.21.15.55:0/3251119654 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7febb806be30 msgr2=0x7febb806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7febb806be30 0x7febb806e340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7febc0033000 tx=0x7febc0008030 comp rx=0 tx=0).stop 2024-05-20T22:08:56.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 -- 172.21.15.55:0/3251119654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 msgr2=0x7febcc13c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 0x7febcc13c140 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7febbc011d00 tx=0x7febbc00f580 comp rx=0 tx=0).stop 2024-05-20T22:08:56.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 -- 172.21.15.55:0/3251119654 shutdown_connections 2024-05-20T22:08:56.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7febb806be30 0x7febb806e340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febcc0b08a0 0x7febcc154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febcc148330 0x7febcc0b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 --2- 172.21.15.55:0/3251119654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febb4004800 0x7febcc13c140 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 -- 172.21.15.55:0/3251119654 >> 172.21.15.55:0/3251119654 conn(0x7febcc0b3980 msgr2=0x7febcc156970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:56.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 -- 172.21.15.55:0/3251119654 shutdown_connections 2024-05-20T22:08:56.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.505+0000 7febd3537640 1 -- 172.21.15.55:0/3251119654 wait complete. 2024-05-20T22:08:56.518 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:56.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.634+0000 7fe63ff2c640 1 Processor -- start 2024-05-20T22:08:56.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.634+0000 7fe63ff2c640 1 -- start start 2024-05-20T22:08:56.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.635+0000 7fe63ff2c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe6380ceb10 0x7fe6380d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.635+0000 7fe63ff2c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe6380d1ad0 0x7fe6380cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.635+0000 7fe63ff2c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6380d1ee0 con 0x7fe63806e0c0 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.635+0000 7fe63ff2c640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fe6380cef20 con 0x7fe6380ceb10 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.635+0000 7fe63ff2c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe63809a260 con 0x7fe6380d1ad0 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.635+0000 7fe63d4c7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe6380ceb10 0x7fe6380d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.635+0000 7fe63d4c7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe6380ceb10 0x7fe6380d1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60482/0 (socket says 172.21.15.55:60482) 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.635+0000 7fe63d4c7640 1 -- 172.21.15.55:0/3120973394 learned_addr learned my addr 172.21.15.55:0/3120973394 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.636+0000 7fe63e4c9640 1 --2- 172.21.15.55:0/3120973394 >> v2:172.21.15.138:3300/0 conn(0x7fe6380d1ad0 0x7fe6380cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.636+0000 7fe63d4c7640 1 -- 172.21.15.55:0/3120973394 >> v2:172.21.15.138:3300/0 conn(0x7fe6380d1ad0 msgr2=0x7fe6380cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.636+0000 7fe63d4c7640 1 --2- 172.21.15.55:0/3120973394 >> v2:172.21.15.138:3300/0 conn(0x7fe6380d1ad0 0x7fe6380cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.636+0000 7fe63d4c7640 1 -- 172.21.15.55:0/3120973394 >> v1:172.21.15.138:6789/0 conn(0x7fe63806e0c0 legacy=0x7fe63815cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:56.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.636+0000 7fe63d4c7640 1 -- 172.21.15.55:0/3120973394 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63809a540 con 0x7fe6380ceb10 2024-05-20T22:08:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.636+0000 7fe63d4c7640 1 --2- 172.21.15.55:0/3120973394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380ceb10 0x7fe6380d1590 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe63809b2e0 tx=0x7fe62802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dd492e786a9e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.637+0000 7fe63ccc6640 1 -- 172.21.15.55:0/3120973394 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628037070 con 0x7fe6380ceb10 2024-05-20T22:08:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.637+0000 7fe63ccc6640 1 -- 172.21.15.55:0/3120973394 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe628032440 con 0x7fe6380ceb10 2024-05-20T22:08:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.637+0000 7fe63ccc6640 1 -- 172.21.15.55:0/3120973394 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628032810 con 0x7fe6380ceb10 2024-05-20T22:08:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.637+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3120973394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380ceb10 msgr2=0x7fe6380d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.637+0000 7fe63ff2c640 1 --2- 172.21.15.55:0/3120973394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380ceb10 0x7fe6380d1590 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe63809b2e0 tx=0x7fe62802a210 comp rx=0 tx=0).stop 2024-05-20T22:08:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.637+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3120973394 shutdown_connections 2024-05-20T22:08:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.637+0000 7fe63ff2c640 1 --2- 172.21.15.55:0/3120973394 >> v2:172.21.15.138:3300/0 conn(0x7fe6380d1ad0 0x7fe6380cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.637+0000 7fe63ff2c640 1 --2- 172.21.15.55:0/3120973394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380ceb10 0x7fe6380d1590 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.637+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3120973394 >> 172.21.15.55:0/3120973394 conn(0x7fe6380736f0 msgr2=0x7fe6380b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.637+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3120973394 shutdown_connections 2024-05-20T22:08:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.638+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3120973394 wait complete. 2024-05-20T22:08:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.639+0000 7fe63ff2c640 1 Processor -- start 2024-05-20T22:08:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.639+0000 7fe63ff2c640 1 -- start start 2024-05-20T22:08:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.640+0000 7fe63ff2c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe63806e0c0 0x7fe638159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63ff2c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6380ceb10 0x7fe6380c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63ff2c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380d1ad0 0x7fe638158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63ff2c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe63815d2a0 con 0x7fe6380d1ad0 2024-05-20T22:08:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63ff2c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe63815c670 con 0x7fe6380ceb10 2024-05-20T22:08:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63ff2c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6380b2fe0 con 0x7fe63806e0c0 2024-05-20T22:08:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63e4c9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380d1ad0 0x7fe638158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63e4c9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380d1ad0 0x7fe638158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60486/0 (socket says 172.21.15.55:60486) 2024-05-20T22:08:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63e4c9640 1 -- 172.21.15.55:0/3535314558 learned_addr learned my addr 172.21.15.55:0/3535314558 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63d4c7640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6380ceb10 0x7fe6380c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63dcc8640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe63806e0c0 0x7fe638159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63e4c9640 1 -- 172.21.15.55:0/3535314558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe63806e0c0 msgr2=0x7fe638159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63e4c9640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe63806e0c0 0x7fe638159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63e4c9640 1 -- 172.21.15.55:0/3535314558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6380ceb10 msgr2=0x7fe6380c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63e4c9640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6380ceb10 0x7fe6380c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63e4c9640 1 -- 172.21.15.55:0/3535314558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6380c5e30 con 0x7fe6380d1ad0 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63e4c9640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380d1ad0 0x7fe638158a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe63000d5a0 tx=0x7fe63000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe62e7fc640 1 -- 172.21.15.55:0/3535314558 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6300125f0 con 0x7fe6380d1ad0 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe62e7fc640 1 -- 172.21.15.55:0/3535314558 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe630019070 con 0x7fe6380d1ad0 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3535314558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6380c6110 con 0x7fe6380d1ad0 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.641+0000 7fe62e7fc640 1 -- 172.21.15.55:0/3535314558 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6300129f0 con 0x7fe6380d1ad0 2024-05-20T22:08:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.642+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3535314558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6380c66d0 con 0x7fe6380d1ad0 2024-05-20T22:08:56.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.643+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3535314558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe638069090 con 0x7fe6380d1ad0 2024-05-20T22:08:56.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.643+0000 7fe62e7fc640 1 -- 172.21.15.55:0/3535314558 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe630012b90 con 0x7fe6380d1ad0 2024-05-20T22:08:56.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.643+0000 7fe62e7fc640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe61c06be30 0x7fe61c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.644+0000 7fe62e7fc640 1 -- 172.21.15.55:0/3535314558 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe63008b520 con 0x7fe6380d1ad0 2024-05-20T22:08:56.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.644+0000 7fe63dcc8640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe61c06be30 0x7fe61c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.644+0000 7fe63dcc8640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe61c06be30 0x7fe61c06e340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe6380b7830 tx=0x7fe63400d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:56.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.648+0000 7fe62e7fc640 1 -- 172.21.15.55:0/3535314558 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe630059400 con 0x7fe6380d1ad0 2024-05-20T22:08:56.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.838+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3535314558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe6380d1340 con 0x7fe6380d1ad0 2024-05-20T22:08:56.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.838+0000 7fe62e7fc640 1 -- 172.21.15.55:0/3535314558 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fe63005d700 con 0x7fe6380d1ad0 2024-05-20T22:08:56.840 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:56.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.840+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3535314558 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe61c06be30 msgr2=0x7fe61c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.840+0000 7fe63ff2c640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe61c06be30 0x7fe61c06e340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe6380b7830 tx=0x7fe63400d040 comp rx=0 tx=0).stop 2024-05-20T22:08:56.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.840+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3535314558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380d1ad0 msgr2=0x7fe638158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.840+0000 7fe63ff2c640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380d1ad0 0x7fe638158a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe63000d5a0 tx=0x7fe63000da60 comp rx=0 tx=0).stop 2024-05-20T22:08:56.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.841+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3535314558 shutdown_connections 2024-05-20T22:08:56.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.841+0000 7fe63ff2c640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe61c06be30 0x7fe61c06e340 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.841+0000 7fe63ff2c640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6380d1ad0 0x7fe638158a70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.841+0000 7fe63ff2c640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6380ceb10 0x7fe6380c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.841+0000 7fe63ff2c640 1 --2- 172.21.15.55:0/3535314558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe63806e0c0 0x7fe638159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.841+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3535314558 >> 172.21.15.55:0/3535314558 conn(0x7fe6380736f0 msgr2=0x7fe6380b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:56.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.841+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3535314558 shutdown_connections 2024-05-20T22:08:56.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.841+0000 7fe63ff2c640 1 -- 172.21.15.55:0/3535314558 wait complete. 2024-05-20T22:08:56.854 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:08:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.968+0000 7ff5ffa06640 1 Processor -- start 2024-05-20T22:08:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.968+0000 7ff5ffa06640 1 -- start start 2024-05-20T22:08:56.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5ffa06640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b1ef0 0x7ff5f80d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5ffa06640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff5f80d1540 0x7ff5f80b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5ffa06640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f80b40d0 con 0x7ff5f80b7d90 2024-05-20T22:08:56.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5ffa06640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f80b6160 con 0x7ff5f80d1540 2024-05-20T22:08:56.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5ffa06640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f80b5b10 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5fd7a2640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff5f80b7d90 0x7ff5f80b3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53386/0 (socket says 172.21.15.55:53386) 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5fd7a2640 1 -- 172.21.15.55:0/3030780281 learned_addr learned my addr 172.21.15.55:0/3030780281 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5fcfa1640 1 --2- 172.21.15.55:0/3030780281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b1ef0 0x7ff5f80d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5fdfa3640 1 --2- 172.21.15.55:0/3030780281 >> v2:172.21.15.138:3300/0 conn(0x7ff5f80d1540 0x7ff5f80b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5effff640 1 -- 172.21.15.55:0/3030780281 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1603275017 0 0) 0x7ff5f80b40d0 con 0x7ff5f80b7d90 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5effff640 1 -- 172.21.15.55:0/3030780281 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5e4003520 con 0x7ff5f80b7d90 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5fdfa3640 1 -- 172.21.15.55:0/3030780281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b1ef0 msgr2=0x7ff5f80d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5fdfa3640 1 --2- 172.21.15.55:0/3030780281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b1ef0 0x7ff5f80d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5fdfa3640 1 -- 172.21.15.55:0/3030780281 >> v1:172.21.15.55:6789/0 conn(0x7ff5f80b7d90 legacy=0x7ff5f80b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5fdfa3640 1 -- 172.21.15.55:0/3030780281 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f80b5df0 con 0x7ff5f80d1540 2024-05-20T22:08:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.970+0000 7ff5fcfa1640 1 --2- 172.21.15.55:0/3030780281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b1ef0 0x7ff5f80d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.971+0000 7ff5fdfa3640 1 --2- 172.21.15.55:0/3030780281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f80d1540 0x7ff5f80b65f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009d90 tx=0x7ff5f002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e1678d8725781a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.971+0000 7ff5effff640 1 -- 172.21.15.55:0/3030780281 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0007740 con 0x7ff5f80d1540 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.971+0000 7ff5effff640 1 -- 172.21.15.55:0/3030780281 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f00078e0 con 0x7ff5f80d1540 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.971+0000 7ff5effff640 1 -- 172.21.15.55:0/3030780281 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0007cb0 con 0x7ff5f80d1540 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.972+0000 7ff5ffa06640 1 -- 172.21.15.55:0/3030780281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f80d1540 msgr2=0x7ff5f80b65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.972+0000 7ff5ffa06640 1 --2- 172.21.15.55:0/3030780281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f80d1540 0x7ff5f80b65f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009d90 tx=0x7ff5f002a660 comp rx=0 tx=0).stop 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.972+0000 7ff5ffa06640 1 -- 172.21.15.55:0/3030780281 shutdown_connections 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.972+0000 7ff5ffa06640 1 --2- 172.21.15.55:0/3030780281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f80d1540 0x7ff5f80b65f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.972+0000 7ff5ffa06640 1 --2- 172.21.15.55:0/3030780281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b1ef0 0x7ff5f80d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.972+0000 7ff5ffa06640 1 -- 172.21.15.55:0/3030780281 >> 172.21.15.55:0/3030780281 conn(0x7ff5f809be10 msgr2=0x7ff5f80d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.972+0000 7ff5ffa06640 1 -- 172.21.15.55:0/3030780281 shutdown_connections 2024-05-20T22:08:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.972+0000 7ff5ffa06640 1 -- 172.21.15.55:0/3030780281 wait complete. 2024-05-20T22:08:56.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.974+0000 7ff5ffa06640 1 Processor -- start 2024-05-20T22:08:56.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.974+0000 7ff5ffa06640 1 -- start start 2024-05-20T22:08:56.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5ffa06640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5ffa06640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b7d90 0x7ff5f81f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5ffa06640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f80d1540 0x7ff5f81f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5ffa06640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f80b6cb0 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5ffa06640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f80b6b30 con 0x7ff5f80d1540 2024-05-20T22:08:56.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5ffa06640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f80b6e30 con 0x7ff5f80b7d90 2024-05-20T22:08:56.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5fd7a2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5fd7a2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f815f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60500/0 (socket says 172.21.15.55:60500) 2024-05-20T22:08:56.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5fd7a2640 1 -- 172.21.15.55:0/1398459248 learned_addr learned my addr 172.21.15.55:0/1398459248 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:56.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5fcfa1640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b7d90 0x7ff5f81f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.975+0000 7ff5fdfa3640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f80d1540 0x7ff5f81f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5fd7a2640 1 -- 172.21.15.55:0/1398459248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b7d90 msgr2=0x7ff5f81f8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5fd7a2640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b7d90 0x7ff5f81f8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5fd7a2640 1 -- 172.21.15.55:0/1398459248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f80d1540 msgr2=0x7ff5f81f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5fd7a2640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f80d1540 0x7ff5f81f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5fd7a2640 1 -- 172.21.15.55:0/1398459248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f806e0d0 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5fd7a2640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f815f3c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4001eb0 tx=0x7ff5f400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5edffb640 1 -- 172.21.15.55:0/1398459248 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4017070 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5edffb640 1 -- 172.21.15.55:0/1398459248 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f4005d00 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5edffb640 1 -- 172.21.15.55:0/1398459248 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4007310 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5ffa06640 1 -- 172.21.15.55:0/1398459248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f806e3b0 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.976+0000 7ff5ffa06640 1 -- 172.21.15.55:0/1398459248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f806e970 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.977+0000 7ff5ffa06640 1 -- 172.21.15.55:0/1398459248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f8069090 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.978+0000 7ff5edffb640 1 -- 172.21.15.55:0/1398459248 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5f401a070 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.978+0000 7ff5edffb640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff5dc06be60 0x7ff5dc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:56.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.978+0000 7ff5edffb640 1 -- 172.21.15.55:0/1398459248 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff5f408a670 con 0x7ff5f80b1ef0 2024-05-20T22:08:56.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.978+0000 7ff5fcfa1640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff5dc06be60 0x7ff5dc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:56.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.979+0000 7ff5fcfa1640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff5dc06be60 0x7ff5dc06e370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff5f81f87d0 tx=0x7ff5e8009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:56.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:56.983+0000 7ff5edffb640 1 -- 172.21.15.55:0/1398459248 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff5f4058540 con 0x7ff5f80b1ef0 2024-05-20T22:08:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.180+0000 7ff5ffa06640 1 -- 172.21.15.55:0/1398459248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff5f80b1970 con 0x7ff5f80b1ef0 2024-05-20T22:08:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.180+0000 7ff5edffb640 1 -- 172.21.15.55:0/1398459248 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7ff5f405c840 con 0x7ff5f80b1ef0 2024-05-20T22:08:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-20T22:08:57.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.182+0000 7ff5ffa06640 1 -- 172.21.15.55:0/1398459248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff5dc06be60 msgr2=0x7ff5dc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.182+0000 7ff5ffa06640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff5dc06be60 0x7ff5dc06e370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff5f81f87d0 tx=0x7ff5e8009190 comp rx=0 tx=0).stop 2024-05-20T22:08:57.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.182+0000 7ff5ffa06640 1 -- 172.21.15.55:0/1398459248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f80b1ef0 msgr2=0x7ff5f815f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.182+0000 7ff5ffa06640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f815f3c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4001eb0 tx=0x7ff5f400c820 comp rx=0 tx=0).stop 2024-05-20T22:08:57.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.182+0000 7ff5ffa06640 1 -- 172.21.15.55:0/1398459248 shutdown_connections 2024-05-20T22:08:57.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.182+0000 7ff5ffa06640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff5dc06be60 0x7ff5dc06e370 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.182+0000 7ff5ffa06640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5f80d1540 0x7ff5f81f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.182+0000 7ff5ffa06640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5f80b7d90 0x7ff5f81f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.182+0000 7ff5ffa06640 1 --2- 172.21.15.55:0/1398459248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5f80b1ef0 0x7ff5f815f3c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.182+0000 7ff5ffa06640 1 -- 172.21.15.55:0/1398459248 >> 172.21.15.55:0/1398459248 conn(0x7ff5f809be10 msgr2=0x7ff5f80b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:57.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.183+0000 7ff5ffa06640 1 -- 172.21.15.55:0/1398459248 shutdown_connections 2024-05-20T22:08:57.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.183+0000 7ff5ffa06640 1 -- 172.21.15.55:0/1398459248 wait complete. 2024-05-20T22:08:57.196 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:08:57.196 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:08:57.212 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:57.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.324+0000 7f932b691640 1 Processor -- start 2024-05-20T22:08:57.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.325+0000 7f932b691640 1 -- start start 2024-05-20T22:08:57.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.326+0000 7f932b691640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f93240b3b30 0x7f93240b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.326+0000 7f932b691640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932406c790 0x7f93240cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.326+0000 7f932b691640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93240d3dd0 con 0x7f932415f100 2024-05-20T22:08:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.326+0000 7f932b691640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f93240d40b0 con 0x7f93240b3b30 2024-05-20T22:08:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.326+0000 7f932b691640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f93240d4390 con 0x7f932406c790 2024-05-20T22:08:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.326+0000 7f932942d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f93240b3b30 0x7f93240b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.326+0000 7f9328c2c640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f932415f100 0x7f932406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53388/0 (socket says 172.21.15.55:53388) 2024-05-20T22:08:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.326+0000 7f9328c2c640 1 -- 172.21.15.55:0/3732329840 learned_addr learned my addr 172.21.15.55:0/3732329840 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.327+0000 7f932942d640 1 -- 172.21.15.55:0/3732329840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932406c790 msgr2=0x7f93240cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.327+0000 7f932942d640 1 --2- 172.21.15.55:0/3732329840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932406c790 0x7f93240cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.327+0000 7f932942d640 1 -- 172.21.15.55:0/3732329840 >> v1:172.21.15.55:6789/0 conn(0x7f932415f100 legacy=0x7f932406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.327+0000 7f932942d640 1 -- 172.21.15.55:0/3732329840 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932406b5a0 con 0x7f93240b3b30 2024-05-20T22:08:57.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.327+0000 7f932942d640 1 --2- 172.21.15.55:0/3732329840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240b3b30 0x7f93240b7fe0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f93200097a0 tx=0x7f932002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81bb58545e947aa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:57.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.327+0000 7f931bfff640 1 -- 172.21.15.55:0/3732329840 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320037070 con 0x7f93240b3b30 2024-05-20T22:08:57.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.328+0000 7f931bfff640 1 -- 172.21.15.55:0/3732329840 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9320032440 con 0x7f93240b3b30 2024-05-20T22:08:57.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.328+0000 7f931bfff640 1 -- 172.21.15.55:0/3732329840 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320032810 con 0x7f93240b3b30 2024-05-20T22:08:57.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.328+0000 7f932b691640 1 -- 172.21.15.55:0/3732329840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240b3b30 msgr2=0x7f93240b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.328+0000 7f932b691640 1 --2- 172.21.15.55:0/3732329840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240b3b30 0x7f93240b7fe0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f93200097a0 tx=0x7f932002a180 comp rx=0 tx=0).stop 2024-05-20T22:08:57.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.328+0000 7f932b691640 1 -- 172.21.15.55:0/3732329840 shutdown_connections 2024-05-20T22:08:57.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.328+0000 7f932b691640 1 --2- 172.21.15.55:0/3732329840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932406c790 0x7f93240cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.328+0000 7f932b691640 1 --2- 172.21.15.55:0/3732329840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240b3b30 0x7f93240b7fe0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.328+0000 7f932b691640 1 -- 172.21.15.55:0/3732329840 >> 172.21.15.55:0/3732329840 conn(0x7f93240d51d0 msgr2=0x7f93240b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:57.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.328+0000 7f932b691640 1 -- 172.21.15.55:0/3732329840 shutdown_connections 2024-05-20T22:08:57.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.328+0000 7f932b691640 1 -- 172.21.15.55:0/3732329840 wait complete. 2024-05-20T22:08:57.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.330+0000 7f932b691640 1 Processor -- start 2024-05-20T22:08:57.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.330+0000 7f932b691640 1 -- start start 2024-05-20T22:08:57.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.331+0000 7f932b691640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932406c790 0x7f93241490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.331+0000 7f932b691640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93240b3b30 0x7f9324147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.331+0000 7f932b691640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f932415f100 0x7f9324161d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.331+0000 7f932b691640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f932406cba0 con 0x7f932415f100 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.331+0000 7f932b691640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f932406bb60 con 0x7f93240b3b30 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.331+0000 7f932b691640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f932406cd20 con 0x7f932406c790 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9329c2e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f932415f100 0x7f9324161d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9329c2e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f932415f100 0x7f9324161d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60516/0 (socket says 172.21.15.55:60516) 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9329c2e640 1 -- 172.21.15.55:0/334922742 learned_addr learned my addr 172.21.15.55:0/334922742 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f932942d640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932406c790 0x7f93241490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9328c2c640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93240b3b30 0x7f9324147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9329c2e640 1 -- 172.21.15.55:0/334922742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932406c790 msgr2=0x7f93241490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9329c2e640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932406c790 0x7f93241490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9329c2e640 1 -- 172.21.15.55:0/334922742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93240b3b30 msgr2=0x7f9324147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9329c2e640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93240b3b30 0x7f9324147620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9329c2e640 1 -- 172.21.15.55:0/334922742 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9324148000 con 0x7f932415f100 2024-05-20T22:08:57.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9329c2e640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f932415f100 0x7f9324161d40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f931c00b970 tx=0x7f931c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:57.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9319ffb640 1 -- 172.21.15.55:0/334922742 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c019070 con 0x7f932415f100 2024-05-20T22:08:57.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9319ffb640 1 -- 172.21.15.55:0/334922742 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c00cb60 con 0x7f932415f100 2024-05-20T22:08:57.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f932b691640 1 -- 172.21.15.55:0/334922742 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93241482e0 con 0x7f932415f100 2024-05-20T22:08:57.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.332+0000 7f9319ffb640 1 -- 172.21.15.55:0/334922742 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c014440 con 0x7f932415f100 2024-05-20T22:08:57.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.333+0000 7f932b691640 1 -- 172.21.15.55:0/334922742 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93241488a0 con 0x7f932415f100 2024-05-20T22:08:57.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.333+0000 7f932b691640 1 -- 172.21.15.55:0/334922742 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9324069090 con 0x7f932415f100 2024-05-20T22:08:57.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.334+0000 7f9319ffb640 1 -- 172.21.15.55:0/334922742 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f931c01c070 con 0x7f932415f100 2024-05-20T22:08:57.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.335+0000 7f9319ffb640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f930806be30 0x7f930806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.335+0000 7f9319ffb640 1 -- 172.21.15.55:0/334922742 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f931c08aed0 con 0x7f932415f100 2024-05-20T22:08:57.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.335+0000 7f932942d640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f930806be30 0x7f930806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.335+0000 7f932942d640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f930806be30 0x7f930806e340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9320034000 tx=0x7f932002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:57.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.340+0000 7f9319ffb640 1 -- 172.21.15.55:0/334922742 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f931c058e30 con 0x7f932415f100 2024-05-20T22:08:57.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.525+0000 7f932b691640 1 -- 172.21.15.55:0/334922742 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9324147d90 con 0x7f932415f100 2024-05-20T22:08:57.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.526+0000 7f9319ffb640 1 -- 172.21.15.55:0/334922742 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7f931c05d130 con 0x7f932415f100 2024-05-20T22:08:57.528 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:57.528 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:57.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 -- 172.21.15.55:0/334922742 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f930806be30 msgr2=0x7f930806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f930806be30 0x7f930806e340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9320034000 tx=0x7f932002e000 comp rx=0 tx=0).stop 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 -- 172.21.15.55:0/334922742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f932415f100 msgr2=0x7f9324161d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f932415f100 0x7f9324161d40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f931c00b970 tx=0x7f931c00be30 comp rx=0 tx=0).stop 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 -- 172.21.15.55:0/334922742 shutdown_connections 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f930806be30 0x7f930806e340 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f932415f100 0x7f9324161d40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93240b3b30 0x7f9324147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 --2- 172.21.15.55:0/334922742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932406c790 0x7f93241490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 -- 172.21.15.55:0/334922742 >> 172.21.15.55:0/334922742 conn(0x7f93240d51d0 msgr2=0x7f93240cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 -- 172.21.15.55:0/334922742 shutdown_connections 2024-05-20T22:08:57.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.528+0000 7f932b691640 1 -- 172.21.15.55:0/334922742 wait complete. 2024-05-20T22:08:57.542 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:08:57.542 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:08:57.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.659+0000 7f6432d72640 1 Processor -- start 2024-05-20T22:08:57.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.659+0000 7f6432d72640 1 -- start start 2024-05-20T22:08:57.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.660+0000 7f6432d72640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f642c0ceb10 0x7f642c0d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.660+0000 7f6432d72640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f642c0d1ad0 0x7f642c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.660+0000 7f6432d72640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f642c0d1ee0 con 0x7f642c06e0c0 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.660+0000 7f6432d72640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f642c0cef20 con 0x7f642c0ceb10 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.660+0000 7f6432d72640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f642c09a260 con 0x7f642c0d1ad0 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.660+0000 7f642bfff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f642c0ceb10 0x7f642c0d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.660+0000 7f642bfff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f642c0ceb10 0x7f642c0d1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60528/0 (socket says 172.21.15.55:60528) 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.660+0000 7f6430b0e640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f642c06e0c0 0x7f642c15cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53394/0 (socket says 172.21.15.55:53394) 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.660+0000 7f642bfff640 1 -- 172.21.15.55:0/2336748567 learned_addr learned my addr 172.21.15.55:0/2336748567 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.660+0000 7f643130f640 1 --2- 172.21.15.55:0/2336748567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f642c0d1ad0 0x7f642c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.661+0000 7f642bfff640 1 -- 172.21.15.55:0/2336748567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f642c0d1ad0 msgr2=0x7f642c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.661+0000 7f642bfff640 1 --2- 172.21.15.55:0/2336748567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f642c0d1ad0 0x7f642c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.661+0000 7f642bfff640 1 -- 172.21.15.55:0/2336748567 >> v1:172.21.15.55:6789/0 conn(0x7f642c06e0c0 legacy=0x7f642c15cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.661+0000 7f642b7fe640 1 -- 172.21.15.55:0/2336748567 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 639572145 0 0) 0x7f6420001110 con 0x7f642c06e0c0 2024-05-20T22:08:57.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.661+0000 7f642bfff640 1 -- 172.21.15.55:0/2336748567 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c09a540 con 0x7f642c0ceb10 2024-05-20T22:08:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.661+0000 7f642bfff640 1 --2- 172.21.15.55:0/2336748567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0ceb10 0x7f642c0d1590 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6418009870 tx=0x7f641802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7b4031b97d950d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f642b7fe640 1 -- 172.21.15.55:0/2336748567 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418037070 con 0x7f642c0ceb10 2024-05-20T22:08:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f642b7fe640 1 -- 172.21.15.55:0/2336748567 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6418032440 con 0x7f642c0ceb10 2024-05-20T22:08:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f642b7fe640 1 -- 172.21.15.55:0/2336748567 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418032810 con 0x7f642c0ceb10 2024-05-20T22:08:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f6432d72640 1 -- 172.21.15.55:0/2336748567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0ceb10 msgr2=0x7f642c0d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f6432d72640 1 --2- 172.21.15.55:0/2336748567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0ceb10 0x7f642c0d1590 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6418009870 tx=0x7f641802a270 comp rx=0 tx=0).stop 2024-05-20T22:08:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f6432d72640 1 -- 172.21.15.55:0/2336748567 shutdown_connections 2024-05-20T22:08:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f6432d72640 1 --2- 172.21.15.55:0/2336748567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f642c0d1ad0 0x7f642c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f6432d72640 1 --2- 172.21.15.55:0/2336748567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0ceb10 0x7f642c0d1590 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f6432d72640 1 -- 172.21.15.55:0/2336748567 >> 172.21.15.55:0/2336748567 conn(0x7f642c0736f0 msgr2=0x7f642c0b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:57.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f6432d72640 1 -- 172.21.15.55:0/2336748567 shutdown_connections 2024-05-20T22:08:57.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.662+0000 7f6432d72640 1 -- 172.21.15.55:0/2336748567 wait complete. 2024-05-20T22:08:57.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.664+0000 7f6432d72640 1 Processor -- start 2024-05-20T22:08:57.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.664+0000 7f6432d72640 1 -- start start 2024-05-20T22:08:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f6432d72640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f642c06e0c0 0x7f642c0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f6432d72640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f642c0ceb10 0x7f642c1f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f6432d72640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0d1ad0 0x7f642c1e8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f6432d72640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f642c15d2a0 con 0x7f642c0d1ad0 2024-05-20T22:08:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f6432d72640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f642c15c670 con 0x7f642c06e0c0 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f6432d72640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f642c0b2fe0 con 0x7f642c0ceb10 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f643130f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0d1ad0 0x7f642c1e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f643130f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0d1ad0 0x7f642c1e8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60544/0 (socket says 172.21.15.55:60544) 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f643130f640 1 -- 172.21.15.55:0/1122569928 learned_addr learned my addr 172.21.15.55:0/1122569928 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f6430b0e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f642c06e0c0 0x7f642c0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.666+0000 7f642bfff640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f642c0ceb10 0x7f642c1f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f643130f640 1 -- 172.21.15.55:0/1122569928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f642c0ceb10 msgr2=0x7f642c1f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f643130f640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f642c0ceb10 0x7f642c1f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f643130f640 1 -- 172.21.15.55:0/1122569928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f642c06e0c0 msgr2=0x7f642c0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f643130f640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f642c06e0c0 0x7f642c0c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f643130f640 1 -- 172.21.15.55:0/1122569928 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c0cb6a0 con 0x7f642c0d1ad0 2024-05-20T22:08:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f643130f640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0d1ad0 0x7f642c1e8870 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6424009cb0 tx=0x7f642400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:57.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f64297fa640 1 -- 172.21.15.55:0/1122569928 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424018020 con 0x7f642c0d1ad0 2024-05-20T22:08:57.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f64297fa640 1 -- 172.21.15.55:0/1122569928 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6424014480 con 0x7f642c0d1ad0 2024-05-20T22:08:57.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f6432d72640 1 -- 172.21.15.55:0/1122569928 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642c0c8d30 con 0x7f642c0d1ad0 2024-05-20T22:08:57.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f64297fa640 1 -- 172.21.15.55:0/1122569928 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424014870 con 0x7f642c0d1ad0 2024-05-20T22:08:57.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.667+0000 7f6432d72640 1 -- 172.21.15.55:0/1122569928 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f642c0c56f0 con 0x7f642c0d1ad0 2024-05-20T22:08:57.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.668+0000 7f6432d72640 1 -- 172.21.15.55:0/1122569928 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642c0c8f00 con 0x7f642c0d1ad0 2024-05-20T22:08:57.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.669+0000 7f64297fa640 1 -- 172.21.15.55:0/1122569928 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6424005d10 con 0x7f642c0d1ad0 2024-05-20T22:08:57.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.669+0000 7f64297fa640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f641006be30 0x7f641006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.669+0000 7f64297fa640 1 -- 172.21.15.55:0/1122569928 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f642408beb0 con 0x7f642c0d1ad0 2024-05-20T22:08:57.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.669+0000 7f6430b0e640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f641006be30 0x7f641006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.670+0000 7f6430b0e640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f641006be30 0x7f641006e340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6420007fd0 tx=0x7f6420007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:57.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.674+0000 7f64297fa640 1 -- 172.21.15.55:0/1122569928 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6424022030 con 0x7f642c0d1ad0 2024-05-20T22:08:57.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.859+0000 7f6432d72640 1 -- 172.21.15.55:0/1122569928 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f642c069090 con 0x7f642c0d1ad0 2024-05-20T22:08:57.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.860+0000 7f64297fa640 1 -- 172.21.15.55:0/1122569928 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f6424010040 con 0x7f642c0d1ad0 2024-05-20T22:08:57.862 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:08:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 -- 172.21.15.55:0/1122569928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f641006be30 msgr2=0x7f641006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f641006be30 0x7f641006e340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6420007fd0 tx=0x7f6420007de0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 -- 172.21.15.55:0/1122569928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0d1ad0 msgr2=0x7f642c1e8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0d1ad0 0x7f642c1e8870 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6424009cb0 tx=0x7f642400c490 comp rx=0 tx=0).stop 2024-05-20T22:08:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 -- 172.21.15.55:0/1122569928 shutdown_connections 2024-05-20T22:08:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f641006be30 0x7f641006e340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f642c0d1ad0 0x7f642c1e8870 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f642c0ceb10 0x7f642c1f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 --2- 172.21.15.55:0/1122569928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f642c06e0c0 0x7f642c0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 -- 172.21.15.55:0/1122569928 >> 172.21.15.55:0/1122569928 conn(0x7f642c0736f0 msgr2=0x7f642c1545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 -- 172.21.15.55:0/1122569928 shutdown_connections 2024-05-20T22:08:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.862+0000 7f6432d72640 1 -- 172.21.15.55:0/1122569928 wait complete. 2024-05-20T22:08:57.876 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:08:57.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.993+0000 7f98a7473640 1 Processor -- start 2024-05-20T22:08:57.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.993+0000 7f98a7473640 1 -- start start 2024-05-20T22:08:57.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.994+0000 7f98a7473640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98a00d32a0 0x7f98a015f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.994+0000 7f98a7473640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a00cf690 0x7f98a00cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:57.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.994+0000 7f98a7473640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a00d3780 con 0x7f98a00cffe0 2024-05-20T22:08:57.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.995+0000 7f98a7473640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f98a006c510 con 0x7f98a00d32a0 2024-05-20T22:08:57.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.995+0000 7f98a7473640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98a00d3a60 con 0x7f98a00cf690 2024-05-20T22:08:57.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.995+0000 7f98a520f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98a00d32a0 0x7f98a015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.995+0000 7f98a520f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98a00d32a0 0x7f98a015f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60560/0 (socket says 172.21.15.55:60560) 2024-05-20T22:08:57.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.995+0000 7f98a520f640 1 -- 172.21.15.55:0/4288808352 learned_addr learned my addr 172.21.15.55:0/4288808352 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:57.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.995+0000 7f98a4a0e640 1 --2- 172.21.15.55:0/4288808352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a00cf690 0x7f98a00cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:57.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.995+0000 7f98a520f640 1 -- 172.21.15.55:0/4288808352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a00cf690 msgr2=0x7f98a00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:57.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.995+0000 7f98a520f640 1 --2- 172.21.15.55:0/4288808352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a00cf690 0x7f98a00cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:57.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.995+0000 7f98a520f640 1 -- 172.21.15.55:0/4288808352 >> v1:172.21.15.138:6789/0 conn(0x7f98a00cffe0 legacy=0x7f98a0162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:57.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.995+0000 7f98a520f640 1 -- 172.21.15.55:0/4288808352 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a00d3d40 con 0x7f98a00d32a0 2024-05-20T22:08:57.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.996+0000 7f98a520f640 1 --2- 172.21.15.55:0/4288808352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 0x7f98a015f150 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f989c009870 tx=0x7f989c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05d255b450db31c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:57.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.996+0000 7f9897fff640 1 -- 172.21.15.55:0/4288808352 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c037070 con 0x7f98a00d32a0 2024-05-20T22:08:57.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.996+0000 7f9897fff640 1 -- 172.21.15.55:0/4288808352 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989c032440 con 0x7f98a00d32a0 2024-05-20T22:08:57.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.996+0000 7f9897fff640 1 -- 172.21.15.55:0/4288808352 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c032810 con 0x7f98a00d32a0 2024-05-20T22:08:57.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.997+0000 7f98a7473640 1 -- 172.21.15.55:0/4288808352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 msgr2=0x7f98a015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.997+0000 7f98a7473640 1 --2- 172.21.15.55:0/4288808352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 0x7f98a015f150 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f989c009870 tx=0x7f989c02a210 comp rx=0 tx=0).stop 2024-05-20T22:08:58.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.997+0000 7f98a7473640 1 -- 172.21.15.55:0/4288808352 shutdown_connections 2024-05-20T22:08:58.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.997+0000 7f98a7473640 1 --2- 172.21.15.55:0/4288808352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a00cf690 0x7f98a00cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.997+0000 7f98a7473640 1 --2- 172.21.15.55:0/4288808352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 0x7f98a015f150 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.997+0000 7f98a7473640 1 -- 172.21.15.55:0/4288808352 >> 172.21.15.55:0/4288808352 conn(0x7f98a00b6de0 msgr2=0x7f98a00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:58.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.997+0000 7f98a7473640 1 -- 172.21.15.55:0/4288808352 shutdown_connections 2024-05-20T22:08:58.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.997+0000 7f98a7473640 1 -- 172.21.15.55:0/4288808352 wait complete. 2024-05-20T22:08:58.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.999+0000 7f98a7473640 1 Processor -- start 2024-05-20T22:08:58.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:57.999+0000 7f98a7473640 1 -- start start 2024-05-20T22:08:58.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a7473640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a00cf690 0x7f98a015d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a7473640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00cffe0 0x7f98a00ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a7473640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 0x7f98a00ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a7473640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00d03f0 con 0x7f98a00d32a0 2024-05-20T22:08:58.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a7473640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00d4300 con 0x7f98a00cffe0 2024-05-20T22:08:58.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a7473640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98a00d4820 con 0x7f98a00cf690 2024-05-20T22:08:58.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a5a10640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 0x7f98a00ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a5a10640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 0x7f98a00ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60562/0 (socket says 172.21.15.55:60562) 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a5a10640 1 -- 172.21.15.55:0/2864852566 learned_addr learned my addr 172.21.15.55:0/2864852566 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a520f640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a00cf690 0x7f98a015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a4a0e640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00cffe0 0x7f98a00ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a5a10640 1 -- 172.21.15.55:0/2864852566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a00cf690 msgr2=0x7f98a015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a5a10640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a00cf690 0x7f98a015d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a5a10640 1 -- 172.21.15.55:0/2864852566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00cffe0 msgr2=0x7f98a00ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a5a10640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00cffe0 0x7f98a00ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.000+0000 7f98a5a10640 1 -- 172.21.15.55:0/2864852566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a015f870 con 0x7f98a00d32a0 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.001+0000 7f98a5a10640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 0x7f98a00ce5b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f989800be30 tx=0x7f989800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.001+0000 7f9895ffb640 1 -- 172.21.15.55:0/2864852566 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898019070 con 0x7f98a00d32a0 2024-05-20T22:08:58.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.001+0000 7f9895ffb640 1 -- 172.21.15.55:0/2864852566 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989800caf0 con 0x7f98a00d32a0 2024-05-20T22:08:58.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.001+0000 7f98a7473640 1 -- 172.21.15.55:0/2864852566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a015fb50 con 0x7f98a00d32a0 2024-05-20T22:08:58.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.001+0000 7f9895ffb640 1 -- 172.21.15.55:0/2864852566 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898014440 con 0x7f98a00d32a0 2024-05-20T22:08:58.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.001+0000 7f98a7473640 1 -- 172.21.15.55:0/2864852566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a0160110 con 0x7f98a00d32a0 2024-05-20T22:08:58.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.002+0000 7f98a7473640 1 -- 172.21.15.55:0/2864852566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a0069090 con 0x7f98a00d32a0 2024-05-20T22:08:58.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.003+0000 7f9895ffb640 1 -- 172.21.15.55:0/2864852566 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f989801c070 con 0x7f98a00d32a0 2024-05-20T22:08:58.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.003+0000 7f9895ffb640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f988406be60 0x7f988406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.003+0000 7f9895ffb640 1 -- 172.21.15.55:0/2864852566 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f989808ad20 con 0x7f98a00d32a0 2024-05-20T22:08:58.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.003+0000 7f98a520f640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f988406be60 0x7f988406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.004+0000 7f98a520f640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f988406be60 0x7f988406e370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f989c034000 tx=0x7f989c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:58.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.008+0000 7f9895ffb640 1 -- 172.21.15.55:0/2864852566 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9898058c00 con 0x7f98a00d32a0 2024-05-20T22:08:58.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.195+0000 7f98a7473640 1 -- 172.21.15.55:0/2864852566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f98a00d2e50 con 0x7f98a00d32a0 2024-05-20T22:08:58.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.195+0000 7f9895ffb640 1 -- 172.21.15.55:0/2864852566 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f989805cf00 con 0x7f98a00d32a0 2024-05-20T22:08:58.197 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.197+0000 7f98a7473640 1 -- 172.21.15.55:0/2864852566 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f988406be60 msgr2=0x7f988406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.197+0000 7f98a7473640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f988406be60 0x7f988406e370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f989c034000 tx=0x7f989c02e000 comp rx=0 tx=0).stop 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.197+0000 7f98a7473640 1 -- 172.21.15.55:0/2864852566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 msgr2=0x7f98a00ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.197+0000 7f98a7473640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 0x7f98a00ce5b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f989800be30 tx=0x7f989800bf20 comp rx=0 tx=0).stop 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.198+0000 7f98a7473640 1 -- 172.21.15.55:0/2864852566 shutdown_connections 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.198+0000 7f98a7473640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f988406be60 0x7f988406e370 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.198+0000 7f98a7473640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98a00d32a0 0x7f98a00ce5b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.198+0000 7f98a7473640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00cffe0 0x7f98a00ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.198+0000 7f98a7473640 1 --2- 172.21.15.55:0/2864852566 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a00cf690 0x7f98a015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.198+0000 7f98a7473640 1 -- 172.21.15.55:0/2864852566 >> 172.21.15.55:0/2864852566 conn(0x7f98a00b6de0 msgr2=0x7f98a00b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.198+0000 7f98a7473640 1 -- 172.21.15.55:0/2864852566 shutdown_connections 2024-05-20T22:08:58.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.198+0000 7f98a7473640 1 -- 172.21.15.55:0/2864852566 wait complete. 2024-05-20T22:08:58.211 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:08:58.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.326+0000 7f1c3903f640 1 Processor -- start 2024-05-20T22:08:58.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.326+0000 7f1c3903f640 1 -- start start 2024-05-20T22:08:58.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3903f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340b4080 0x7f1c3415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3903f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c340d3290 0x7f1c340d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3903f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c3406b500 con 0x7f1c340b35e0 2024-05-20T22:08:58.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3903f640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f1c34163960 con 0x7f1c340d3290 2024-05-20T22:08:58.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3903f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c340b3e10 con 0x7f1c340b4080 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3359e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c340d3290 0x7f1c340d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3359e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c340d3290 0x7f1c340d36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60570/0 (socket says 172.21.15.55:60570) 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3359e640 1 -- 172.21.15.55:0/2646270271 learned_addr learned my addr 172.21.15.55:0/2646270271 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c32d9d640 1 --2- 172.21.15.55:0/2646270271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340b4080 0x7f1c3415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3359e640 1 -- 172.21.15.55:0/2646270271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340b4080 msgr2=0x7f1c3415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3359e640 1 --2- 172.21.15.55:0/2646270271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340b4080 0x7f1c3415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3359e640 1 -- 172.21.15.55:0/2646270271 >> v1:172.21.15.138:6789/0 conn(0x7f1c340b35e0 legacy=0x7f1c3415f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.328+0000 7f1c3359e640 1 -- 172.21.15.55:0/2646270271 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c340b3b30 con 0x7f1c340d3290 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.329+0000 7f1c3359e640 1 --2- 172.21.15.55:0/2646270271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 0x7f1c340d36a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009870 tx=0x7f1c2402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d306257f2503780 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.329+0000 7f1c31d9b640 1 -- 172.21.15.55:0/2646270271 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24037070 con 0x7f1c340d3290 2024-05-20T22:08:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.329+0000 7f1c31d9b640 1 -- 172.21.15.55:0/2646270271 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c24032440 con 0x7f1c340d3290 2024-05-20T22:08:58.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.329+0000 7f1c31d9b640 1 -- 172.21.15.55:0/2646270271 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24032810 con 0x7f1c340d3290 2024-05-20T22:08:58.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.330+0000 7f1c3903f640 1 -- 172.21.15.55:0/2646270271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 msgr2=0x7f1c340d36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.330+0000 7f1c3903f640 1 --2- 172.21.15.55:0/2646270271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 0x7f1c340d36a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009870 tx=0x7f1c2402a210 comp rx=0 tx=0).stop 2024-05-20T22:08:58.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.330+0000 7f1c3903f640 1 -- 172.21.15.55:0/2646270271 shutdown_connections 2024-05-20T22:08:58.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.330+0000 7f1c3903f640 1 --2- 172.21.15.55:0/2646270271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 0x7f1c340d36a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.330+0000 7f1c3903f640 1 --2- 172.21.15.55:0/2646270271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340b4080 0x7f1c3415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.330+0000 7f1c3903f640 1 -- 172.21.15.55:0/2646270271 >> 172.21.15.55:0/2646270271 conn(0x7f1c340d51b0 msgr2=0x7f1c340b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:58.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.330+0000 7f1c3903f640 1 -- 172.21.15.55:0/2646270271 shutdown_connections 2024-05-20T22:08:58.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.330+0000 7f1c3903f640 1 -- 172.21.15.55:0/2646270271 wait complete. 2024-05-20T22:08:58.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.332+0000 7f1c3903f640 1 Processor -- start 2024-05-20T22:08:58.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.332+0000 7f1c3903f640 1 -- start start 2024-05-20T22:08:58.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.333+0000 7f1c3903f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340b35e0 0x7f1c340c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.333+0000 7f1c3903f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c340b4080 0x7f1c34156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.333+0000 7f1c3903f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 0x7f1c340ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.333+0000 7f1c3903f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341635f0 con 0x7f1c340d3290 2024-05-20T22:08:58.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.333+0000 7f1c3903f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341604f0 con 0x7f1c340b4080 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.333+0000 7f1c3903f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3415ff30 con 0x7f1c340b35e0 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3359e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 0x7f1c340ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3359e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 0x7f1c340ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60580/0 (socket says 172.21.15.55:60580) 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3359e640 1 -- 172.21.15.55:0/1528303146 learned_addr learned my addr 172.21.15.55:0/1528303146 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c32d9d640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340b35e0 0x7f1c340c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3259c640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c340b4080 0x7f1c34156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3359e640 1 -- 172.21.15.55:0/1528303146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340b35e0 msgr2=0x7f1c340c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3359e640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340b35e0 0x7f1c340c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3359e640 1 -- 172.21.15.55:0/1528303146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c340b4080 msgr2=0x7f1c34156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3359e640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c340b4080 0x7f1c34156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3359e640 1 -- 172.21.15.55:0/1528303146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c34160950 con 0x7f1c340d3290 2024-05-20T22:08:58.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3359e640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 0x7f1c340ca070 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009870 tx=0x7f1c240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:58.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c2b7fe640 1 -- 172.21.15.55:0/1528303146 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24041020 con 0x7f1c340d3290 2024-05-20T22:08:58.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c2b7fe640 1 -- 172.21.15.55:0/1528303146 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c240306c0 con 0x7f1c340d3290 2024-05-20T22:08:58.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c3903f640 1 -- 172.21.15.55:0/1528303146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c34160c30 con 0x7f1c340d3290 2024-05-20T22:08:58.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.334+0000 7f1c2b7fe640 1 -- 172.21.15.55:0/1528303146 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24037040 con 0x7f1c340d3290 2024-05-20T22:08:58.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.335+0000 7f1c3903f640 1 -- 172.21.15.55:0/1528303146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c341611f0 con 0x7f1c340d3290 2024-05-20T22:08:58.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.335+0000 7f1c3903f640 1 -- 172.21.15.55:0/1528303146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c340c95c0 con 0x7f1c340d3290 2024-05-20T22:08:58.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.336+0000 7f1c2b7fe640 1 -- 172.21.15.55:0/1528303146 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c24044020 con 0x7f1c340d3290 2024-05-20T22:08:58.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.337+0000 7f1c2b7fe640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c1406be30 0x7f1c1406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.337+0000 7f1c32d9d640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c1406be30 0x7f1c1406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.337+0000 7f1c2b7fe640 1 -- 172.21.15.55:0/1528303146 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1c240aba80 con 0x7f1c340d3290 2024-05-20T22:08:58.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.337+0000 7f1c32d9d640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c1406be30 0x7f1c1406e340 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1c340d0980 tx=0x7f1c2c009830 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:58.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.342+0000 7f1c2b7fe640 1 -- 172.21.15.55:0/1528303146 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c240ade20 con 0x7f1c340d3290 2024-05-20T22:08:58.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.524+0000 7f1c3903f640 1 -- 172.21.15.55:0/1528303146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c34069090 con 0x7f1c340d3290 2024-05-20T22:08:58.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.525+0000 7f1c2b7fe640 1 -- 172.21.15.55:0/1528303146 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f1c2404b030 con 0x7f1c340d3290 2024-05-20T22:08:58.527 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:08:58.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 -- 172.21.15.55:0/1528303146 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c1406be30 msgr2=0x7f1c1406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c1406be30 0x7f1c1406e340 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1c340d0980 tx=0x7f1c2c009830 comp rx=0 tx=0).stop 2024-05-20T22:08:58.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 -- 172.21.15.55:0/1528303146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 msgr2=0x7f1c340ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 0x7f1c340ca070 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009870 tx=0x7f1c240046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 -- 172.21.15.55:0/1528303146 shutdown_connections 2024-05-20T22:08:58.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1c1406be30 0x7f1c1406e340 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c340d3290 0x7f1c340ca070 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c340b4080 0x7f1c34156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 --2- 172.21.15.55:0/1528303146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c340b35e0 0x7f1c340c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 -- 172.21.15.55:0/1528303146 >> 172.21.15.55:0/1528303146 conn(0x7f1c340d51b0 msgr2=0x7f1c340b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:58.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 -- 172.21.15.55:0/1528303146 shutdown_connections 2024-05-20T22:08:58.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.527+0000 7f1c3903f640 1 -- 172.21.15.55:0/1528303146 wait complete. 2024-05-20T22:08:58.540 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:08:58.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.656+0000 7faaa778f640 1 Processor -- start 2024-05-20T22:08:58.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.656+0000 7faaa778f640 1 -- start start 2024-05-20T22:08:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.657+0000 7faaa778f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faaa008c960 0x7faaa00b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.657+0000 7faaa778f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa00cf270 0x7faaa00cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.657+0000 7faaa778f640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaa008cd70 con 0x7faaa00cd520 2024-05-20T22:08:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.657+0000 7faaa778f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faaa00cfbc0 con 0x7faaa008c960 2024-05-20T22:08:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.657+0000 7faaa778f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faaa00b1f30 con 0x7faaa00cf270 2024-05-20T22:08:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.657+0000 7faaa552b640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7faaa00cd520 0x7faaa00d0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53404/0 (socket says 172.21.15.55:53404) 2024-05-20T22:08:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.657+0000 7faaa552b640 1 -- 172.21.15.55:0/3331626096 learned_addr learned my addr 172.21.15.55:0/3331626096 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faaa5d2c640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa00cf270 0x7faaa00cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faaa4d2a640 1 --2- 172.21.15.55:0/3331626096 >> v2:172.21.15.138:3300/0 conn(0x7faaa008c960 0x7faaa00b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 216920313 0 0) 0x7faa9c001110 con 0x7faaa00cd520 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa8c003520 con 0x7faaa00cd520 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1505202008 0 0) 0x7faa8c003520 con 0x7faaa00cd520 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa00cf270 msgr2=0x7faaa00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa00cf270 0x7faaa00cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 >> v2:172.21.15.138:3300/0 conn(0x7faaa008c960 msgr2=0x7faaa00b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 --2- 172.21.15.55:0/3331626096 >> v2:172.21.15.138:3300/0 conn(0x7faaa008c960 0x7faaa00b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa00d1020 con 0x7faaa00cd520 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7faa9c001840 con 0x7faaa00cd520 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faaa4d2a640 1 --2- 172.21.15.55:0/3331626096 >> v2:172.21.15.138:3300/0 conn(0x7faaa008c960 0x7faaa00b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 >> v1:172.21.15.55:6789/0 conn(0x7faaa00cd520 legacy=0x7faaa00d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa00cf270 0x7faa8c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c003f30 0x7faa8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa8c004d40 0x7faa8c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7faaa00cfbc0 con 0x7faa8c004d40 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faaa00b1f30 con 0x7faa8c003f30 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa8c00bac0 con 0x7faaa00cf270 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faaa552b640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa00cf270 0x7faa8c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faaa5d2c640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa8c004d40 0x7faa8c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.658+0000 7faaa4d2a640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c003f30 0x7faa8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.659+0000 7faaa5d2c640 1 -- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa00cf270 msgr2=0x7faa8c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.659+0000 7faaa5d2c640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa00cf270 0x7faa8c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.659+0000 7faaa5d2c640 1 -- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c003f30 msgr2=0x7faa8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:58.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.659+0000 7faaa5d2c640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c003f30 0x7faa8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.659+0000 7faaa5d2c640 1 -- 172.21.15.55:0/3331626096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c003520 con 0x7faa8c004d40 2024-05-20T22:08:58.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.659+0000 7faaa5d2c640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa8c004d40 0x7faa8c005150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faa98009f90 tx=0x7faa8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ac7cbefdb492ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:58.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.660+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98009c10 con 0x7faa8c004d40 2024-05-20T22:08:58.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.660+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa98036070 con 0x7faa8c004d40 2024-05-20T22:08:58.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.660+0000 7faa97fff640 1 -- 172.21.15.55:0/3331626096 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98031850 con 0x7faa8c004d40 2024-05-20T22:08:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.660+0000 7faaa778f640 1 -- 172.21.15.55:0/3331626096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa8c004d40 msgr2=0x7faa8c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.660+0000 7faaa778f640 1 --2- 172.21.15.55:0/3331626096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa8c004d40 0x7faa8c005150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faa98009f90 tx=0x7faa8c005830 comp rx=0 tx=0).stop 2024-05-20T22:08:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.660+0000 7faaa5d2c640 1 -- 172.21.15.55:0/3331626096 reap_dead start 2024-05-20T22:08:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.660+0000 7faaa778f640 1 -- 172.21.15.55:0/3331626096 shutdown_connections 2024-05-20T22:08:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.660+0000 7faaa778f640 1 -- 172.21.15.55:0/3331626096 >> 172.21.15.55:0/3331626096 conn(0x7faaa00d2860 msgr2=0x7faaa00d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.660+0000 7faaa778f640 1 -- 172.21.15.55:0/3331626096 shutdown_connections 2024-05-20T22:08:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.660+0000 7faaa778f640 1 -- 172.21.15.55:0/3331626096 wait complete. 2024-05-20T22:08:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.662+0000 7faaa778f640 1 Processor -- start 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.662+0000 7faaa778f640 1 -- start start 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.663+0000 7faaa778f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c004d40 0x7faaa0157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa778f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faaa015abc0 0x7faaa01f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa778f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa01f3880 0x7faaa01f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa778f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7faaa00d18c0 con 0x7faaa015abc0 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa778f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faaa00d15e0 con 0x7faa8c004d40 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa778f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faaa00b2cb0 con 0x7faaa01f3880 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa4d2a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faaa015abc0 0x7faaa01f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa4d2a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faaa015abc0 0x7faaa01f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47750/0 (socket says 172.21.15.55:47750) 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa4d2a640 1 -- 172.21.15.55:0/3080876115 learned_addr learned my addr 172.21.15.55:0/3080876115 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa5d2c640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa01f3880 0x7faaa01f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa552b640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c004d40 0x7faaa0157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa4d2a640 1 -- 172.21.15.55:0/3080876115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa01f3880 msgr2=0x7faaa01f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa4d2a640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa01f3880 0x7faaa01f6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa4d2a640 1 -- 172.21.15.55:0/3080876115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c004d40 msgr2=0x7faaa0157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa4d2a640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c004d40 0x7faaa0157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa4d2a640 1 -- 172.21.15.55:0/3080876115 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa0157cd0 con 0x7faaa015abc0 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa5d2c640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa01f3880 0x7faaa01f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa552b640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c004d40 0x7faaa0157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa4d2a640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faaa015abc0 0x7faaa01f3340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faa90011d00 tx=0x7faa9000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faa95ffb640 1 -- 172.21.15.55:0/3080876115 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9000fd20 con 0x7faaa015abc0 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faa95ffb640 1 -- 172.21.15.55:0/3080876115 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa90004020 con 0x7faaa015abc0 2024-05-20T22:08:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faaa778f640 1 -- 172.21.15.55:0/3080876115 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa0159160 con 0x7faaa015abc0 2024-05-20T22:08:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.664+0000 7faa95ffb640 1 -- 172.21.15.55:0/3080876115 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90004410 con 0x7faaa015abc0 2024-05-20T22:08:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.665+0000 7faaa778f640 1 -- 172.21.15.55:0/3080876115 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa0159720 con 0x7faaa015abc0 2024-05-20T22:08:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.666+0000 7faaa778f640 1 -- 172.21.15.55:0/3080876115 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa0069090 con 0x7faaa015abc0 2024-05-20T22:08:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.666+0000 7faa95ffb640 1 -- 172.21.15.55:0/3080876115 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa90012040 con 0x7faaa015abc0 2024-05-20T22:08:58.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.667+0000 7faa95ffb640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa8406be30 0x7faa8406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.667+0000 7faa95ffb640 1 -- 172.21.15.55:0/3080876115 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7faa9008b800 con 0x7faaa015abc0 2024-05-20T22:08:58.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.667+0000 7faaa552b640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa8406be30 0x7faa8406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.667+0000 7faaa552b640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa8406be30 0x7faa8406e340 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faa9c000fb0 tx=0x7faa9c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:58.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.671+0000 7faa95ffb640 1 -- 172.21.15.55:0/3080876115 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faa90019030 con 0x7faaa015abc0 2024-05-20T22:08:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.857+0000 7faaa778f640 1 -- 172.21.15.55:0/3080876115 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faaa00b27c0 con 0x7faaa015abc0 2024-05-20T22:08:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.857+0000 7faa95ffb640 1 -- 172.21.15.55:0/3080876115 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7faa90059db0 con 0x7faaa015abc0 2024-05-20T22:08:58.859 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:08:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.859+0000 7faaa778f640 1 -- 172.21.15.55:0/3080876115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa8406be30 msgr2=0x7faa8406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.859+0000 7faaa778f640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa8406be30 0x7faa8406e340 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faa9c000fb0 tx=0x7faa9c009040 comp rx=0 tx=0).stop 2024-05-20T22:08:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.859+0000 7faaa778f640 1 -- 172.21.15.55:0/3080876115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faaa015abc0 msgr2=0x7faaa01f3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.859+0000 7faaa778f640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faaa015abc0 0x7faaa01f3340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faa90011d00 tx=0x7faa9000f610 comp rx=0 tx=0).stop 2024-05-20T22:08:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.859+0000 7faaa778f640 1 -- 172.21.15.55:0/3080876115 shutdown_connections 2024-05-20T22:08:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.859+0000 7faaa778f640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faa8406be30 0x7faa8406e340 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.859+0000 7faaa778f640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaa01f3880 0x7faaa01f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.859+0000 7faaa778f640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faaa015abc0 0x7faaa01f3340 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.859+0000 7faaa778f640 1 --2- 172.21.15.55:0/3080876115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c004d40 0x7faaa0157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.859+0000 7faaa778f640 1 -- 172.21.15.55:0/3080876115 >> 172.21.15.55:0/3080876115 conn(0x7faaa00d2860 msgr2=0x7faaa00706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.860+0000 7faaa778f640 1 -- 172.21.15.55:0/3080876115 shutdown_connections 2024-05-20T22:08:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.860+0000 7faaa778f640 1 -- 172.21.15.55:0/3080876115 wait complete. 2024-05-20T22:08:58.873 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.988+0000 7fb554702640 1 Processor -- start 2024-05-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.988+0000 7fb554702640 1 -- start start 2024-05-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.989+0000 7fb554702640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 0x7fb54c15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.989+0000 7fb554702640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb54c0d3290 0x7fb54c0d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.989+0000 7fb554702640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb54c06b500 con 0x7fb54c0b4080 2024-05-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.989+0000 7fb554702640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb54c163960 con 0x7fb54c0d3290 2024-05-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.989+0000 7fb554702640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb54c0b3e10 con 0x7fb54c0b35e0 2024-05-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.990+0000 7fb552c9f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb54c0d3290 0x7fb54c0d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.990+0000 7fb551c9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 0x7fb54c15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.990+0000 7fb55249e640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb54c0b4080 0x7fb54c15f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:43444/0 (socket says 172.21.15.55:43444) 2024-05-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.990+0000 7fb55249e640 1 -- 172.21.15.55:0/753536382 learned_addr learned my addr 172.21.15.55:0/753536382 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.990+0000 7fb552c9f640 1 -- 172.21.15.55:0/753536382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 msgr2=0x7fb54c15f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.990+0000 7fb552c9f640 1 --2- 172.21.15.55:0/753536382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 0x7fb54c15f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.990+0000 7fb552c9f640 1 -- 172.21.15.55:0/753536382 >> v1:172.21.15.138:6789/0 conn(0x7fb54c0b4080 legacy=0x7fb54c15f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.990+0000 7fb55149c640 1 -- 172.21.15.55:0/753536382 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1346574668 0 0) 0x7fb548001110 con 0x7fb54c0b4080 2024-05-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.990+0000 7fb552c9f640 1 -- 172.21.15.55:0/753536382 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c0b3b30 con 0x7fb54c0d3290 2024-05-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.990+0000 7fb551c9d640 1 --2- 172.21.15.55:0/753536382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 0x7fb54c15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.991+0000 7fb552c9f640 1 --2- 172.21.15.55:0/753536382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb54c0d3290 0x7fb54c0d36a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb544009d90 tx=0x7fb54402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a52d05e9490ca6f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:58.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.991+0000 7fb55149c640 1 -- 172.21.15.55:0/753536382 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544007740 con 0x7fb54c0d3290 2024-05-20T22:08:58.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.991+0000 7fb55149c640 1 -- 172.21.15.55:0/753536382 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5440078e0 con 0x7fb54c0d3290 2024-05-20T22:08:58.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.991+0000 7fb55149c640 1 -- 172.21.15.55:0/753536382 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544007cb0 con 0x7fb54c0d3290 2024-05-20T22:08:58.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.992+0000 7fb554702640 1 -- 172.21.15.55:0/753536382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb54c0d3290 msgr2=0x7fb54c0d36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:58.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.992+0000 7fb554702640 1 --2- 172.21.15.55:0/753536382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb54c0d3290 0x7fb54c0d36a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb544009d90 tx=0x7fb54402a660 comp rx=0 tx=0).stop 2024-05-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.992+0000 7fb554702640 1 -- 172.21.15.55:0/753536382 shutdown_connections 2024-05-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.992+0000 7fb554702640 1 --2- 172.21.15.55:0/753536382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb54c0d3290 0x7fb54c0d36a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.992+0000 7fb554702640 1 --2- 172.21.15.55:0/753536382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 0x7fb54c15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.992+0000 7fb554702640 1 -- 172.21.15.55:0/753536382 >> 172.21.15.55:0/753536382 conn(0x7fb54c0d51b0 msgr2=0x7fb54c0b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.992+0000 7fb554702640 1 -- 172.21.15.55:0/753536382 shutdown_connections 2024-05-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.992+0000 7fb554702640 1 -- 172.21.15.55:0/753536382 wait complete. 2024-05-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.994+0000 7fb554702640 1 Processor -- start 2024-05-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.994+0000 7fb554702640 1 -- start start 2024-05-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb554702640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 0x7fb54c0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb554702640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb54c0b4080 0x7fb54c147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb554702640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb54c0d3290 0x7fb54c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb554702640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1635f0 con 0x7fb54c0b4080 2024-05-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb554702640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1604f0 con 0x7fb54c0d3290 2024-05-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb554702640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb54c15ff30 con 0x7fb54c0b35e0 2024-05-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb551c9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb54c0b4080 0x7fb54c147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb551c9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb54c0b4080 0x7fb54c147f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47764/0 (socket says 172.21.15.55:47764) 2024-05-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb551c9d640 1 -- 172.21.15.55:0/2519336985 learned_addr learned my addr 172.21.15.55:0/2519336985 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb552c9f640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb54c0d3290 0x7fb54c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb55249e640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 0x7fb54c0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb551c9d640 1 -- 172.21.15.55:0/2519336985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 msgr2=0x7fb54c0c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb551c9d640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 0x7fb54c0c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.995+0000 7fb551c9d640 1 -- 172.21.15.55:0/2519336985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb54c0d3290 msgr2=0x7fb54c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.996+0000 7fb551c9d640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb54c0d3290 0x7fb54c0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.996+0000 7fb551c9d640 1 -- 172.21.15.55:0/2519336985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c0b50b0 con 0x7fb54c0b4080 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.996+0000 7fb552c9f640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb54c0d3290 0x7fb54c0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.996+0000 7fb551c9d640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb54c0b4080 0x7fb54c147f20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00e960 tx=0x7fb53c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.996+0000 7fb542ffd640 1 -- 172.21.15.55:0/2519336985 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c017070 con 0x7fb54c0b4080 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.996+0000 7fb542ffd640 1 -- 172.21.15.55:0/2519336985 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb53c00c9a0 con 0x7fb54c0b4080 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.996+0000 7fb554702640 1 -- 172.21.15.55:0/2519336985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c0cac50 con 0x7fb54c0b4080 2024-05-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.996+0000 7fb542ffd640 1 -- 172.21.15.55:0/2519336985 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00cd70 con 0x7fb54c0b4080 2024-05-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.996+0000 7fb554702640 1 -- 172.21.15.55:0/2519336985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c1484f0 con 0x7fb54c0b4080 2024-05-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.997+0000 7fb554702640 1 -- 172.21.15.55:0/2519336985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54c069090 con 0x7fb54c0b4080 2024-05-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.997+0000 7fb542ffd640 1 -- 172.21.15.55:0/2519336985 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb53c01a070 con 0x7fb54c0b4080 2024-05-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.998+0000 7fb542ffd640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb53006be60 0x7fb53006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.998+0000 7fb542ffd640 1 -- 172.21.15.55:0/2519336985 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb53c08b540 con 0x7fb54c0b4080 2024-05-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.998+0000 7fb55249e640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb53006be60 0x7fb53006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:58.999+0000 7fb55249e640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb53006be60 0x7fb53006e370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb548000f60 tx=0x7fb548007c00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.002+0000 7fb542ffd640 1 -- 172.21.15.55:0/2519336985 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb53c0599e0 con 0x7fb54c0b4080 2024-05-20T22:08:59.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.205+0000 7fb554702640 1 -- 172.21.15.55:0/2519336985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb54c0cb170 con 0x7fb54c0b4080 2024-05-20T22:08:59.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.205+0000 7fb542ffd640 1 -- 172.21.15.55:0/2519336985 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fb53c0592e0 con 0x7fb54c0b4080 2024-05-20T22:08:59.207 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-20T22:08:59.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 -- 172.21.15.55:0/2519336985 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb53006be60 msgr2=0x7fb53006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb53006be60 0x7fb53006e370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb548000f60 tx=0x7fb548007c00 comp rx=0 tx=0).stop 2024-05-20T22:08:59.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 -- 172.21.15.55:0/2519336985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb54c0b4080 msgr2=0x7fb54c147f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb54c0b4080 0x7fb54c147f20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00e960 tx=0x7fb53c00ee20 comp rx=0 tx=0).stop 2024-05-20T22:08:59.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 -- 172.21.15.55:0/2519336985 shutdown_connections 2024-05-20T22:08:59.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb53006be60 0x7fb53006e370 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb54c0d3290 0x7fb54c0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb54c0b4080 0x7fb54c147f20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 --2- 172.21.15.55:0/2519336985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb54c0b35e0 0x7fb54c0c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 -- 172.21.15.55:0/2519336985 >> 172.21.15.55:0/2519336985 conn(0x7fb54c0d51b0 msgr2=0x7fb54c0ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:59.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 -- 172.21.15.55:0/2519336985 shutdown_connections 2024-05-20T22:08:59.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.208+0000 7fb554702640 1 -- 172.21.15.55:0/2519336985 wait complete. 2024-05-20T22:08:59.220 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-20T22:08:59.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:59.221 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-20T22:08:59.252 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:08:59.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:08:59.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:08:59.252 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-20T22:08:59.312 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-20T22:08:59.312 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:08:59.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.425+0000 7fb86ea56640 1 Processor -- start 2024-05-20T22:08:59.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.425+0000 7fb86ea56640 1 -- start start 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.427+0000 7fb86ea56640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8680b7fe0 0x7fb868169f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.427+0000 7fb86ea56640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8680b83f0 con 0x7fb86815f100 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.427+0000 7fb86ea56640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8680d3dd0 con 0x7fb8680b3b30 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.427+0000 7fb86ea56640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb86806cea0 con 0x7fb8680b7fe0 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.427+0000 7fb85ffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb86815f100 0x7fb86806c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50948/0 (socket says 172.21.15.55:50948) 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.427+0000 7fb85ffff640 1 -- 172.21.15.55:0/2567700306 learned_addr learned my addr 172.21.15.55:0/2567700306 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.427+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677104498 0 0) 0x7fb8680b83f0 con 0x7fb86815f100 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.427+0000 7fb86cff3640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8680b7fe0 0x7fb868169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.427+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb854003520 con 0x7fb86815f100 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1021730387 0 0) 0x7fb854003520 con 0x7fb86815f100 2024-05-20T22:08:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8680b7fe0 msgr2=0x7fb868169f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8680b7fe0 0x7fb868169f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 >> v1:172.21.15.138:6789/0 conn(0x7fb8680b3b30 legacy=0x7fb86806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8680d40b0 con 0x7fb86815f100 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fb850001690 con 0x7fb86815f100 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 >> v1:172.21.15.55:6789/0 conn(0x7fb86815f100 legacy=0x7fb86806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8680b7fe0 0x7fb8540039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb854003f60 0x7fb854004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb854004d40 0x7fb854005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb86806cea0 con 0x7fb854003f60 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb85400bb50 con 0x7fb854004d40 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb85400bd50 con 0x7fb8680b7fe0 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb85ffff640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb854003f60 0x7fb854004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb867fff640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8680b7fe0 0x7fb8540039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb86cff3640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb854004d40 0x7fb854005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb85ffff640 1 -- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8680b7fe0 msgr2=0x7fb8540039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb85ffff640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8680b7fe0 0x7fb8540039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb85ffff640 1 -- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb854004d40 msgr2=0x7fb854005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb85ffff640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb854004d40 0x7fb854005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb85ffff640 1 -- 172.21.15.55:0/2567700306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb854003520 con 0x7fb854003f60 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb867fff640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8680b7fe0 0x7fb8540039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.428+0000 7fb86cff3640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb854004d40 0x7fb854005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.429+0000 7fb85ffff640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb854003f60 0x7fb854004800 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb8680c8380 tx=0x7fb854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97245b114b48393 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.429+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850007d30 con 0x7fb854003f60 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.429+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb850005ad0 con 0x7fb854003f60 2024-05-20T22:08:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.429+0000 7fb8677fe640 1 -- 172.21.15.55:0/2567700306 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850007070 con 0x7fb854003f60 2024-05-20T22:08:59.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.430+0000 7fb86ea56640 1 -- 172.21.15.55:0/2567700306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb854003f60 msgr2=0x7fb854004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.430+0000 7fb86ea56640 1 --2- 172.21.15.55:0/2567700306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb854003f60 0x7fb854004800 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb8680c8380 tx=0x7fb854005830 comp rx=0 tx=0).stop 2024-05-20T22:08:59.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.430+0000 7fb86cff3640 1 -- 172.21.15.55:0/2567700306 reap_dead start 2024-05-20T22:08:59.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.430+0000 7fb86ea56640 1 -- 172.21.15.55:0/2567700306 shutdown_connections 2024-05-20T22:08:59.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.430+0000 7fb86ea56640 1 -- 172.21.15.55:0/2567700306 >> 172.21.15.55:0/2567700306 conn(0x7fb8680d51d0 msgr2=0x7fb8680b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:59.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.430+0000 7fb86ea56640 1 -- 172.21.15.55:0/2567700306 shutdown_connections 2024-05-20T22:08:59.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.430+0000 7fb86ea56640 1 -- 172.21.15.55:0/2567700306 wait complete. 2024-05-20T22:08:59.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.433+0000 7fb86ea56640 1 Processor -- start 2024-05-20T22:08:59.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.433+0000 7fb86ea56640 1 -- start start 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb86ea56640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb854003f60 0x7fb868153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb86ea56640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8680d00b0 0x7fb8681616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb86ea56640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb868161c20 0x7fb868157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb86ea56640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb86806b880 con 0x7fb8680d00b0 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb86ea56640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb86806b5a0 con 0x7fb868161c20 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb86ea56640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb86806bb60 con 0x7fb854003f60 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8680d00b0 0x7fb8681616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8680d00b0 0x7fb8681616e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47788/0 (socket says 172.21.15.55:47788) 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb85ffff640 1 -- 172.21.15.55:0/157263797 learned_addr learned my addr 172.21.15.55:0/157263797 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb86cff3640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb868161c20 0x7fb868157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb867fff640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb854003f60 0x7fb868153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb85ffff640 1 -- 172.21.15.55:0/157263797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb854003f60 msgr2=0x7fb868153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb85ffff640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb854003f60 0x7fb868153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb85ffff640 1 -- 172.21.15.55:0/157263797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb868161c20 msgr2=0x7fb868157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb85ffff640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb868161c20 0x7fb868157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb85ffff640 1 -- 172.21.15.55:0/157263797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb868162030 con 0x7fb8680d00b0 2024-05-20T22:08:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb85ffff640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8680d00b0 0x7fb8681616e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb85002ad80 tx=0x7fb8680d3fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb8657fa640 1 -- 172.21.15.55:0/157263797 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850001650 con 0x7fb8680d00b0 2024-05-20T22:08:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb8657fa640 1 -- 172.21.15.55:0/157263797 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb850042070 con 0x7fb8680d00b0 2024-05-20T22:08:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb86ea56640 1 -- 172.21.15.55:0/157263797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8680d04c0 con 0x7fb8680d00b0 2024-05-20T22:08:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb8657fa640 1 -- 172.21.15.55:0/157263797 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8500072b0 con 0x7fb8680d00b0 2024-05-20T22:08:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.434+0000 7fb86ea56640 1 -- 172.21.15.55:0/157263797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8681589e0 con 0x7fb8680d00b0 2024-05-20T22:08:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.435+0000 7fb86ea56640 1 -- 172.21.15.55:0/157263797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb868069090 con 0x7fb8680d00b0 2024-05-20T22:08:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.436+0000 7fb8657fa640 1 -- 172.21.15.55:0/157263797 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb850007450 con 0x7fb8680d00b0 2024-05-20T22:08:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.438+0000 7fb8657fa640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb84806be30 0x7fb84806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.438+0000 7fb867fff640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb84806be30 0x7fb84806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.438+0000 7fb8657fa640 1 -- 172.21.15.55:0/157263797 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb8500ab390 con 0x7fb8680d00b0 2024-05-20T22:08:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.438+0000 7fb867fff640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb84806be30 0x7fb84806e340 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb86806d0c0 tx=0x7fb858009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.441+0000 7fb8657fa640 1 -- 172.21.15.55:0/157263797 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb868162030 con 0x7fb8680d00b0 2024-05-20T22:08:59.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.665+0000 7fb86ea56640 1 -- 172.21.15.55:0/157263797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8680d2ee0 con 0x7fb8680d00b0 2024-05-20T22:08:59.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.666+0000 7fb8657fa640 1 -- 172.21.15.55:0/157263797 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7fb8500799d0 con 0x7fb8680d00b0 2024-05-20T22:08:59.668 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:59.668 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:07:53.028952+0000","modified":"2024-05-20T22:08:11.135754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-05-20T22:08:59.668 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-20T22:08:59.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 -- 172.21.15.55:0/157263797 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb84806be30 msgr2=0x7fb84806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb84806be30 0x7fb84806e340 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb86806d0c0 tx=0x7fb858009040 comp rx=0 tx=0).stop 2024-05-20T22:08:59.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 -- 172.21.15.55:0/157263797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8680d00b0 msgr2=0x7fb8681616e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8680d00b0 0x7fb8681616e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb85002ad80 tx=0x7fb8680d3fa0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 -- 172.21.15.55:0/157263797 shutdown_connections 2024-05-20T22:08:59.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb84806be30 0x7fb84806e340 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb868161c20 0x7fb868157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8680d00b0 0x7fb8681616e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 --2- 172.21.15.55:0/157263797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb854003f60 0x7fb868153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 -- 172.21.15.55:0/157263797 >> 172.21.15.55:0/157263797 conn(0x7fb8680d51d0 msgr2=0x7fb8680ba2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:59.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 -- 172.21.15.55:0/157263797 shutdown_connections 2024-05-20T22:08:59.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.669+0000 7fb86ea56640 1 -- 172.21.15.55:0/157263797 wait complete. 2024-05-20T22:08:59.681 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:08:59.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.795+0000 7f8d25545640 1 Processor -- start 2024-05-20T22:08:59.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.795+0000 7f8d25545640 1 -- start start 2024-05-20T22:08:59.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.796+0000 7f8d25545640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d2015f140 0x7f8d20169f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.796+0000 7f8d25545640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d20163990 con 0x7f8d200b3b50 2024-05-20T22:08:59.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.796+0000 7f8d25545640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d200d3ff0 con 0x7f8d200b8090 2024-05-20T22:08:59.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.796+0000 7f8d25545640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f8d200b43a0 con 0x7f8d2015f140 2024-05-20T22:08:59.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.796+0000 7f8d1f7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d2015f140 0x7f8d20169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.796+0000 7f8d1e7fc640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8d200b3b50 0x7f8d20163280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50950/0 (socket says 172.21.15.55:50950) 2024-05-20T22:08:59.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.796+0000 7f8d1e7fc640 1 -- 172.21.15.55:0/2452421437 learned_addr learned my addr 172.21.15.55:0/2452421437 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:08:59.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.797+0000 7f8d1f7fe640 1 -- 172.21.15.55:0/2452421437 >> v1:172.21.15.138:6789/0 conn(0x7f8d200b8090 legacy=0x7f8d20162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.797+0000 7f8d1dffb640 1 -- 172.21.15.55:0/2452421437 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3805200410 0 0) 0x7f8d20163990 con 0x7f8d200b3b50 2024-05-20T22:08:59.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.797+0000 7f8d1f7fe640 1 -- 172.21.15.55:0/2452421437 >> v1:172.21.15.55:6789/0 conn(0x7f8d200b3b50 legacy=0x7f8d20163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.797+0000 7f8d1f7fe640 1 -- 172.21.15.55:0/2452421437 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d200d42d0 con 0x7f8d2015f140 2024-05-20T22:08:59.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.798+0000 7f8d1f7fe640 1 --2- 172.21.15.55:0/2452421437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 0x7f8d20169f00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8d100097a0 tx=0x7f8d1002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533d36786295968 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:59.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.798+0000 7f8d1dffb640 1 -- 172.21.15.55:0/2452421437 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10037070 con 0x7f8d2015f140 2024-05-20T22:08:59.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.798+0000 7f8d1dffb640 1 -- 172.21.15.55:0/2452421437 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d10032440 con 0x7f8d2015f140 2024-05-20T22:08:59.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.798+0000 7f8d1dffb640 1 -- 172.21.15.55:0/2452421437 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10032810 con 0x7f8d2015f140 2024-05-20T22:08:59.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.799+0000 7f8d25545640 1 -- 172.21.15.55:0/2452421437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 msgr2=0x7f8d20169f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.799+0000 7f8d25545640 1 --2- 172.21.15.55:0/2452421437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 0x7f8d20169f00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8d100097a0 tx=0x7f8d1002a230 comp rx=0 tx=0).stop 2024-05-20T22:08:59.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.799+0000 7f8d25545640 1 -- 172.21.15.55:0/2452421437 shutdown_connections 2024-05-20T22:08:59.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.799+0000 7f8d25545640 1 --2- 172.21.15.55:0/2452421437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 0x7f8d20169f00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.799+0000 7f8d25545640 1 -- 172.21.15.55:0/2452421437 >> 172.21.15.55:0/2452421437 conn(0x7f8d200d1bb0 msgr2=0x7f8d200d4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:59.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.799+0000 7f8d25545640 1 -- 172.21.15.55:0/2452421437 shutdown_connections 2024-05-20T22:08:59.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.799+0000 7f8d25545640 1 -- 172.21.15.55:0/2452421437 wait complete. 2024-05-20T22:08:59.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.801+0000 7f8d25545640 1 Processor -- start 2024-05-20T22:08:59.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.801+0000 7f8d25545640 1 -- start start 2024-05-20T22:08:59.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d25545640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d200b3b50 0x7f8d201620b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d25545640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d200b8090 0x7f8d20160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d25545640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 0x7f8d2014df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d25545640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2006b960 con 0x7f8d2015f140 2024-05-20T22:08:59.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d25545640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2006ce90 con 0x7f8d200b8090 2024-05-20T22:08:59.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d25545640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d2016a440 con 0x7f8d200b3b50 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 0x7f8d2014df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 0x7f8d2014df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47794/0 (socket says 172.21.15.55:47794) 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1f7fe640 1 -- 172.21.15.55:0/1564037150 learned_addr learned my addr 172.21.15.55:0/1564037150 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1f7fe640 1 -- 172.21.15.55:0/1564037150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d200b3b50 msgr2=0x7f8d201620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1effd640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d200b3b50 0x7f8d201620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1e7fc640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d200b8090 0x7f8d20160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1f7fe640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d200b3b50 0x7f8d201620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1f7fe640 1 -- 172.21.15.55:0/1564037150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d200b8090 msgr2=0x7f8d20160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1f7fe640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d200b8090 0x7f8d20160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1f7fe640 1 -- 172.21.15.55:0/1564037150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d20160f80 con 0x7f8d2015f140 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.802+0000 7f8d1effd640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d200b3b50 0x7f8d201620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:08:59.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.803+0000 7f8d1f7fe640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 0x7f8d2014df00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8d100097a0 tx=0x7f8d100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:59.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.803+0000 7f8d177fe640 1 -- 172.21.15.55:0/1564037150 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10037040 con 0x7f8d2015f140 2024-05-20T22:08:59.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.803+0000 7f8d177fe640 1 -- 172.21.15.55:0/1564037150 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d10042070 con 0x7f8d2015f140 2024-05-20T22:08:59.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.803+0000 7f8d25545640 1 -- 172.21.15.55:0/1564037150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d20161260 con 0x7f8d2015f140 2024-05-20T22:08:59.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.803+0000 7f8d177fe640 1 -- 172.21.15.55:0/1564037150 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10030b30 con 0x7f8d2015f140 2024-05-20T22:08:59.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.803+0000 7f8d25545640 1 -- 172.21.15.55:0/1564037150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d20157520 con 0x7f8d2015f140 2024-05-20T22:08:59.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.804+0000 7f8d25545640 1 -- 172.21.15.55:0/1564037150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d20069090 con 0x7f8d2015f140 2024-05-20T22:08:59.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.804+0000 7f8d177fe640 1 -- 172.21.15.55:0/1564037150 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d10030cd0 con 0x7f8d2015f140 2024-05-20T22:08:59.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.805+0000 7f8d177fe640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d0006be30 0x7f8d0006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:08:59.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.805+0000 7f8d177fe640 1 -- 172.21.15.55:0/1564037150 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8d100abd50 con 0x7f8d2015f140 2024-05-20T22:08:59.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.805+0000 7f8d1effd640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d0006be30 0x7f8d0006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:08:59.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.806+0000 7f8d1effd640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d0006be30 0x7f8d0006e340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d180012e0 tx=0x7f8d18009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:08:59.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.810+0000 7f8d177fe640 1 -- 172.21.15.55:0/1564037150 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8d100af030 con 0x7f8d2015f140 2024-05-20T22:08:59.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.993+0000 7f8d25545640 1 -- 172.21.15.55:0/1564037150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d200d4d90 con 0x7f8d2015f140 2024-05-20T22:08:59.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.994+0000 7f8d177fe640 1 -- 172.21.15.55:0/1564037150 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7f8d1004c020 con 0x7f8d2015f140 2024-05-20T22:08:59.995 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:08:59.996 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:08:59.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 -- 172.21.15.55:0/1564037150 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d0006be30 msgr2=0x7f8d0006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d0006be30 0x7f8d0006e340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d180012e0 tx=0x7f8d18009040 comp rx=0 tx=0).stop 2024-05-20T22:08:59.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 -- 172.21.15.55:0/1564037150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 msgr2=0x7f8d2014df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:08:59.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 0x7f8d2014df00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8d100097a0 tx=0x7f8d100046c0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 -- 172.21.15.55:0/1564037150 shutdown_connections 2024-05-20T22:08:59.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8d0006be30 0x7f8d0006e340 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2015f140 0x7f8d2014df00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d200b8090 0x7f8d20160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 --2- 172.21.15.55:0/1564037150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d200b3b50 0x7f8d201620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:08:59.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 -- 172.21.15.55:0/1564037150 >> 172.21.15.55:0/1564037150 conn(0x7f8d200d1bb0 msgr2=0x7f8d200b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:08:59.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 -- 172.21.15.55:0/1564037150 shutdown_connections 2024-05-20T22:08:59.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:08:59.996+0000 7f8d25545640 1 -- 172.21.15.55:0/1564037150 wait complete. 2024-05-20T22:09:00.009 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:07:51.275733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:07:52.008991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T22:07:55.711810+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T22:09:00.009 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:09:00.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.126+0000 7f48fc5b5640 1 Processor -- start 2024-05-20T22:09:00.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.126+0000 7f48fc5b5640 1 -- start start 2024-05-20T22:09:00.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fc5b5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48f40d32a0 0x7f48f415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fc5b5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f48f40cf690 0x7f48f40cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fc5b5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40cffe0 0x7f48f4162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fc5b5640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f48f40d3780 con 0x7f48f40d32a0 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fc5b5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f48f40d3a60 con 0x7f48f40cf690 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fc5b5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48f40d3d40 con 0x7f48f40cffe0 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fa351640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48f40d32a0 0x7f48f415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fa351640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48f40d32a0 0x7f48f415f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47796/0 (socket says 172.21.15.55:47796) 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fa351640 1 -- 172.21.15.55:0/2484413654 learned_addr learned my addr 172.21.15.55:0/2484413654 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48f9b50640 1 --2- 172.21.15.55:0/2484413654 >> v2:172.21.15.138:3300/0 conn(0x7f48f40cf690 0x7f48f40cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fa351640 1 -- 172.21.15.55:0/2484413654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40cffe0 msgr2=0x7f48f4162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fab52640 1 --2- 172.21.15.55:0/2484413654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40cffe0 0x7f48f4162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fa351640 1 --2- 172.21.15.55:0/2484413654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40cffe0 0x7f48f4162cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fa351640 1 -- 172.21.15.55:0/2484413654 >> v2:172.21.15.138:3300/0 conn(0x7f48f40cf690 msgr2=0x7f48f40cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:00.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fa351640 1 --2- 172.21.15.55:0/2484413654 >> v2:172.21.15.138:3300/0 conn(0x7f48f40cf690 0x7f48f40cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.128+0000 7f48fa351640 1 -- 172.21.15.55:0/2484413654 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f40d4020 con 0x7f48f40d32a0 2024-05-20T22:09:00.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.129+0000 7f48f9b50640 1 --2- 172.21.15.55:0/2484413654 >> v2:172.21.15.138:3300/0 conn(0x7f48f40cf690 0x7f48f40cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:00.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.129+0000 7f48fab52640 1 --2- 172.21.15.55:0/2484413654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40cffe0 0x7f48f4162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:00.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.129+0000 7f48fa351640 1 --2- 172.21.15.55:0/2484413654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40d32a0 0x7f48f415f150 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f48f408e150 tx=0x7f48e402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db3cb781719e7a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:00.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.129+0000 7f48f934f640 1 -- 172.21.15.55:0/2484413654 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4036070 con 0x7f48f40d32a0 2024-05-20T22:09:00.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.129+0000 7f48f934f640 1 -- 172.21.15.55:0/2484413654 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e4031600 con 0x7f48f40d32a0 2024-05-20T22:09:00.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.129+0000 7f48f934f640 1 -- 172.21.15.55:0/2484413654 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e40319d0 con 0x7f48f40d32a0 2024-05-20T22:09:00.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.130+0000 7f48fc5b5640 1 -- 172.21.15.55:0/2484413654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40d32a0 msgr2=0x7f48f415f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.130+0000 7f48fc5b5640 1 --2- 172.21.15.55:0/2484413654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40d32a0 0x7f48f415f150 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f48f408e150 tx=0x7f48e402a6a0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.130+0000 7f48fc5b5640 1 -- 172.21.15.55:0/2484413654 shutdown_connections 2024-05-20T22:09:00.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.130+0000 7f48fc5b5640 1 --2- 172.21.15.55:0/2484413654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40cffe0 0x7f48f4162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.130+0000 7f48fc5b5640 1 --2- 172.21.15.55:0/2484413654 >> v2:172.21.15.138:3300/0 conn(0x7f48f40cf690 0x7f48f40cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.130+0000 7f48fc5b5640 1 --2- 172.21.15.55:0/2484413654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40d32a0 0x7f48f415f150 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.130+0000 7f48fc5b5640 1 -- 172.21.15.55:0/2484413654 >> 172.21.15.55:0/2484413654 conn(0x7f48f40b6de0 msgr2=0x7f48f40d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:00.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.130+0000 7f48fc5b5640 1 -- 172.21.15.55:0/2484413654 shutdown_connections 2024-05-20T22:09:00.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.130+0000 7f48fc5b5640 1 -- 172.21.15.55:0/2484413654 wait complete. 2024-05-20T22:09:00.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.132+0000 7f48fc5b5640 1 Processor -- start 2024-05-20T22:09:00.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.132+0000 7f48fc5b5640 1 -- start start 2024-05-20T22:09:00.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.133+0000 7f48fc5b5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40cf690 0x7f48f4152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.133+0000 7f48fc5b5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f40cffe0 0x7f48f41f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.133+0000 7f48fc5b5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40d32a0 0x7f48f40c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.133+0000 7f48fc5b5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f48f40d4820 con 0x7f48f40cf690 2024-05-20T22:09:00.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.133+0000 7f48fc5b5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48f40d03f0 con 0x7f48f40cffe0 2024-05-20T22:09:00.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.133+0000 7f48fc5b5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48f4160b30 con 0x7f48f40d32a0 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.133+0000 7f48fa351640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40cf690 0x7f48f4152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.133+0000 7f48fa351640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40cf690 0x7f48f4152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47808/0 (socket says 172.21.15.55:47808) 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.133+0000 7f48fa351640 1 -- 172.21.15.55:0/396750317 learned_addr learned my addr 172.21.15.55:0/396750317 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48fa351640 1 -- 172.21.15.55:0/396750317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40d32a0 msgr2=0x7f48f40c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48fa351640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40d32a0 0x7f48f40c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48fa351640 1 -- 172.21.15.55:0/396750317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f40cffe0 msgr2=0x7f48f41f9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48fa351640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f40cffe0 0x7f48f41f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48fa351640 1 -- 172.21.15.55:0/396750317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f40ca2f0 con 0x7f48f40cf690 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48fa351640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40cf690 0x7f48f4152c40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f48e4034040 tx=0x7f48e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48f2ffd640 1 -- 172.21.15.55:0/396750317 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4036050 con 0x7f48f40cf690 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48f2ffd640 1 -- 172.21.15.55:0/396750317 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e4042070 con 0x7f48f40cf690 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48fc5b5640 1 -- 172.21.15.55:0/396750317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f40ca5c0 con 0x7f48f40cf690 2024-05-20T22:09:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48f2ffd640 1 -- 172.21.15.55:0/396750317 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4009510 con 0x7f48f40cf690 2024-05-20T22:09:00.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.134+0000 7f48fc5b5640 1 -- 172.21.15.55:0/396750317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f40cab80 con 0x7f48f40cf690 2024-05-20T22:09:00.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.135+0000 7f48fc5b5640 1 -- 172.21.15.55:0/396750317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f40cad80 con 0x7f48f40cf690 2024-05-20T22:09:00.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.136+0000 7f48f2ffd640 1 -- 172.21.15.55:0/396750317 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48e4007c30 con 0x7f48f40cf690 2024-05-20T22:09:00.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.136+0000 7f48f2ffd640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48e006be60 0x7f48e006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.136+0000 7f48f2ffd640 1 -- 172.21.15.55:0/396750317 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f48e40ac3a0 con 0x7f48f40cf690 2024-05-20T22:09:00.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.136+0000 7f48f9b50640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48e006be60 0x7f48e006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.137+0000 7f48f9b50640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48e006be60 0x7f48e006e370 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f48e8006b10 tx=0x7f48e8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:00.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.141+0000 7f48f2ffd640 1 -- 172.21.15.55:0/396750317 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f48f40cad80 con 0x7f48f40cf690 2024-05-20T22:09:00.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.327+0000 7f48fc5b5640 1 -- 172.21.15.55:0/396750317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48f415f150 con 0x7f48f40cf690 2024-05-20T22:09:00.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.327+0000 7f48f2ffd640 1 -- 172.21.15.55:0/396750317 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f48e4045360 con 0x7f48f40cf690 2024-05-20T22:09:00.328 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 -- 172.21.15.55:0/396750317 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48e006be60 msgr2=0x7f48e006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48e006be60 0x7f48e006e370 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f48e8006b10 tx=0x7f48e8008040 comp rx=0 tx=0).stop 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 -- 172.21.15.55:0/396750317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40cf690 msgr2=0x7f48f4152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40cf690 0x7f48f4152c40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f48e4034040 tx=0x7f48e4004a40 comp rx=0 tx=0).stop 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 -- 172.21.15.55:0/396750317 shutdown_connections 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48e006be60 0x7f48e006e370 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f40d32a0 0x7f48f40c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f40cffe0 0x7f48f41f9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 --2- 172.21.15.55:0/396750317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f40cf690 0x7f48f4152c40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 -- 172.21.15.55:0/396750317 >> 172.21.15.55:0/396750317 conn(0x7f48f40b6de0 msgr2=0x7f48f415ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:00.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 -- 172.21.15.55:0/396750317 shutdown_connections 2024-05-20T22:09:00.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.330+0000 7f48fc5b5640 1 -- 172.21.15.55:0/396750317 wait complete. 2024-05-20T22:09:00.342 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:09:00.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.459+0000 7fa18e837640 1 Processor -- start 2024-05-20T22:09:00.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.460+0000 7fa18e837640 1 -- start start 2024-05-20T22:09:00.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa18e837640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa1880d32a0 0x7fa18815f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa18e837640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cf690 0x7fa1880cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa18e837640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1880d3780 con 0x7fa1880cffe0 2024-05-20T22:09:00.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa18e837640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa18806c510 con 0x7fa1880d32a0 2024-05-20T22:09:00.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa18e837640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa1880d3a60 con 0x7fa1880cf690 2024-05-20T22:09:00.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa18cdd4640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa1880cffe0 0x7fa188162cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50958/0 (socket says 172.21.15.55:50958) 2024-05-20T22:09:00.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa18cdd4640 1 -- 172.21.15.55:0/3360456385 learned_addr learned my addr 172.21.15.55:0/3360456385 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:00.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa1877fe640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cf690 0x7fa1880cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa187fff640 1 --2- 172.21.15.55:0/3360456385 >> v2:172.21.15.138:3300/0 conn(0x7fa1880d32a0 0x7fa18815f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3342009882 0 0) 0x7fa1880d3780 con 0x7fa1880cffe0 2024-05-20T22:09:00.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.461+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa170003520 con 0x7fa1880cffe0 2024-05-20T22:09:00.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 703195182 0 0) 0x7fa170003520 con 0x7fa1880cffe0 2024-05-20T22:09:00.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cf690 msgr2=0x7fa1880cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:00.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cf690 0x7fa1880cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 >> v2:172.21.15.138:3300/0 conn(0x7fa1880d32a0 msgr2=0x7fa18815f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:00.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 --2- 172.21.15.55:0/3360456385 >> v2:172.21.15.138:3300/0 conn(0x7fa1880d32a0 0x7fa18815f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1880d3d40 con 0x7fa1880cffe0 2024-05-20T22:09:00.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fa1800019f0 con 0x7fa1880cffe0 2024-05-20T22:09:00.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa1877fe640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cf690 0x7fa1880cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:09:00.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 >> v1:172.21.15.55:6789/0 conn(0x7fa1880cffe0 legacy=0x7fa188162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cf690 0x7fa1700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa170003f10 0x7fa170004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa170004d40 0x7fa170005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa18806c510 con 0x7fa170004d40 2024-05-20T22:09:00.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1880d3a60 con 0x7fa170003f10 2024-05-20T22:09:00.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa17000b9f0 con 0x7fa1880cf690 2024-05-20T22:09:00.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa187fff640 1 --2- 172.21.15.55:0/3360456385 >> v2:172.21.15.138:3300/0 conn(0x7fa1880d32a0 0x7fa18815f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:09:00.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa18cdd4640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa170004d40 0x7fa170005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa1877fe640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa170003f10 0x7fa170004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa187fff640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cf690 0x7fa1700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa18cdd4640 1 -- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cf690 msgr2=0x7fa1700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:00.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa18cdd4640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cf690 0x7fa1700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa18cdd4640 1 -- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa170003f10 msgr2=0x7fa170004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:00.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa18cdd4640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa170003f10 0x7fa170004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.462+0000 7fa18cdd4640 1 -- 172.21.15.55:0/3360456385 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa170003520 con 0x7fa170004d40 2024-05-20T22:09:00.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.463+0000 7fa18cdd4640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa170004d40 0x7fa170005150 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa1880b7830 tx=0x7fa170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbeae000428873b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:00.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.463+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18000a4d0 con 0x7fa170004d40 2024-05-20T22:09:00.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.463+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18002f070 con 0x7fa170004d40 2024-05-20T22:09:00.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.463+0000 7fa186ffd640 1 -- 172.21.15.55:0/3360456385 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18002bca0 con 0x7fa170004d40 2024-05-20T22:09:00.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.464+0000 7fa18e837640 1 -- 172.21.15.55:0/3360456385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa170004d40 msgr2=0x7fa170005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.464+0000 7fa18e837640 1 --2- 172.21.15.55:0/3360456385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa170004d40 0x7fa170005150 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa1880b7830 tx=0x7fa170005830 comp rx=0 tx=0).stop 2024-05-20T22:09:00.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.464+0000 7fa18cdd4640 1 -- 172.21.15.55:0/3360456385 reap_dead start 2024-05-20T22:09:00.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.464+0000 7fa18e837640 1 -- 172.21.15.55:0/3360456385 shutdown_connections 2024-05-20T22:09:00.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.464+0000 7fa18e837640 1 -- 172.21.15.55:0/3360456385 >> 172.21.15.55:0/3360456385 conn(0x7fa1880b6de0 msgr2=0x7fa1880d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:00.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.464+0000 7fa18e837640 1 -- 172.21.15.55:0/3360456385 shutdown_connections 2024-05-20T22:09:00.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.464+0000 7fa18e837640 1 -- 172.21.15.55:0/3360456385 wait complete. 2024-05-20T22:09:00.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.466+0000 7fa18e837640 1 Processor -- start 2024-05-20T22:09:00.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.466+0000 7fa18e837640 1 -- start start 2024-05-20T22:09:00.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.467+0000 7fa18e837640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa170004d40 0x7fa188161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.467+0000 7fa18e837640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cd700 0x7fa1880cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.467+0000 7fa18e837640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1880ce050 0x7fa188160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.467+0000 7fa18e837640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa1880d03f0 con 0x7fa1880ce050 2024-05-20T22:09:00.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.467+0000 7fa18e837640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1880d4300 con 0x7fa170004d40 2024-05-20T22:09:00.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.467+0000 7fa18e837640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa1880d4820 con 0x7fa1880cd700 2024-05-20T22:09:00.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.467+0000 7fa18cdd4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1880ce050 0x7fa188160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.467+0000 7fa18cdd4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1880ce050 0x7fa188160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47828/0 (socket says 172.21.15.55:47828) 2024-05-20T22:09:00.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa18cdd4640 1 -- 172.21.15.55:0/2097339241 learned_addr learned my addr 172.21.15.55:0/2097339241 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:00.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa187fff640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa170004d40 0x7fa188161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa1877fe640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cd700 0x7fa1880cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa18cdd4640 1 -- 172.21.15.55:0/2097339241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cd700 msgr2=0x7fa1880cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa18cdd4640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cd700 0x7fa1880cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa18cdd4640 1 -- 172.21.15.55:0/2097339241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa170004d40 msgr2=0x7fa188161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa18cdd4640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa170004d40 0x7fa188161bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa18cdd4640 1 -- 172.21.15.55:0/2097339241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa188147e60 con 0x7fa1880ce050 2024-05-20T22:09:00.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa18cdd4640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1880ce050 0x7fa188160790 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa18002b4d0 tx=0x7fa1880d3f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:00.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa184ff9640 1 -- 172.21.15.55:0/2097339241 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18000a4d0 con 0x7fa1880ce050 2024-05-20T22:09:00.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa184ff9640 1 -- 172.21.15.55:0/2097339241 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa180004d70 con 0x7fa1880ce050 2024-05-20T22:09:00.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa184ff9640 1 -- 172.21.15.55:0/2097339241 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180002020 con 0x7fa1880ce050 2024-05-20T22:09:00.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa18e837640 1 -- 172.21.15.55:0/2097339241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1880cb3d0 con 0x7fa1880ce050 2024-05-20T22:09:00.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.468+0000 7fa18e837640 1 -- 172.21.15.55:0/2097339241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1880ccf00 con 0x7fa1880ce050 2024-05-20T22:09:00.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.469+0000 7fa18e837640 1 -- 172.21.15.55:0/2097339241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1880cb140 con 0x7fa1880ce050 2024-05-20T22:09:00.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.470+0000 7fa184ff9640 1 -- 172.21.15.55:0/2097339241 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa18003e030 con 0x7fa1880ce050 2024-05-20T22:09:00.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.471+0000 7fa184ff9640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa17406be30 0x7fa17406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.471+0000 7fa184ff9640 1 -- 172.21.15.55:0/2097339241 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa1800aab10 con 0x7fa1880ce050 2024-05-20T22:09:00.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.471+0000 7fa187fff640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa17406be30 0x7fa17406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.471+0000 7fa187fff640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa17406be30 0x7fa17406e340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa178009d10 tx=0x7fa178008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:00.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.475+0000 7fa184ff9640 1 -- 172.21.15.55:0/2097339241 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa180079030 con 0x7fa1880ce050 2024-05-20T22:09:00.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.660+0000 7fa18e837640 1 -- 172.21.15.55:0/2097339241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa188069090 con 0x7fa1880ce050 2024-05-20T22:09:00.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.660+0000 7fa184ff9640 1 -- 172.21.15.55:0/2097339241 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fa180078930 con 0x7fa1880ce050 2024-05-20T22:09:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:09:00.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.662+0000 7fa18e837640 1 -- 172.21.15.55:0/2097339241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa17406be30 msgr2=0x7fa17406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.662+0000 7fa18e837640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa17406be30 0x7fa17406e340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa178009d10 tx=0x7fa178008040 comp rx=0 tx=0).stop 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.662+0000 7fa18e837640 1 -- 172.21.15.55:0/2097339241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1880ce050 msgr2=0x7fa188160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.662+0000 7fa18e837640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1880ce050 0x7fa188160790 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa18002b4d0 tx=0x7fa1880d3f10 comp rx=0 tx=0).stop 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.662+0000 7fa18e837640 1 -- 172.21.15.55:0/2097339241 shutdown_connections 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.662+0000 7fa18e837640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa17406be30 0x7fa17406e340 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.662+0000 7fa18e837640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1880ce050 0x7fa188160790 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.662+0000 7fa18e837640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1880cd700 0x7fa1880cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.662+0000 7fa18e837640 1 --2- 172.21.15.55:0/2097339241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa170004d40 0x7fa188161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.662+0000 7fa18e837640 1 -- 172.21.15.55:0/2097339241 >> 172.21.15.55:0/2097339241 conn(0x7fa1880b6de0 msgr2=0x7fa1880b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.663+0000 7fa18e837640 1 -- 172.21.15.55:0/2097339241 shutdown_connections 2024-05-20T22:09:00.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.663+0000 7fa18e837640 1 -- 172.21.15.55:0/2097339241 wait complete. 2024-05-20T22:09:00.675 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:09:00.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.792+0000 7f6c0233e640 1 Processor -- start 2024-05-20T22:09:00.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.792+0000 7f6c0233e640 1 -- start start 2024-05-20T22:09:00.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6c0233e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6bfc0b4080 0x7f6bfc15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6c0233e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bfc0b35e0 0x7f6bfc15f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6c0233e640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bfc06b500 con 0x7f6bfc15fd60 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6c0233e640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f6bfc163960 con 0x7f6bfc0b4080 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6c0233e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6bfc0b3e10 con 0x7f6bfc0b35e0 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6bfbfff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6bfc0b4080 0x7f6bfc15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6c008db640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6bfc15fd60 0x7f6bfc0cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50964/0 (socket says 172.21.15.55:50964) 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6c008db640 1 -- 172.21.15.55:0/3632458814 learned_addr learned my addr 172.21.15.55:0/3632458814 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6bfb7fe640 1 --2- 172.21.15.55:0/3632458814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bfc0b35e0 0x7f6bfc15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6bfbfff640 1 -- 172.21.15.55:0/3632458814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bfc0b35e0 msgr2=0x7f6bfc15f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6bfbfff640 1 --2- 172.21.15.55:0/3632458814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bfc0b35e0 0x7f6bfc15f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.793+0000 7f6bfbfff640 1 -- 172.21.15.55:0/3632458814 >> v1:172.21.15.55:6789/0 conn(0x7f6bfc15fd60 legacy=0x7f6bfc0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.794+0000 7f6bfaffd640 1 -- 172.21.15.55:0/3632458814 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620623772 0 0) 0x7f6bf4001490 con 0x7f6bfc15fd60 2024-05-20T22:09:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.794+0000 7f6bfbfff640 1 -- 172.21.15.55:0/3632458814 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc0b3b30 con 0x7f6bfc0b4080 2024-05-20T22:09:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.794+0000 7f6bfbfff640 1 --2- 172.21.15.55:0/3632458814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 0x7f6bfc15f2e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0097a0 tx=0x7f6bec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a44eefc1e70af5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.794+0000 7f6bfaffd640 1 -- 172.21.15.55:0/3632458814 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec037070 con 0x7f6bfc0b4080 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.794+0000 7f6bfaffd640 1 -- 172.21.15.55:0/3632458814 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bec032440 con 0x7f6bfc0b4080 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.794+0000 7f6bfaffd640 1 -- 172.21.15.55:0/3632458814 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec032810 con 0x7f6bfc0b4080 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.795+0000 7f6c0233e640 1 -- 172.21.15.55:0/3632458814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 msgr2=0x7f6bfc15f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.795+0000 7f6c0233e640 1 --2- 172.21.15.55:0/3632458814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 0x7f6bfc15f2e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0097a0 tx=0x7f6bec02a180 comp rx=0 tx=0).stop 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.795+0000 7f6c0233e640 1 -- 172.21.15.55:0/3632458814 shutdown_connections 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.795+0000 7f6c0233e640 1 --2- 172.21.15.55:0/3632458814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bfc0b35e0 0x7f6bfc15f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.795+0000 7f6c0233e640 1 --2- 172.21.15.55:0/3632458814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 0x7f6bfc15f2e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.795+0000 7f6c0233e640 1 -- 172.21.15.55:0/3632458814 >> 172.21.15.55:0/3632458814 conn(0x7f6bfc0d51b0 msgr2=0x7f6bfc0b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.795+0000 7f6c0233e640 1 -- 172.21.15.55:0/3632458814 shutdown_connections 2024-05-20T22:09:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.795+0000 7f6c0233e640 1 -- 172.21.15.55:0/3632458814 wait complete. 2024-05-20T22:09:00.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.797+0000 7f6c0233e640 1 Processor -- start 2024-05-20T22:09:00.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.797+0000 7f6c0233e640 1 -- start start 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.798+0000 7f6c0233e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bfc0b35e0 0x7f6bfc0b9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6c0233e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 0x7f6bfc0b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6c0233e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc15fd60 0x7f6bfc0b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6c0233e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc1635f0 con 0x7f6bfc0b4080 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6c0233e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc1604f0 con 0x7f6bfc15fd60 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6c0233e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6bfc0d42a0 con 0x7f6bfc0b35e0 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bfb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 0x7f6bfc0b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bfb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 0x7f6bfc0b7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47846/0 (socket says 172.21.15.55:47846) 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bfb7fe640 1 -- 172.21.15.55:0/908327867 learned_addr learned my addr 172.21.15.55:0/908327867 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bfbfff640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bfc0b35e0 0x7f6bfc0b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6c008db640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc15fd60 0x7f6bfc0b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bfb7fe640 1 -- 172.21.15.55:0/908327867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bfc0b35e0 msgr2=0x7f6bfc0b9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bfb7fe640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bfc0b35e0 0x7f6bfc0b9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bfb7fe640 1 -- 172.21.15.55:0/908327867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc15fd60 msgr2=0x7f6bfc0b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bfb7fe640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc15fd60 0x7f6bfc0b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bfb7fe640 1 -- 172.21.15.55:0/908327867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc1574a0 con 0x7f6bfc0b4080 2024-05-20T22:09:00.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bfb7fe640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 0x7f6bfc0b7f70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000c7f0 tx=0x7f6bf000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:00.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bf8ff9640 1 -- 172.21.15.55:0/908327867 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0015070 con 0x7f6bfc0b4080 2024-05-20T22:09:00.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bf8ff9640 1 -- 172.21.15.55:0/908327867 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf0004220 con 0x7f6bfc0b4080 2024-05-20T22:09:00.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6bf8ff9640 1 -- 172.21.15.55:0/908327867 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0003610 con 0x7f6bfc0b4080 2024-05-20T22:09:00.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6c0233e640 1 -- 172.21.15.55:0/908327867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bfc0c8e00 con 0x7f6bfc0b4080 2024-05-20T22:09:00.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.799+0000 7f6c0233e640 1 -- 172.21.15.55:0/908327867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bfc1e8c10 con 0x7f6bfc0b4080 2024-05-20T22:09:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.800+0000 7f6c0233e640 1 -- 172.21.15.55:0/908327867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bfc144b30 con 0x7f6bfc0b4080 2024-05-20T22:09:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.801+0000 7f6bf8ff9640 1 -- 172.21.15.55:0/908327867 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bf00039c0 con 0x7f6bfc0b4080 2024-05-20T22:09:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.802+0000 7f6bf8ff9640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6be806be60 0x7f6be806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.802+0000 7f6bfbfff640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6be806be60 0x7f6be806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.802+0000 7f6bf8ff9640 1 -- 172.21.15.55:0/908327867 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6bf008b230 con 0x7f6bfc0b4080 2024-05-20T22:09:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.802+0000 7f6bfbfff640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6be806be60 0x7f6be806e370 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6bec034000 tx=0x7f6bec02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.806+0000 7f6bf8ff9640 1 -- 172.21.15.55:0/908327867 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6bfc1574a0 con 0x7f6bfc0b4080 2024-05-20T22:09:00.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.990+0000 7f6c0233e640 1 -- 172.21.15.55:0/908327867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6bfc069090 con 0x7f6bfc0b4080 2024-05-20T22:09:00.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.990+0000 7f6bf8ff9640 1 -- 172.21.15.55:0/908327867 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f6bf00590f0 con 0x7f6bfc0b4080 2024-05-20T22:09:00.991 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:09:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 -- 172.21.15.55:0/908327867 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6be806be60 msgr2=0x7f6be806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6be806be60 0x7f6be806e370 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6bec034000 tx=0x7f6bec02e000 comp rx=0 tx=0).stop 2024-05-20T22:09:00.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 -- 172.21.15.55:0/908327867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 msgr2=0x7f6bfc0b7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:00.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 0x7f6bfc0b7f70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000c7f0 tx=0x7f6bf000ccb0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 -- 172.21.15.55:0/908327867 shutdown_connections 2024-05-20T22:09:00.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6be806be60 0x7f6be806e370 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc15fd60 0x7f6bfc0b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bfc0b4080 0x7f6bfc0b7f70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 --2- 172.21.15.55:0/908327867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bfc0b35e0 0x7f6bfc0b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:00.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 -- 172.21.15.55:0/908327867 >> 172.21.15.55:0/908327867 conn(0x7f6bfc0d51b0 msgr2=0x7f6bfc1610f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:00.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 -- 172.21.15.55:0/908327867 shutdown_connections 2024-05-20T22:09:00.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:00.992+0000 7f6c0233e640 1 -- 172.21.15.55:0/908327867 wait complete. 2024-05-20T22:09:01.005 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T22:09:01.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.122+0000 7f98b19b7640 1 Processor -- start 2024-05-20T22:09:01.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.122+0000 7f98b19b7640 1 -- start start 2024-05-20T22:09:01.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.123+0000 7f98b19b7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98ac0cd520 0x7f98ac08c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.123+0000 7f98b19b7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac0cf270 0x7f98ac0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.123+0000 7f98b19b7640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ac0cfbc0 con 0x7f98ac0d0860 2024-05-20T22:09:01.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.123+0000 7f98b19b7640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f98ac0d1020 con 0x7f98ac0cd520 2024-05-20T22:09:01.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.123+0000 7f98b19b7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98ac0d1300 con 0x7f98ac0cf270 2024-05-20T22:09:01.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.123+0000 7f98aaffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98ac0cd520 0x7f98ac08c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.123+0000 7f98aaffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98ac0cd520 0x7f98ac08c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47848/0 (socket says 172.21.15.55:47848) 2024-05-20T22:09:01.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.123+0000 7f98aaffd640 1 -- 172.21.15.55:0/1530901887 learned_addr learned my addr 172.21.15.55:0/1530901887 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:01.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.124+0000 7f98ab7fe640 1 --2- 172.21.15.55:0/1530901887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac0cf270 0x7f98ac0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.124+0000 7f98aaffd640 1 -- 172.21.15.55:0/1530901887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac0cf270 msgr2=0x7f98ac0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:01.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.124+0000 7f98aaffd640 1 --2- 172.21.15.55:0/1530901887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac0cf270 0x7f98ac0cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.124+0000 7f98aaffd640 1 -- 172.21.15.55:0/1530901887 >> v1:172.21.15.55:6789/0 conn(0x7f98ac0d0860 legacy=0x7f98ac0b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.124+0000 7f98aaffd640 1 -- 172.21.15.55:0/1530901887 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac0d15e0 con 0x7f98ac0cd520 2024-05-20T22:09:01.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.124+0000 7f98a9ffb640 1 -- 172.21.15.55:0/1530901887 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2114496015 0 0) 0x7f98a0001110 con 0x7f98ac0d0860 2024-05-20T22:09:01.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98aaffd640 1 --2- 172.21.15.55:0/1530901887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 0x7f98ac08c960 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f989c009870 tx=0x7f989c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dfc286013971c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:01.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98a9ffb640 1 -- 172.21.15.55:0/1530901887 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c037070 con 0x7f98ac0cd520 2024-05-20T22:09:01.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98a9ffb640 1 -- 172.21.15.55:0/1530901887 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989c032440 con 0x7f98ac0cd520 2024-05-20T22:09:01.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98a9ffb640 1 -- 172.21.15.55:0/1530901887 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c032810 con 0x7f98ac0cd520 2024-05-20T22:09:01.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98b19b7640 1 -- 172.21.15.55:0/1530901887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 msgr2=0x7f98ac08c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98b19b7640 1 --2- 172.21.15.55:0/1530901887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 0x7f98ac08c960 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f989c009870 tx=0x7f989c02a1c0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98b19b7640 1 -- 172.21.15.55:0/1530901887 shutdown_connections 2024-05-20T22:09:01.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98b19b7640 1 --2- 172.21.15.55:0/1530901887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac0cf270 0x7f98ac0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98b19b7640 1 --2- 172.21.15.55:0/1530901887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 0x7f98ac08c960 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98b19b7640 1 -- 172.21.15.55:0/1530901887 >> 172.21.15.55:0/1530901887 conn(0x7f98ac0d2860 msgr2=0x7f98ac0d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:01.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98b19b7640 1 -- 172.21.15.55:0/1530901887 shutdown_connections 2024-05-20T22:09:01.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.125+0000 7f98b19b7640 1 -- 172.21.15.55:0/1530901887 wait complete. 2024-05-20T22:09:01.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.127+0000 7f98b19b7640 1 Processor -- start 2024-05-20T22:09:01.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.127+0000 7f98b19b7640 1 -- start start 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98b19b7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 0x7f98ac15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98b19b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ac0cf270 0x7f98ac159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98b19b7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac0d0860 0x7f98ac1f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98b19b7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac0d0c70 con 0x7f98ac0cd520 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98b19b7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac0b2cb0 con 0x7f98ac0cf270 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98b19b7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98ac15df30 con 0x7f98ac0d0860 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 0x7f98ac15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 0x7f98ac15d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47850/0 (socket says 172.21.15.55:47850) 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98aaffd640 1 -- 172.21.15.55:0/4288835898 learned_addr learned my addr 172.21.15.55:0/4288835898 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98ab7fe640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac0d0860 0x7f98ac1f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98aa7fc640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ac0cf270 0x7f98ac159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98aaffd640 1 -- 172.21.15.55:0/4288835898 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac0d0860 msgr2=0x7f98ac1f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98aaffd640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac0d0860 0x7f98ac1f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98aaffd640 1 -- 172.21.15.55:0/4288835898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ac0cf270 msgr2=0x7f98ac159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98aaffd640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ac0cf270 0x7f98ac159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98aaffd640 1 -- 172.21.15.55:0/4288835898 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac1f60a0 con 0x7f98ac0cd520 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.129+0000 7f98aaffd640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 0x7f98ac15d6a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f989c02a200 tx=0x7f989c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.130+0000 7f9893fff640 1 -- 172.21.15.55:0/4288835898 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c041020 con 0x7f98ac0cd520 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.130+0000 7f9893fff640 1 -- 172.21.15.55:0/4288835898 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989c0306c0 con 0x7f98ac0cd520 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.130+0000 7f98b19b7640 1 -- 172.21.15.55:0/4288835898 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ac1f6380 con 0x7f98ac0cd520 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.130+0000 7f9893fff640 1 -- 172.21.15.55:0/4288835898 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c037040 con 0x7f98ac0cd520 2024-05-20T22:09:01.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.130+0000 7f98b19b7640 1 -- 172.21.15.55:0/4288835898 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ac15f630 con 0x7f98ac0cd520 2024-05-20T22:09:01.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.131+0000 7f98b19b7640 1 -- 172.21.15.55:0/4288835898 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ac069090 con 0x7f98ac0cd520 2024-05-20T22:09:01.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.131+0000 7f9893fff640 1 -- 172.21.15.55:0/4288835898 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f989c044020 con 0x7f98ac0cd520 2024-05-20T22:09:01.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.132+0000 7f9893fff640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989806be30 0x7f989806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.132+0000 7f98aa7fc640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989806be30 0x7f989806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.132+0000 7f9893fff640 1 -- 172.21.15.55:0/4288835898 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f989c0abac0 con 0x7f98ac0cd520 2024-05-20T22:09:01.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.132+0000 7f98aa7fc640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989806be30 0x7f989806e340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f98a0007fd0 tx=0x7f98a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:01.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.136+0000 7f9893fff640 1 -- 172.21.15.55:0/4288835898 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f989c04b030 con 0x7f98ac0cd520 2024-05-20T22:09:01.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.324+0000 7f98b19b7640 1 -- 172.21.15.55:0/4288835898 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f98ac15a370 con 0x7f98ac0cd520 2024-05-20T22:09:01.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.324+0000 7f9893fff640 1 -- 172.21.15.55:0/4288835898 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f989c07a070 con 0x7f98ac0cd520 2024-05-20T22:09:01.325 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:09:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 -- 172.21.15.55:0/4288835898 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989806be30 msgr2=0x7f989806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989806be30 0x7f989806e340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f98a0007fd0 tx=0x7f98a0009040 comp rx=0 tx=0).stop 2024-05-20T22:09:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 -- 172.21.15.55:0/4288835898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 msgr2=0x7f98ac15d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 0x7f98ac15d6a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f989c02a200 tx=0x7f989c0046c0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 -- 172.21.15.55:0/4288835898 shutdown_connections 2024-05-20T22:09:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f989806be30 0x7f989806e340 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98ac0d0860 0x7f98ac1f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ac0cf270 0x7f98ac159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 --2- 172.21.15.55:0/4288835898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98ac0cd520 0x7f98ac15d6a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 -- 172.21.15.55:0/4288835898 >> 172.21.15.55:0/4288835898 conn(0x7f98ac0d2860 msgr2=0x7f98ac0706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:01.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 -- 172.21.15.55:0/4288835898 shutdown_connections 2024-05-20T22:09:01.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.327+0000 7f98b19b7640 1 -- 172.21.15.55:0/4288835898 wait complete. 2024-05-20T22:09:01.339 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:09:01.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.453+0000 7efd91221640 1 Processor -- start 2024-05-20T22:09:01.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.453+0000 7efd91221640 1 -- start start 2024-05-20T22:09:01.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.454+0000 7efd91221640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c15f100 0x7efd8c0b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd91221640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd8c0d3dd0 con 0x7efd8c0b3b30 2024-05-20T22:09:01.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd91221640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd8c0d40b0 con 0x7efd8c06c790 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd91221640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c0d4390 con 0x7efd8c15f100 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd8ad9d640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efd8c0b3b30 0x7efd8c06c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50974/0 (socket says 172.21.15.55:50974) 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd8ad9d640 1 -- 172.21.15.55:0/411123149 learned_addr learned my addr 172.21.15.55:0/411123149 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd8a59c640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c15f100 0x7efd8c0b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354328372 0 0) 0x7efd8c0d3dd0 con 0x7efd8c0b3b30 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd74003520 con 0x7efd8c0b3b30 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3202585857 0 0) 0x7efd74003520 con 0x7efd8c0b3b30 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c15f100 msgr2=0x7efd8c0b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c15f100 0x7efd8c0b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 >> v1:172.21.15.138:6789/0 conn(0x7efd8c06c790 legacy=0x7efd8c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c06b5a0 con 0x7efd8c0b3b30 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7efd7c001670 con 0x7efd8c0b3b30 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 >> v1:172.21.15.55:6789/0 conn(0x7efd8c0b3b30 legacy=0x7efd8c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c15f100 0x7efd740039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd74003f40 0x7efd74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd74004d40 0x7efd74005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c0d4390 con 0x7efd74004d40 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd7400bad0 con 0x7efd74003f40 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.455+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efd7400bcd0 con 0x7efd8c15f100 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.456+0000 7efd8b59e640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd74004d40 0x7efd74005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.456+0000 7efd8ad9d640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c15f100 0x7efd740039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.456+0000 7efd8a59c640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd74003f40 0x7efd74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.456+0000 7efd8b59e640 1 -- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c15f100 msgr2=0x7efd740039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.456+0000 7efd8b59e640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c15f100 0x7efd740039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.456+0000 7efd8b59e640 1 -- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd74003f40 msgr2=0x7efd74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.456+0000 7efd8b59e640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd74003f40 0x7efd74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.456+0000 7efd8b59e640 1 -- 172.21.15.55:0/411123149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd74003520 con 0x7efd74004d40 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.457+0000 7efd8b59e640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd74004d40 0x7efd74005150 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efd84001f30 tx=0x7efd74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a718e20c09855a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:01.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.457+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84001b30 con 0x7efd74004d40 2024-05-20T22:09:01.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.457+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd84031620 con 0x7efd74004d40 2024-05-20T22:09:01.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.457+0000 7efd89d9b640 1 -- 172.21.15.55:0/411123149 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84031a10 con 0x7efd74004d40 2024-05-20T22:09:01.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.457+0000 7efd91221640 1 -- 172.21.15.55:0/411123149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd74004d40 msgr2=0x7efd74005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.457+0000 7efd91221640 1 --2- 172.21.15.55:0/411123149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd74004d40 0x7efd74005150 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efd84001f30 tx=0x7efd74005830 comp rx=0 tx=0).stop 2024-05-20T22:09:01.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.457+0000 7efd8b59e640 1 -- 172.21.15.55:0/411123149 reap_dead start 2024-05-20T22:09:01.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.457+0000 7efd91221640 1 -- 172.21.15.55:0/411123149 shutdown_connections 2024-05-20T22:09:01.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.457+0000 7efd91221640 1 -- 172.21.15.55:0/411123149 >> 172.21.15.55:0/411123149 conn(0x7efd8c0d51d0 msgr2=0x7efd8c0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:01.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.458+0000 7efd91221640 1 -- 172.21.15.55:0/411123149 shutdown_connections 2024-05-20T22:09:01.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.458+0000 7efd91221640 1 -- 172.21.15.55:0/411123149 wait complete. 2024-05-20T22:09:01.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.460+0000 7efd91221640 1 Processor -- start 2024-05-20T22:09:01.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.460+0000 7efd91221640 1 -- start start 2024-05-20T22:09:01.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd91221640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c159b40 0x7efd8c1f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd91221640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd8c0ae970 0x7efd8c0ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd91221640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c0aed80 0x7efd8c0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd91221640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c06cba0 con 0x7efd8c0ae970 2024-05-20T22:09:01.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd91221640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c06bb60 con 0x7efd8c159b40 2024-05-20T22:09:01.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd91221640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c06cd20 con 0x7efd8c0aed80 2024-05-20T22:09:01.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd8a59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd8c0ae970 0x7efd8c0ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd8a59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd8c0ae970 0x7efd8c0ac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47868/0 (socket says 172.21.15.55:47868) 2024-05-20T22:09:01.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd8a59c640 1 -- 172.21.15.55:0/1516931628 learned_addr learned my addr 172.21.15.55:0/1516931628 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd8b59e640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c0aed80 0x7efd8c0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd8ad9d640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c159b40 0x7efd8c1f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd8a59c640 1 -- 172.21.15.55:0/1516931628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c0aed80 msgr2=0x7efd8c0accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd8a59c640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c0aed80 0x7efd8c0accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd8a59c640 1 -- 172.21.15.55:0/1516931628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c159b40 msgr2=0x7efd8c1f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd8a59c640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c159b40 0x7efd8c1f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.461+0000 7efd8a59c640 1 -- 172.21.15.55:0/1516931628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c0ad5c0 con 0x7efd8c0ae970 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.462+0000 7efd8a59c640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd8c0ae970 0x7efd8c0ac770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efd800119a0 tx=0x7efd80011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.462+0000 7efd737fe640 1 -- 172.21.15.55:0/1516931628 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8000faf0 con 0x7efd8c0ae970 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.462+0000 7efd737fe640 1 -- 172.21.15.55:0/1516931628 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8000fc90 con 0x7efd8c0ae970 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.462+0000 7efd737fe640 1 -- 172.21.15.55:0/1516931628 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80014440 con 0x7efd8c0ae970 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.462+0000 7efd91221640 1 -- 172.21.15.55:0/1516931628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c0ac280 con 0x7efd8c0ae970 2024-05-20T22:09:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.462+0000 7efd91221640 1 -- 172.21.15.55:0/1516931628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c0ae170 con 0x7efd8c0ae970 2024-05-20T22:09:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.463+0000 7efd91221640 1 -- 172.21.15.55:0/1516931628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8c0b6cc0 con 0x7efd8c0ae970 2024-05-20T22:09:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.464+0000 7efd737fe640 1 -- 172.21.15.55:0/1516931628 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd80012040 con 0x7efd8c0ae970 2024-05-20T22:09:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.464+0000 7efd737fe640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efd7806be30 0x7efd7806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.464+0000 7efd8ad9d640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efd7806be30 0x7efd7806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.464+0000 7efd737fe640 1 -- 172.21.15.55:0/1516931628 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efd8008b920 con 0x7efd8c0ae970 2024-05-20T22:09:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.465+0000 7efd8ad9d640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efd7806be30 0x7efd7806e340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efd8c1638b0 tx=0x7efd7c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:01.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.469+0000 7efd737fe640 1 -- 172.21.15.55:0/1516931628 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efd8c0b6cc0 con 0x7efd8c0ae970 2024-05-20T22:09:01.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.692+0000 7efd91221640 1 -- 172.21.15.55:0/1516931628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd8c0ac560 con 0x7efd8c0ae970 2024-05-20T22:09:01.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.692+0000 7efd737fe640 1 -- 172.21.15.55:0/1516931628 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7efd80019030 con 0x7efd8c0ae970 2024-05-20T22:09:01.693 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:01.693 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:07:53.028952+0000","modified":"2024-05-20T22:08:11.135754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-05-20T22:09:01.694 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-20T22:09:01.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 -- 172.21.15.55:0/1516931628 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efd7806be30 msgr2=0x7efd7806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efd7806be30 0x7efd7806e340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efd8c1638b0 tx=0x7efd7c009040 comp rx=0 tx=0).stop 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 -- 172.21.15.55:0/1516931628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd8c0ae970 msgr2=0x7efd8c0ac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd8c0ae970 0x7efd8c0ac770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efd800119a0 tx=0x7efd80011e60 comp rx=0 tx=0).stop 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 -- 172.21.15.55:0/1516931628 shutdown_connections 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efd7806be30 0x7efd7806e340 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd8c0aed80 0x7efd8c0accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd8c0ae970 0x7efd8c0ac770 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 --2- 172.21.15.55:0/1516931628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c159b40 0x7efd8c1f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 -- 172.21.15.55:0/1516931628 >> 172.21.15.55:0/1516931628 conn(0x7efd8c0d51d0 msgr2=0x7efd8c162b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 -- 172.21.15.55:0/1516931628 shutdown_connections 2024-05-20T22:09:01.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.694+0000 7efd91221640 1 -- 172.21.15.55:0/1516931628 wait complete. 2024-05-20T22:09:01.707 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:09:01.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.823+0000 7f2b06f07640 1 Processor -- start 2024-05-20T22:09:01.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.823+0000 7f2b06f07640 1 -- start start 2024-05-20T22:09:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.824+0000 7f2b06f07640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b0006d6b0 0x7f2b0006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.824+0000 7f2b06f07640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b000d2700 0x7f2b000d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.824+0000 7f2b06f07640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b0006e320 con 0x7f2b000cd030 2024-05-20T22:09:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.824+0000 7f2b06f07640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f2b000d1d70 con 0x7f2b0006d6b0 2024-05-20T22:09:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.824+0000 7f2b06f07640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2b000b2fa0 con 0x7f2b000d2700 2024-05-20T22:09:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.824+0000 7f2afffff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b0006d6b0 0x7f2b0006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.824+0000 7f2afffff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b0006d6b0 0x7f2b0006dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47880/0 (socket says 172.21.15.55:47880) 2024-05-20T22:09:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.824+0000 7f2afffff640 1 -- 172.21.15.55:0/3765169747 learned_addr learned my addr 172.21.15.55:0/3765169747 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.825+0000 7f2b054a4640 1 --2- 172.21.15.55:0/3765169747 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b000d2700 0x7f2b000d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.825+0000 7f2afffff640 1 -- 172.21.15.55:0/3765169747 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b000d2700 msgr2=0x7f2b000d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:01.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.825+0000 7f2afffff640 1 --2- 172.21.15.55:0/3765169747 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b000d2700 0x7f2b000d2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.825+0000 7f2afffff640 1 -- 172.21.15.55:0/3765169747 >> v1:172.21.15.138:6789/0 conn(0x7f2b000cd030 legacy=0x7f2b00162aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:01.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.825+0000 7f2afffff640 1 -- 172.21.15.55:0/3765169747 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0006e600 con 0x7f2b0006d6b0 2024-05-20T22:09:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.825+0000 7f2afffff640 1 --2- 172.21.15.55:0/3765169747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b0006d6b0 0x7f2b0006dac0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009870 tx=0x7f2aec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1d90accb982ec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.826+0000 7f2aff7fe640 1 -- 172.21.15.55:0/3765169747 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec037070 con 0x7f2b0006d6b0 2024-05-20T22:09:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.826+0000 7f2aff7fe640 1 -- 172.21.15.55:0/3765169747 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aec032440 con 0x7f2b0006d6b0 2024-05-20T22:09:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.826+0000 7f2aff7fe640 1 -- 172.21.15.55:0/3765169747 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec032810 con 0x7f2b0006d6b0 2024-05-20T22:09:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.826+0000 7f2b06f07640 1 -- 172.21.15.55:0/3765169747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b0006d6b0 msgr2=0x7f2b0006dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.826+0000 7f2b06f07640 1 --2- 172.21.15.55:0/3765169747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b0006d6b0 0x7f2b0006dac0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009870 tx=0x7f2aec02a180 comp rx=0 tx=0).stop 2024-05-20T22:09:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.826+0000 7f2b06f07640 1 -- 172.21.15.55:0/3765169747 shutdown_connections 2024-05-20T22:09:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.826+0000 7f2b06f07640 1 --2- 172.21.15.55:0/3765169747 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b000d2700 0x7f2b000d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.826+0000 7f2b06f07640 1 --2- 172.21.15.55:0/3765169747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b0006d6b0 0x7f2b0006dac0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.826+0000 7f2b06f07640 1 -- 172.21.15.55:0/3765169747 >> 172.21.15.55:0/3765169747 conn(0x7f2b0008bff0 msgr2=0x7f2b0008c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.826+0000 7f2b06f07640 1 -- 172.21.15.55:0/3765169747 shutdown_connections 2024-05-20T22:09:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.827+0000 7f2b06f07640 1 -- 172.21.15.55:0/3765169747 wait complete. 2024-05-20T22:09:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.828+0000 7f2b06f07640 1 Processor -- start 2024-05-20T22:09:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.828+0000 7f2b06f07640 1 -- start start 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b06f07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b0006d6b0 0x7f2b000b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b06f07640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b000cd030 0x7f2b000b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b06f07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000d2700 0x7f2b000b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b06f07640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000b2890 con 0x7f2b000d2700 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b06f07640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000b3280 con 0x7f2b0006d6b0 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b06f07640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0006e000 con 0x7f2b000cd030 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b054a4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000d2700 0x7f2b000b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b054a4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000d2700 0x7f2b000b4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47886/0 (socket says 172.21.15.55:47886) 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b054a4640 1 -- 172.21.15.55:0/1623678378 learned_addr learned my addr 172.21.15.55:0/1623678378 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b04ca3640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b0006d6b0 0x7f2b000b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2afffff640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b000cd030 0x7f2b000b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b054a4640 1 -- 172.21.15.55:0/1623678378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b000cd030 msgr2=0x7f2b000b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b054a4640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b000cd030 0x7f2b000b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b054a4640 1 -- 172.21.15.55:0/1623678378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b0006d6b0 msgr2=0x7f2b000b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b054a4640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b0006d6b0 0x7f2b000b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b054a4640 1 -- 172.21.15.55:0/1623678378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b000ccd00 con 0x7f2b000d2700 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.830+0000 7f2b054a4640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000d2700 0x7f2b000b4310 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2af800cc80 tx=0x7f2af800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.831+0000 7f2afd7fa640 1 -- 172.21.15.55:0/1623678378 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af800ce30 con 0x7f2b000d2700 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.831+0000 7f2afd7fa640 1 -- 172.21.15.55:0/1623678378 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af8003990 con 0x7f2b000d2700 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.831+0000 7f2b06f07640 1 -- 172.21.15.55:0/1623678378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b001510d0 con 0x7f2b000d2700 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.831+0000 7f2afd7fa640 1 -- 172.21.15.55:0/1623678378 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8003de0 con 0x7f2b000d2700 2024-05-20T22:09:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.831+0000 7f2b06f07640 1 -- 172.21.15.55:0/1623678378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b00151690 con 0x7f2b000d2700 2024-05-20T22:09:01.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.832+0000 7f2b06f07640 1 -- 172.21.15.55:0/1623678378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b00069090 con 0x7f2b000d2700 2024-05-20T22:09:01.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.832+0000 7f2afd7fa640 1 -- 172.21.15.55:0/1623678378 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2af8003fe0 con 0x7f2b000d2700 2024-05-20T22:09:01.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.833+0000 7f2afd7fa640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ae406be30 0x7f2ae406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:01.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.833+0000 7f2b04ca3640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ae406be30 0x7f2ae406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:01.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.833+0000 7f2afd7fa640 1 -- 172.21.15.55:0/1623678378 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2af808b5e0 con 0x7f2b000d2700 2024-05-20T22:09:01.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.834+0000 7f2b04ca3640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ae406be30 0x7f2ae406e340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2af40098c0 tx=0x7f2af4009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:01.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:01.838+0000 7f2afd7fa640 1 -- 172.21.15.55:0/1623678378 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2af8014030 con 0x7f2b000d2700 2024-05-20T22:09:02.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.024+0000 7f2b06f07640 1 -- 172.21.15.55:0/1623678378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b00074670 con 0x7f2b000d2700 2024-05-20T22:09:02.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.025+0000 7f2afd7fa640 1 -- 172.21.15.55:0/1623678378 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7f2af8059540 con 0x7f2b000d2700 2024-05-20T22:09:02.026 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:02.026 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:09:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 -- 172.21.15.55:0/1623678378 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ae406be30 msgr2=0x7f2ae406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ae406be30 0x7f2ae406e340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2af40098c0 tx=0x7f2af4009060 comp rx=0 tx=0).stop 2024-05-20T22:09:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 -- 172.21.15.55:0/1623678378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000d2700 msgr2=0x7f2b000b4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000d2700 0x7f2b000b4310 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2af800cc80 tx=0x7f2af800a5c0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 -- 172.21.15.55:0/1623678378 shutdown_connections 2024-05-20T22:09:02.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2ae406be30 0x7f2ae406e340 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b000d2700 0x7f2b000b4310 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2b000cd030 0x7f2b000b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 --2- 172.21.15.55:0/1623678378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b0006d6b0 0x7f2b000b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 -- 172.21.15.55:0/1623678378 >> 172.21.15.55:0/1623678378 conn(0x7f2b0008bff0 msgr2=0x7f2b000b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:02.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 -- 172.21.15.55:0/1623678378 shutdown_connections 2024-05-20T22:09:02.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.027+0000 7f2b06f07640 1 -- 172.21.15.55:0/1623678378 wait complete. 2024-05-20T22:09:02.040 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T22:09:02.040 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:09:02.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.154+0000 7fa919821640 1 Processor -- start 2024-05-20T22:09:02.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.154+0000 7fa919821640 1 -- start start 2024-05-20T22:09:02.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa919821640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 0x7fa9140b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa919821640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9140d3dd0 con 0x7fa9140b3b30 2024-05-20T22:09:02.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa919821640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9140d40b0 con 0x7fa91406c790 2024-05-20T22:09:02.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa919821640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa9140d4390 con 0x7fa91415f100 2024-05-20T22:09:02.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa912ffd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa9140b3b30 0x7fa91406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50980/0 (socket says 172.21.15.55:50980) 2024-05-20T22:09:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa912ffd640 1 -- 172.21.15.55:0/610951862 learned_addr learned my addr 172.21.15.55:0/610951862 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa9127fc640 1 --2- 172.21.15.55:0/610951862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 0x7fa9140b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa911ffb640 1 -- 172.21.15.55:0/610951862 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1804438065 0 0) 0x7fa9140d3dd0 con 0x7fa9140b3b30 2024-05-20T22:09:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa911ffb640 1 -- 172.21.15.55:0/610951862 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8fc003520 con 0x7fa9140b3b30 2024-05-20T22:09:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa911ffb640 1 -- 172.21.15.55:0/610951862 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3572886868 0 0) 0x7fa9140d40b0 con 0x7fa91406c790 2024-05-20T22:09:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa911ffb640 1 -- 172.21.15.55:0/610951862 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9140d3dd0 con 0x7fa91406c790 2024-05-20T22:09:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa911ffb640 1 -- 172.21.15.55:0/610951862 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2377994839 0 0) 0x7fa8fc003520 con 0x7fa9140b3b30 2024-05-20T22:09:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa9127fc640 1 -- 172.21.15.55:0/610951862 >> v1:172.21.15.138:6789/0 conn(0x7fa91406c790 legacy=0x7fa9140cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa9127fc640 1 -- 172.21.15.55:0/610951862 >> v1:172.21.15.55:6789/0 conn(0x7fa9140b3b30 legacy=0x7fa91406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.156+0000 7fa9127fc640 1 -- 172.21.15.55:0/610951862 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91406b5a0 con 0x7fa91415f100 2024-05-20T22:09:02.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.157+0000 7fa9127fc640 1 --2- 172.21.15.55:0/610951862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 0x7fa9140b7fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9080097d0 tx=0x7fa90802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad9fcc7975d606e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:02.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.157+0000 7fa911ffb640 1 -- 172.21.15.55:0/610951862 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908037070 con 0x7fa91415f100 2024-05-20T22:09:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.157+0000 7fa911ffb640 1 -- 172.21.15.55:0/610951862 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90802ac30 con 0x7fa91415f100 2024-05-20T22:09:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.157+0000 7fa911ffb640 1 -- 172.21.15.55:0/610951862 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904001670 con 0x7fa91415f100 2024-05-20T22:09:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.158+0000 7fa919821640 1 -- 172.21.15.55:0/610951862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 msgr2=0x7fa9140b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.158+0000 7fa919821640 1 --2- 172.21.15.55:0/610951862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 0x7fa9140b7fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9080097d0 tx=0x7fa90802a1a0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.158+0000 7fa919821640 1 -- 172.21.15.55:0/610951862 shutdown_connections 2024-05-20T22:09:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.158+0000 7fa919821640 1 --2- 172.21.15.55:0/610951862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 0x7fa9140b7fe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.158+0000 7fa919821640 1 -- 172.21.15.55:0/610951862 >> 172.21.15.55:0/610951862 conn(0x7fa9140d51d0 msgr2=0x7fa9140b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.158+0000 7fa919821640 1 -- 172.21.15.55:0/610951862 shutdown_connections 2024-05-20T22:09:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.158+0000 7fa919821640 1 -- 172.21.15.55:0/610951862 wait complete. 2024-05-20T22:09:02.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.160+0000 7fa919821640 1 Processor -- start 2024-05-20T22:09:02.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.160+0000 7fa919821640 1 -- start start 2024-05-20T22:09:02.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.161+0000 7fa919821640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa91406c790 0x7fa91415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.161+0000 7fa919821640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9140b3b30 0x7fa9141f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.161+0000 7fa919821640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 0x7fa914160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.161+0000 7fa919821640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa91406cba0 con 0x7fa91406c790 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.161+0000 7fa919821640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa91406bb60 con 0x7fa9140b3b30 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.161+0000 7fa919821640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa91406cd20 con 0x7fa91415f100 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.161+0000 7fa912ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa91406c790 0x7fa91415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.161+0000 7fa912ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa91406c790 0x7fa91415a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47898/0 (socket says 172.21.15.55:47898) 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.161+0000 7fa912ffd640 1 -- 172.21.15.55:0/1667650241 learned_addr learned my addr 172.21.15.55:0/1667650241 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa9127fc640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9140b3b30 0x7fa9141f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa9137fe640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 0x7fa914160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa912ffd640 1 -- 172.21.15.55:0/1667650241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 msgr2=0x7fa914160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa912ffd640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 0x7fa914160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa912ffd640 1 -- 172.21.15.55:0/1667650241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9140b3b30 msgr2=0x7fa9141f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa912ffd640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9140b3b30 0x7fa9141f8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa912ffd640 1 -- 172.21.15.55:0/1667650241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9141545e0 con 0x7fa91406c790 2024-05-20T22:09:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa912ffd640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa91406c790 0x7fa91415a3b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa90400c4d0 tx=0x7fa90400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:02.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa8fbfff640 1 -- 172.21.15.55:0/1667650241 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904017070 con 0x7fa91406c790 2024-05-20T22:09:02.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa8fbfff640 1 -- 172.21.15.55:0/1667650241 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa904005ce0 con 0x7fa91406c790 2024-05-20T22:09:02.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa919821640 1 -- 172.21.15.55:0/1667650241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9141548c0 con 0x7fa91406c790 2024-05-20T22:09:02.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa8fbfff640 1 -- 172.21.15.55:0/1667650241 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9040072b0 con 0x7fa91406c790 2024-05-20T22:09:02.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.162+0000 7fa919821640 1 -- 172.21.15.55:0/1667650241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa914154e80 con 0x7fa91406c790 2024-05-20T22:09:02.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.164+0000 7fa919821640 1 -- 172.21.15.55:0/1667650241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa914069090 con 0x7fa91406c790 2024-05-20T22:09:02.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.164+0000 7fa8fbfff640 1 -- 172.21.15.55:0/1667650241 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa90401a070 con 0x7fa91406c790 2024-05-20T22:09:02.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.164+0000 7fa8fbfff640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90006be60 0x7fa90006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.165+0000 7fa8fbfff640 1 -- 172.21.15.55:0/1667650241 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa90408a540 con 0x7fa91406c790 2024-05-20T22:09:02.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.165+0000 7fa9127fc640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90006be60 0x7fa90006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.165+0000 7fa9127fc640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90006be60 0x7fa90006e370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa908034000 tx=0x7fa90802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:02.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.169+0000 7fa8fbfff640 1 -- 172.21.15.55:0/1667650241 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa9040584a0 con 0x7fa91406c790 2024-05-20T22:09:02.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.391+0000 7fa919821640 1 -- 172.21.15.55:0/1667650241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9140d2ee0 con 0x7fa91406c790 2024-05-20T22:09:02.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.391+0000 7fa8fbfff640 1 -- 172.21.15.55:0/1667650241 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7fa90405c7a0 con 0x7fa91406c790 2024-05-20T22:09:02.392 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:02.393 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:07:53.028952+0000","modified":"2024-05-20T22:08:11.135754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-05-20T22:09:02.393 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-20T22:09:02.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 -- 172.21.15.55:0/1667650241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90006be60 msgr2=0x7fa90006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90006be60 0x7fa90006e370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa908034000 tx=0x7fa90802e000 comp rx=0 tx=0).stop 2024-05-20T22:09:02.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 -- 172.21.15.55:0/1667650241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa91406c790 msgr2=0x7fa91415a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa91406c790 0x7fa91415a3b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa90400c4d0 tx=0x7fa90400c990 comp rx=0 tx=0).stop 2024-05-20T22:09:02.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 -- 172.21.15.55:0/1667650241 shutdown_connections 2024-05-20T22:09:02.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90006be60 0x7fa90006e370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa91415f100 0x7fa914160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9140b3b30 0x7fa9141f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 --2- 172.21.15.55:0/1667650241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa91406c790 0x7fa91415a3b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 -- 172.21.15.55:0/1667650241 >> 172.21.15.55:0/1667650241 conn(0x7fa9140d51d0 msgr2=0x7fa914162ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 -- 172.21.15.55:0/1667650241 shutdown_connections 2024-05-20T22:09:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.394+0000 7fa919821640 1 -- 172.21.15.55:0/1667650241 wait complete. 2024-05-20T22:09:02.406 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:09:02.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.521+0000 7f3bb4ffe640 1 Processor -- start 2024-05-20T22:09:02.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.521+0000 7f3bb4ffe640 1 -- start start 2024-05-20T22:09:02.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.522+0000 7f3bb4ffe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3bb00b5830 0x7f3bb00b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.522+0000 7f3bb4ffe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb00d2e10 0x7f3bb00d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.522+0000 7f3bb4ffe640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb015da90 con 0x7f3bb00cd840 2024-05-20T22:09:02.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.522+0000 7f3bb4ffe640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb00b6990 con 0x7f3bb00b5830 2024-05-20T22:09:02.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.522+0000 7f3bb4ffe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb00b5d80 con 0x7f3bb00d2e10 2024-05-20T22:09:02.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.522+0000 7f3bae59c640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3bb00cd840 0x7f3bb00d2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50994/0 (socket says 172.21.15.55:50994) 2024-05-20T22:09:02.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.522+0000 7f3bae59c640 1 -- 172.21.15.55:0/4142699444 learned_addr learned my addr 172.21.15.55:0/4142699444 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3540339923 0 0) 0x7f3bb015da90 con 0x7f3bb00cd840 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3badd9b640 1 --2- 172.21.15.55:0/4142699444 >> v2:172.21.15.138:3300/0 conn(0x7f3bb00b5830 0x7f3bb00b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b9c003520 con 0x7f3bb00cd840 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3baed9d640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb00d2e10 0x7f3bb00d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1748652542 0 0) 0x7f3b9c003520 con 0x7f3bb00cd840 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb00d2e10 msgr2=0x7f3bb00d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb00d2e10 0x7f3bb00d1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 >> v2:172.21.15.138:3300/0 conn(0x7f3bb00b5830 msgr2=0x7f3bb00b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 --2- 172.21.15.55:0/4142699444 >> v2:172.21.15.138:3300/0 conn(0x7f3bb00b5830 0x7f3bb00b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb00b6060 con 0x7f3bb00cd840 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f3ba00017f0 con 0x7f3bb00cd840 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 >> v1:172.21.15.55:6789/0 conn(0x7f3bb00cd840 legacy=0x7f3bb00d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb00d2e10 0x7f3b9c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b9c003f30 0x7f3b9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b9c004d40 0x7f3b9c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00b6990 con 0x7f3b9c003f30 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00b5d80 con 0x7f3b9c004d40 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c00bac0 con 0x7f3bb00d2e10 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3badd9b640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b9c003f30 0x7f3b9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3bae59c640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb00d2e10 0x7f3b9c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3baed9d640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b9c004d40 0x7f3b9c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3badd9b640 1 -- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb00d2e10 msgr2=0x7f3b9c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3badd9b640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb00d2e10 0x7f3b9c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3badd9b640 1 -- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b9c004d40 msgr2=0x7f3b9c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.523+0000 7f3badd9b640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b9c004d40 0x7f3b9c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.524+0000 7f3badd9b640 1 -- 172.21.15.55:0/4142699444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c003520 con 0x7f3b9c003f30 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.524+0000 7f3badd9b640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b9c003f30 0x7f3b9c004800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009ac0 tx=0x7f3b9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd6c2df85cf208d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:02.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.524+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98036070 con 0x7f3b9c003f30 2024-05-20T22:09:02.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.524+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b98031400 con 0x7f3b9c003f30 2024-05-20T22:09:02.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.524+0000 7f3bad59a640 1 -- 172.21.15.55:0/4142699444 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98009690 con 0x7f3b9c003f30 2024-05-20T22:09:02.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.525+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/4142699444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b9c003f30 msgr2=0x7f3b9c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.525+0000 7f3bb4ffe640 1 --2- 172.21.15.55:0/4142699444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b9c003f30 0x7f3b9c004800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009ac0 tx=0x7f3b9c005830 comp rx=0 tx=0).stop 2024-05-20T22:09:02.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.525+0000 7f3baed9d640 1 -- 172.21.15.55:0/4142699444 reap_dead start 2024-05-20T22:09:02.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.525+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/4142699444 shutdown_connections 2024-05-20T22:09:02.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.525+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/4142699444 >> 172.21.15.55:0/4142699444 conn(0x7f3bb008bff0 msgr2=0x7f3bb008c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:02.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.525+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/4142699444 shutdown_connections 2024-05-20T22:09:02.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.525+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/4142699444 wait complete. 2024-05-20T22:09:02.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.527+0000 7f3bb4ffe640 1 Processor -- start 2024-05-20T22:09:02.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.527+0000 7f3bb4ffe640 1 -- start start 2024-05-20T22:09:02.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3bb4ffe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b9c003f30 0x7f3bb00ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3bb4ffe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bb015ffd0 0x7f3bb0156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3bb4ffe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bb0157520 0x7f3bb00b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3bb4ffe640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00b8160 con 0x7f3bb0157520 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3bb4ffe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00b6620 con 0x7f3bb015ffd0 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3bb4ffe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb00d1890 con 0x7f3b9c003f30 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3baed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bb0157520 0x7f3bb00b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3baed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bb0157520 0x7f3bb00b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47914/0 (socket says 172.21.15.55:47914) 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3baed9d640 1 -- 172.21.15.55:0/109779055 learned_addr learned my addr 172.21.15.55:0/109779055 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3badd9b640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bb015ffd0 0x7f3bb0156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3bae59c640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b9c003f30 0x7f3bb00ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3baed9d640 1 -- 172.21.15.55:0/109779055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b9c003f30 msgr2=0x7f3bb00ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3baed9d640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b9c003f30 0x7f3bb00ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3baed9d640 1 -- 172.21.15.55:0/109779055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bb015ffd0 msgr2=0x7f3bb0156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3baed9d640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bb015ffd0 0x7f3bb0156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3baed9d640 1 -- 172.21.15.55:0/109779055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb0157930 con 0x7f3bb0157520 2024-05-20T22:09:02.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3baed9d640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bb0157520 0x7f3bb00b06e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3bb008bba0 tx=0x7f3ba80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:02.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3ba6ffd640 1 -- 172.21.15.55:0/109779055 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8011e10 con 0x7f3bb0157520 2024-05-20T22:09:02.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3ba6ffd640 1 -- 172.21.15.55:0/109779055 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba801b070 con 0x7f3bb0157520 2024-05-20T22:09:02.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/109779055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb015ef30 con 0x7f3bb0157520 2024-05-20T22:09:02.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3ba6ffd640 1 -- 172.21.15.55:0/109779055 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8003990 con 0x7f3bb0157520 2024-05-20T22:09:02.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.529+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/109779055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb015f4f0 con 0x7f3bb0157520 2024-05-20T22:09:02.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.530+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/109779055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb0069090 con 0x7f3bb0157520 2024-05-20T22:09:02.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.531+0000 7f3ba6ffd640 1 -- 172.21.15.55:0/109779055 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ba8005d10 con 0x7f3bb0157520 2024-05-20T22:09:02.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.532+0000 7f3ba6ffd640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3b9006be30 0x7f3b9006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.532+0000 7f3ba6ffd640 1 -- 172.21.15.55:0/109779055 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3ba808b850 con 0x7f3bb0157520 2024-05-20T22:09:02.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.532+0000 7f3bae59c640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3b9006be30 0x7f3b9006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.532+0000 7f3bae59c640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3b9006be30 0x7f3b9006e340 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0000ef0 tx=0x7f3ba0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:02.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.536+0000 7f3ba6ffd640 1 -- 172.21.15.55:0/109779055 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3ba8025020 con 0x7f3bb0157520 2024-05-20T22:09:02.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.757+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/109779055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bb00cae40 con 0x7f3bb0157520 2024-05-20T22:09:02.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.758+0000 7f3ba6ffd640 1 -- 172.21.15.55:0/109779055 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f3ba801d310 con 0x7f3bb0157520 2024-05-20T22:09:02.759 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:02.759 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/589467271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":589467271},{"type":"v1","addr":"172.21.15.55:6837","nonce":589467271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4276,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/373717398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":373717398},{"type":"v1","addr":"172.21.15.55:6836","nonce":373717398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/632219923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":632219923},{"type":"v1","addr":"172.21.15.138:6834","nonce":632219923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:07:53.028952+0000","modified":"2024-05-20T22:08:11.135754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2341014737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2341014737},{"type":"v1","addr":"172.21.15.138:6835","nonce":2341014737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-05-20T22:09:02.759 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-20T22:09:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/109779055 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3b9006be30 msgr2=0x7f3b9006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3b9006be30 0x7f3b9006e340 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0000ef0 tx=0x7f3ba0009040 comp rx=0 tx=0).stop 2024-05-20T22:09:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/109779055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bb0157520 msgr2=0x7f3bb00b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bb0157520 0x7f3bb00b06e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3bb008bba0 tx=0x7f3ba80115a0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/109779055 shutdown_connections 2024-05-20T22:09:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3b9006be30 0x7f3b9006e340 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bb0157520 0x7f3bb00b06e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bb015ffd0 0x7f3bb0156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 --2- 172.21.15.55:0/109779055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b9c003f30 0x7f3bb00ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/109779055 >> 172.21.15.55:0/109779055 conn(0x7f3bb008bff0 msgr2=0x7f3bb00b78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:02.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/109779055 shutdown_connections 2024-05-20T22:09:02.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.760+0000 7f3bb4ffe640 1 -- 172.21.15.55:0/109779055 wait complete. 2024-05-20T22:09:02.772 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:09:02.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.886+0000 7fd4a9d65640 1 Processor -- start 2024-05-20T22:09:02.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.886+0000 7fd4a9d65640 1 -- start start 2024-05-20T22:09:02.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.887+0000 7fd4a9d65640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4a40b3b30 0x7fd4a40b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.887+0000 7fd4a9d65640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 0x7fd4a40cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.887+0000 7fd4a9d65640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a40d3dd0 con 0x7fd4a415f100 2024-05-20T22:09:02.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.887+0000 7fd4a9d65640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd4a40d40b0 con 0x7fd4a40b3b30 2024-05-20T22:09:02.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.887+0000 7fd4a9d65640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a40d4390 con 0x7fd4a406c790 2024-05-20T22:09:02.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.887+0000 7fd4a37fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4a40b3b30 0x7fd4a40b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.887+0000 7fd4a3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 0x7fd4a40cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.887+0000 7fd4a2ffd640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd4a415f100 0x7fd4a406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:43490/0 (socket says 172.21.15.55:43490) 2024-05-20T22:09:02.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.887+0000 7fd4a2ffd640 1 -- 172.21.15.55:0/4293766663 learned_addr learned my addr 172.21.15.55:0/4293766663 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:02.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.888+0000 7fd4a37fe640 1 -- 172.21.15.55:0/4293766663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 msgr2=0x7fd4a40cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:02.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.888+0000 7fd4a37fe640 1 --2- 172.21.15.55:0/4293766663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 0x7fd4a40cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.888+0000 7fd4a37fe640 1 -- 172.21.15.55:0/4293766663 >> v1:172.21.15.138:6789/0 conn(0x7fd4a415f100 legacy=0x7fd4a406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.888+0000 7fd4a37fe640 1 -- 172.21.15.55:0/4293766663 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a406b5a0 con 0x7fd4a40b3b30 2024-05-20T22:09:02.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.888+0000 7fd4a27fc640 1 -- 172.21.15.55:0/4293766663 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286285691 0 0) 0x7fd4a40d3dd0 con 0x7fd4a415f100 2024-05-20T22:09:02.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.888+0000 7fd4a3fff640 1 --2- 172.21.15.55:0/4293766663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 0x7fd4a40cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:09:02.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.889+0000 7fd4a37fe640 1 --2- 172.21.15.55:0/4293766663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a40b3b30 0x7fd4a40b7fe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd4940097a0 tx=0x7fd49402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71e7944e6e0158f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:02.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.889+0000 7fd4a27fc640 1 -- 172.21.15.55:0/4293766663 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494037070 con 0x7fd4a40b3b30 2024-05-20T22:09:02.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.889+0000 7fd4a27fc640 1 -- 172.21.15.55:0/4293766663 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd494032440 con 0x7fd4a40b3b30 2024-05-20T22:09:02.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.889+0000 7fd4a27fc640 1 -- 172.21.15.55:0/4293766663 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494032810 con 0x7fd4a40b3b30 2024-05-20T22:09:02.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.889+0000 7fd4a9d65640 1 -- 172.21.15.55:0/4293766663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a40b3b30 msgr2=0x7fd4a40b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.889+0000 7fd4a9d65640 1 --2- 172.21.15.55:0/4293766663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a40b3b30 0x7fd4a40b7fe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd4940097a0 tx=0x7fd49402a180 comp rx=0 tx=0).stop 2024-05-20T22:09:02.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.890+0000 7fd4a9d65640 1 -- 172.21.15.55:0/4293766663 shutdown_connections 2024-05-20T22:09:02.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.890+0000 7fd4a9d65640 1 --2- 172.21.15.55:0/4293766663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 0x7fd4a40cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.890+0000 7fd4a9d65640 1 --2- 172.21.15.55:0/4293766663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a40b3b30 0x7fd4a40b7fe0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.890+0000 7fd4a9d65640 1 -- 172.21.15.55:0/4293766663 >> 172.21.15.55:0/4293766663 conn(0x7fd4a40d51d0 msgr2=0x7fd4a40b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:02.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.890+0000 7fd4a9d65640 1 -- 172.21.15.55:0/4293766663 shutdown_connections 2024-05-20T22:09:02.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.890+0000 7fd4a9d65640 1 -- 172.21.15.55:0/4293766663 wait complete. 2024-05-20T22:09:02.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.891+0000 7fd4a9d65640 1 Processor -- start 2024-05-20T22:09:02.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.892+0000 7fd4a9d65640 1 -- start start 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a9d65640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 0x7fd4a4148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a9d65640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4a40b3b30 0x7fd4a41552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a9d65640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a415f100 0x7fd4a4155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a9d65640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a406cba0 con 0x7fd4a40b3b30 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a9d65640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a406bb60 con 0x7fd4a415f100 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a9d65640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a406cd20 con 0x7fd4a406c790 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4a40b3b30 0x7fd4a41552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4a40b3b30 0x7fd4a41552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47916/0 (socket says 172.21.15.55:47916) 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a2ffd640 1 -- 172.21.15.55:0/3825881882 learned_addr learned my addr 172.21.15.55:0/3825881882 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a37fe640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 0x7fd4a4148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a3fff640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a415f100 0x7fd4a4155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a2ffd640 1 -- 172.21.15.55:0/3825881882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 msgr2=0x7fd4a4148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a2ffd640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 0x7fd4a4148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a2ffd640 1 -- 172.21.15.55:0/3825881882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a415f100 msgr2=0x7fd4a4155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:02.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a2ffd640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a415f100 0x7fd4a4155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:02.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a2ffd640 1 -- 172.21.15.55:0/3825881882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a41575a0 con 0x7fd4a40b3b30 2024-05-20T22:09:02.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.893+0000 7fd4a2ffd640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4a40b3b30 0x7fd4a41552c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd49800c3e0 tx=0x7fd49800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:02.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.894+0000 7fd48bfff640 1 -- 172.21.15.55:0/3825881882 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4980074e0 con 0x7fd4a40b3b30 2024-05-20T22:09:02.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.894+0000 7fd48bfff640 1 -- 172.21.15.55:0/3825881882 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4980047e0 con 0x7fd4a40b3b30 2024-05-20T22:09:02.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.894+0000 7fd48bfff640 1 -- 172.21.15.55:0/3825881882 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498004bd0 con 0x7fd4a40b3b30 2024-05-20T22:09:02.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.894+0000 7fd4a9d65640 1 -- 172.21.15.55:0/3825881882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a4157880 con 0x7fd4a40b3b30 2024-05-20T22:09:02.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.894+0000 7fd4a9d65640 1 -- 172.21.15.55:0/3825881882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a4157e40 con 0x7fd4a40b3b30 2024-05-20T22:09:02.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.895+0000 7fd4a9d65640 1 -- 172.21.15.55:0/3825881882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a4069090 con 0x7fd4a40b3b30 2024-05-20T22:09:02.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.895+0000 7fd48bfff640 1 -- 172.21.15.55:0/3825881882 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd498011070 con 0x7fd4a40b3b30 2024-05-20T22:09:02.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.896+0000 7fd48bfff640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd49006be30 0x7fd49006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:02.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.896+0000 7fd48bfff640 1 -- 172.21.15.55:0/3825881882 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd49808ae30 con 0x7fd4a40b3b30 2024-05-20T22:09:02.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.896+0000 7fd4a37fe640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd49006be30 0x7fd49006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:02.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.896+0000 7fd4a37fe640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd49006be30 0x7fd49006e340 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd494034000 tx=0x7fd49402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:02.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:02.901+0000 7fd48bfff640 1 -- 172.21.15.55:0/3825881882 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd49800e280 con 0x7fd4a40b3b30 2024-05-20T22:09:03.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.087+0000 7fd4a9d65640 1 -- 172.21.15.55:0/3825881882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4a40d2ee0 con 0x7fd4a40b3b30 2024-05-20T22:09:03.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.089+0000 7fd48bfff640 1 -- 172.21.15.55:0/3825881882 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18846 (secure 0 0 0) 0x7fd4980593e0 con 0x7fd4a40b3b30 2024-05-20T22:09:03.090 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:03.090 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:08:33.899778+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:07:51.275733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T22:07:52.008991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T22:07:55.711810+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:09:03.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 -- 172.21.15.55:0/3825881882 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd49006be30 msgr2=0x7fd49006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd49006be30 0x7fd49006e340 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd494034000 tx=0x7fd49402e000 comp rx=0 tx=0).stop 2024-05-20T22:09:03.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 -- 172.21.15.55:0/3825881882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4a40b3b30 msgr2=0x7fd4a41552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4a40b3b30 0x7fd4a41552c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd49800c3e0 tx=0x7fd49800c8a0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 -- 172.21.15.55:0/3825881882 shutdown_connections 2024-05-20T22:09:03.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd49006be30 0x7fd49006e340 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a415f100 0x7fd4a4155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4a40b3b30 0x7fd4a41552c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 --2- 172.21.15.55:0/3825881882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4a406c790 0x7fd4a4148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 -- 172.21.15.55:0/3825881882 >> 172.21.15.55:0/3825881882 conn(0x7fd4a40d51d0 msgr2=0x7fd4a4162b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:03.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 -- 172.21.15.55:0/3825881882 shutdown_connections 2024-05-20T22:09:03.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.091+0000 7fd4a9d65640 1 -- 172.21.15.55:0/3825881882 wait complete. 2024-05-20T22:09:03.104 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T22:09:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.219+0000 7effd9396640 1 Processor -- start 2024-05-20T22:09:03.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.219+0000 7effd9396640 1 -- start start 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd9396640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7effd40cd020 0x7effd40cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd9396640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7effd40cdac0 0x7effd40cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd9396640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd4162db0 0x7effd41631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd9396640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7effd40cc170 con 0x7effd40cd020 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd9396640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7effd40ccc60 con 0x7effd40cdac0 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd9396640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7effd40ce410 con 0x7effd4162db0 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd2ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7effd40cd020 0x7effd40cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd2ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7effd40cd020 0x7effd40cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47918/0 (socket says 172.21.15.55:47918) 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd2ffd640 1 -- 172.21.15.55:0/3419150508 learned_addr learned my addr 172.21.15.55:0/3419150508 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd37fe640 1 --2- 172.21.15.55:0/3419150508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd4162db0 0x7effd41631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd27fc640 1 --2- 172.21.15.55:0/3419150508 >> v2:172.21.15.138:3300/0 conn(0x7effd40cdac0 0x7effd40cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd2ffd640 1 -- 172.21.15.55:0/3419150508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd4162db0 msgr2=0x7effd41631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd2ffd640 1 --2- 172.21.15.55:0/3419150508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd4162db0 0x7effd41631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd2ffd640 1 -- 172.21.15.55:0/3419150508 >> v2:172.21.15.138:3300/0 conn(0x7effd40cdac0 msgr2=0x7effd40cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd2ffd640 1 --2- 172.21.15.55:0/3419150508 >> v2:172.21.15.138:3300/0 conn(0x7effd40cdac0 0x7effd40cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.220+0000 7effd2ffd640 1 -- 172.21.15.55:0/3419150508 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd40cc3c0 con 0x7effd40cd020 2024-05-20T22:09:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.221+0000 7effd27fc640 1 --2- 172.21.15.55:0/3419150508 >> v2:172.21.15.138:3300/0 conn(0x7effd40cdac0 0x7effd40cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.221+0000 7effd2ffd640 1 --2- 172.21.15.55:0/3419150508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cd020 0x7effd40cd430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7effd40d21b0 tx=0x7effc402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e66c2dd7f6d6604 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:03.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.221+0000 7effd1ffb640 1 -- 172.21.15.55:0/3419150508 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4036070 con 0x7effd40cd020 2024-05-20T22:09:03.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.221+0000 7effd1ffb640 1 -- 172.21.15.55:0/3419150508 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc4031600 con 0x7effd40cd020 2024-05-20T22:09:03.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.221+0000 7effd1ffb640 1 -- 172.21.15.55:0/3419150508 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc40319d0 con 0x7effd40cd020 2024-05-20T22:09:03.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.222+0000 7effd9396640 1 -- 172.21.15.55:0/3419150508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cd020 msgr2=0x7effd40cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.222+0000 7effd9396640 1 --2- 172.21.15.55:0/3419150508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cd020 0x7effd40cd430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7effd40d21b0 tx=0x7effc402a6a0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.222+0000 7effd9396640 1 -- 172.21.15.55:0/3419150508 shutdown_connections 2024-05-20T22:09:03.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.222+0000 7effd9396640 1 --2- 172.21.15.55:0/3419150508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd4162db0 0x7effd41631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.222+0000 7effd9396640 1 --2- 172.21.15.55:0/3419150508 >> v2:172.21.15.138:3300/0 conn(0x7effd40cdac0 0x7effd40cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.222+0000 7effd9396640 1 --2- 172.21.15.55:0/3419150508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cd020 0x7effd40cd430 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.222+0000 7effd9396640 1 -- 172.21.15.55:0/3419150508 >> 172.21.15.55:0/3419150508 conn(0x7effd40b62c0 msgr2=0x7effd409a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:03.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.222+0000 7effd9396640 1 -- 172.21.15.55:0/3419150508 shutdown_connections 2024-05-20T22:09:03.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.222+0000 7effd9396640 1 -- 172.21.15.55:0/3419150508 wait complete. 2024-05-20T22:09:03.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.224+0000 7effd9396640 1 Processor -- start 2024-05-20T22:09:03.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.224+0000 7effd9396640 1 -- start start 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.225+0000 7effd9396640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd40cd020 0x7effd40c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd9396640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cdac0 0x7effd415c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd9396640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effd4162db0 0x7effd4070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd9396640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7effd40d0380 con 0x7effd40cdac0 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd9396640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7effd40cc980 con 0x7effd4162db0 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd9396640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7effd415f4d0 con 0x7effd40cd020 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cdac0 0x7effd415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cdac0 0x7effd415c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47934/0 (socket says 172.21.15.55:47934) 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd27fc640 1 -- 172.21.15.55:0/350872357 learned_addr learned my addr 172.21.15.55:0/350872357 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd2ffd640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd40cd020 0x7effd40c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd37fe640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effd4162db0 0x7effd4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd27fc640 1 -- 172.21.15.55:0/350872357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd40cd020 msgr2=0x7effd40c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd27fc640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd40cd020 0x7effd40c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd27fc640 1 -- 172.21.15.55:0/350872357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effd4162db0 msgr2=0x7effd4070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd27fc640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effd4162db0 0x7effd4070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd27fc640 1 -- 172.21.15.55:0/350872357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd40c5470 con 0x7effd40cdac0 2024-05-20T22:09:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effd27fc640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cdac0 0x7effd415c820 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7effbc00c820 tx=0x7effbc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effcb7fe640 1 -- 172.21.15.55:0/350872357 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc015070 con 0x7effd40cdac0 2024-05-20T22:09:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.226+0000 7effcb7fe640 1 -- 172.21.15.55:0/350872357 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effbc004020 con 0x7effd40cdac0 2024-05-20T22:09:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.227+0000 7effcb7fe640 1 -- 172.21.15.55:0/350872357 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc004410 con 0x7effd40cdac0 2024-05-20T22:09:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.227+0000 7effd9396640 1 -- 172.21.15.55:0/350872357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd40c5750 con 0x7effd40cdac0 2024-05-20T22:09:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.227+0000 7effd9396640 1 -- 172.21.15.55:0/350872357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd40c5d10 con 0x7effd40cdac0 2024-05-20T22:09:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.228+0000 7effd9396640 1 -- 172.21.15.55:0/350872357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd4069090 con 0x7effd40cdac0 2024-05-20T22:09:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.228+0000 7effcb7fe640 1 -- 172.21.15.55:0/350872357 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7effbc007830 con 0x7effd40cdac0 2024-05-20T22:09:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.229+0000 7effcb7fe640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7effb406be30 0x7effb406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.229+0000 7effd2ffd640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7effb406be30 0x7effb406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.229+0000 7effcb7fe640 1 -- 172.21.15.55:0/350872357 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7effbc08b110 con 0x7effd40cdac0 2024-05-20T22:09:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.229+0000 7effd2ffd640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7effb406be30 0x7effb406e340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7effc4033000 tx=0x7effc4009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.234+0000 7effcb7fe640 1 -- 172.21.15.55:0/350872357 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7effbc058fe0 con 0x7effd40cdac0 2024-05-20T22:09:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.454+0000 7effd9396640 1 -- 172.21.15.55:0/350872357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7effd41600a0 con 0x7effd40cdac0 2024-05-20T22:09:03.455 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-20T22:09:03.454+0000 7f0b3ef60640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T22:09:03.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.457+0000 7effcb7fe640 1 -- 172.21.15.55:0/350872357 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7effbc05d2e0 con 0x7effd40cdac0 2024-05-20T22:09:03.458 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T22:09:03.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 -- 172.21.15.55:0/350872357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7effb406be30 msgr2=0x7effb406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7effb406be30 0x7effb406e340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7effc4033000 tx=0x7effc4009000 comp rx=0 tx=0).stop 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 -- 172.21.15.55:0/350872357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cdac0 msgr2=0x7effd415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cdac0 0x7effd415c820 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7effbc00c820 tx=0x7effbc00cce0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 -- 172.21.15.55:0/350872357 shutdown_connections 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7effb406be30 0x7effb406e340 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effd4162db0 0x7effd4070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7effd40cdac0 0x7effd415c820 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 --2- 172.21.15.55:0/350872357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effd40cd020 0x7effd40c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 -- 172.21.15.55:0/350872357 >> 172.21.15.55:0/350872357 conn(0x7effd40b62c0 msgr2=0x7effd415d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.459+0000 7effd9396640 1 -- 172.21.15.55:0/350872357 shutdown_connections 2024-05-20T22:09:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.460+0000 7effd9396640 1 -- 172.21.15.55:0/350872357 wait complete. 2024-05-20T22:09:03.465 INFO:tasks.ceph.mds.d.smithi138.stderr: -4790> 2024-05-20T22:07:57.623+0000 7ffaeba0a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T22:09:03.473 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T22:09:03.522 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-05-20T22:09:03.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.589+0000 7f9838bcc640 1 Processor -- start 2024-05-20T22:09:03.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.589+0000 7f9838bcc640 1 -- start start 2024-05-20T22:09:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.590+0000 7f9838bcc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98340d4210 0x7f98340b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.590+0000 7f9838bcc640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f98340d0350 0x7f983406cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.590+0000 7f9838bcc640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9834163980 con 0x7f98340cf7b0 2024-05-20T22:09:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.590+0000 7f9838bcc640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f98340d3d80 con 0x7f98340d4210 2024-05-20T22:09:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.590+0000 7f9838bcc640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f98340d3450 con 0x7f98340d0350 2024-05-20T22:09:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.590+0000 7f983259c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98340d4210 0x7f98340b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.590+0000 7f9832d9d640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f98340cf7b0 0x7f98340cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50996/0 (socket says 172.21.15.55:50996) 2024-05-20T22:09:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.590+0000 7f983259c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98340d4210 0x7f98340b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47938/0 (socket says 172.21.15.55:47938) 2024-05-20T22:09:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.590+0000 7f9832d9d640 1 -- 172.21.15.55:0/3365507992 learned_addr learned my addr 172.21.15.55:0/3365507992 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.591+0000 7f9829d9b640 1 --2- 172.21.15.55:0/3365507992 >> v2:172.21.15.138:3300/0 conn(0x7f98340d0350 0x7f983406cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.591+0000 7f983259c640 1 -- 172.21.15.55:0/3365507992 >> v2:172.21.15.138:3300/0 conn(0x7f98340d0350 msgr2=0x7f983406cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:03.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.591+0000 7f983259c640 1 --2- 172.21.15.55:0/3365507992 >> v2:172.21.15.138:3300/0 conn(0x7f98340d0350 0x7f983406cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.591+0000 7f983259c640 1 -- 172.21.15.55:0/3365507992 >> v1:172.21.15.55:6789/0 conn(0x7f98340cf7b0 legacy=0x7f98340cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.591+0000 7f9831d9b640 1 -- 172.21.15.55:0/3365507992 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213172561 0 0) 0x7f9834163980 con 0x7f98340cf7b0 2024-05-20T22:09:03.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.591+0000 7f983259c640 1 -- 172.21.15.55:0/3365507992 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98340d3730 con 0x7f98340d4210 2024-05-20T22:09:03.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.592+0000 7f983259c640 1 --2- 172.21.15.55:0/3365507992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340d4210 0x7f98340b3b20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f98340d5be0 tx=0x7f982402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb64af9ab0fae7cf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:03.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.592+0000 7f9831d9b640 1 -- 172.21.15.55:0/3365507992 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824037070 con 0x7f98340d4210 2024-05-20T22:09:03.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.592+0000 7f9831d9b640 1 -- 172.21.15.55:0/3365507992 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9824032440 con 0x7f98340d4210 2024-05-20T22:09:03.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.592+0000 7f9831d9b640 1 -- 172.21.15.55:0/3365507992 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982402ae60 con 0x7f98340d4210 2024-05-20T22:09:03.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.592+0000 7f9838bcc640 1 -- 172.21.15.55:0/3365507992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340d4210 msgr2=0x7f98340b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.592+0000 7f9838bcc640 1 --2- 172.21.15.55:0/3365507992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340d4210 0x7f98340b3b20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f98340d5be0 tx=0x7f982402a110 comp rx=0 tx=0).stop 2024-05-20T22:09:03.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.592+0000 7f9838bcc640 1 -- 172.21.15.55:0/3365507992 shutdown_connections 2024-05-20T22:09:03.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.592+0000 7f9838bcc640 1 --2- 172.21.15.55:0/3365507992 >> v2:172.21.15.138:3300/0 conn(0x7f98340d0350 0x7f983406cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.592+0000 7f9838bcc640 1 --2- 172.21.15.55:0/3365507992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340d4210 0x7f98340b3b20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.592+0000 7f9838bcc640 1 -- 172.21.15.55:0/3365507992 >> 172.21.15.55:0/3365507992 conn(0x7f98340d1bb0 msgr2=0x7f98340d4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:03.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.593+0000 7f9838bcc640 1 -- 172.21.15.55:0/3365507992 shutdown_connections 2024-05-20T22:09:03.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.593+0000 7f9838bcc640 1 -- 172.21.15.55:0/3365507992 wait complete. 2024-05-20T22:09:03.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.595+0000 7f9838bcc640 1 Processor -- start 2024-05-20T22:09:03.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.595+0000 7f9838bcc640 1 -- start start 2024-05-20T22:09:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f9838bcc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340cf7b0 0x7f983415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f9838bcc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98340d0350 0x7f98340cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f9838bcc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98340d4210 0x7f9834160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f9838bcc640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9834163720 con 0x7f98340cf7b0 2024-05-20T22:09:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f9838bcc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98341635a0 con 0x7f98340d4210 2024-05-20T22:09:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f9838bcc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f983415e750 con 0x7f98340d0350 2024-05-20T22:09:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f983259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340cf7b0 0x7f983415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f983259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340cf7b0 0x7f983415a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47954/0 (socket says 172.21.15.55:47954) 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f983259c640 1 -- 172.21.15.55:0/1337233829 learned_addr learned my addr 172.21.15.55:0/1337233829 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f9832d9d640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98340d4210 0x7f9834160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f9829d9b640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98340d0350 0x7f98340cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f983259c640 1 -- 172.21.15.55:0/1337233829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98340d0350 msgr2=0x7f98340cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f983259c640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98340d0350 0x7f98340cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f983259c640 1 -- 172.21.15.55:0/1337233829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98340d4210 msgr2=0x7f9834160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f983259c640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98340d4210 0x7f9834160cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.596+0000 7f983259c640 1 -- 172.21.15.55:0/1337233829 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98340ccb50 con 0x7f98340cf7b0 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.597+0000 7f983259c640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340cf7b0 0x7f983415a1f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9824009890 tx=0x7f98240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.597+0000 7f982b7fe640 1 -- 172.21.15.55:0/1337233829 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824037040 con 0x7f98340cf7b0 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.597+0000 7f982b7fe640 1 -- 172.21.15.55:0/1337233829 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9824042070 con 0x7f98340cf7b0 2024-05-20T22:09:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.597+0000 7f9838bcc640 1 -- 172.21.15.55:0/1337233829 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98340cc870 con 0x7f98340cf7b0 2024-05-20T22:09:03.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.597+0000 7f982b7fe640 1 -- 172.21.15.55:0/1337233829 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824030960 con 0x7f98340cf7b0 2024-05-20T22:09:03.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.597+0000 7f9838bcc640 1 -- 172.21.15.55:0/1337233829 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98340cd820 con 0x7f98340cf7b0 2024-05-20T22:09:03.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.598+0000 7f9838bcc640 1 -- 172.21.15.55:0/1337233829 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9834069090 con 0x7f98340cf7b0 2024-05-20T22:09:03.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.599+0000 7f982b7fe640 1 -- 172.21.15.55:0/1337233829 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9824030b00 con 0x7f98340cf7b0 2024-05-20T22:09:03.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.599+0000 7f982b7fe640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f981406be60 0x7f981406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:03.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.599+0000 7f982b7fe640 1 -- 172.21.15.55:0/1337233829 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f98240aba00 con 0x7f98340cf7b0 2024-05-20T22:09:03.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.599+0000 7f9829d9b640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f981406be60 0x7f981406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:03.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.600+0000 7f9829d9b640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f981406be60 0x7f981406e370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f983415d9e0 tx=0x7f981c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:03.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.604+0000 7f982b7fe640 1 -- 172.21.15.55:0/1337233829 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9824079990 con 0x7f98340cf7b0 2024-05-20T22:09:03.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:03.823+0000 7f9838bcc640 1 -- 172.21.15.55:0/1337233829 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f98340d2f70 con 0x7f98340cf7b0 2024-05-20T22:09:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.460+0000 7f982b7fe640 1 -- 172.21.15.55:0/1337233829 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f982407ee20 con 0x7f98340cf7b0 2024-05-20T22:09:04.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.463+0000 7f9838bcc640 1 -- 172.21.15.55:0/1337233829 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f981406be60 msgr2=0x7f981406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.463+0000 7f9838bcc640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f981406be60 0x7f981406e370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f983415d9e0 tx=0x7f981c009190 comp rx=0 tx=0).stop 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.463+0000 7f9838bcc640 1 -- 172.21.15.55:0/1337233829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340cf7b0 msgr2=0x7f983415a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.463+0000 7f9838bcc640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340cf7b0 0x7f983415a1f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9824009890 tx=0x7f98240046c0 comp rx=0 tx=0).stop 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.463+0000 7f9838bcc640 1 -- 172.21.15.55:0/1337233829 shutdown_connections 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.463+0000 7f9838bcc640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f981406be60 0x7f981406e370 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.464+0000 7f9838bcc640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98340d4210 0x7f9834160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.464+0000 7f9838bcc640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98340d0350 0x7f98340cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.464+0000 7f9838bcc640 1 --2- 172.21.15.55:0/1337233829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98340cf7b0 0x7f983415a1f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.464+0000 7f9838bcc640 1 -- 172.21.15.55:0/1337233829 >> 172.21.15.55:0/1337233829 conn(0x7f98340d1bb0 msgr2=0x7f98340b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.464+0000 7f9838bcc640 1 -- 172.21.15.55:0/1337233829 shutdown_connections 2024-05-20T22:09:04.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.464+0000 7f9838bcc640 1 -- 172.21.15.55:0/1337233829 wait complete. 2024-05-20T22:09:04.478 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T22:09:04.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.600+0000 7f11bb4fe640 1 Processor -- start 2024-05-20T22:09:04.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.600+0000 7f11bb4fe640 1 -- start start 2024-05-20T22:09:04.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.601+0000 7f11bb4fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f11b40d32a0 0x7f11b415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:04.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.601+0000 7f11bb4fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d01b0 0x7f11b406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:04.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.601+0000 7f11bb4fe640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11b40d3780 con 0x7f11b40cf690 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.601+0000 7f11bb4fe640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f11b40d3a60 con 0x7f11b40d32a0 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.601+0000 7f11bb4fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11b40d3d40 con 0x7f11b40d01b0 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.601+0000 7f11b929a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f11b40d32a0 0x7f11b415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.601+0000 7f11b8a99640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f11b40cf690 0x7f11b40cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51012/0 (socket says 172.21.15.55:51012) 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.601+0000 7f11b8a99640 1 -- 172.21.15.55:0/940176773 learned_addr learned my addr 172.21.15.55:0/940176773 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.602+0000 7f11b9a9b640 1 --2- 172.21.15.55:0/940176773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d01b0 0x7f11b406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.602+0000 7f11b929a640 1 -- 172.21.15.55:0/940176773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d01b0 msgr2=0x7f11b406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.602+0000 7f11b929a640 1 --2- 172.21.15.55:0/940176773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d01b0 0x7f11b406b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.602+0000 7f11b929a640 1 -- 172.21.15.55:0/940176773 >> v1:172.21.15.55:6789/0 conn(0x7f11b40cf690 legacy=0x7f11b40cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.602+0000 7f11a7fff640 1 -- 172.21.15.55:0/940176773 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237029973 0 0) 0x7f11b40d3780 con 0x7f11b40cf690 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.602+0000 7f11b929a640 1 -- 172.21.15.55:0/940176773 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b40d4020 con 0x7f11b40d32a0 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.602+0000 7f11b9a9b640 1 --2- 172.21.15.55:0/940176773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d01b0 0x7f11b406b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:04.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.603+0000 7f11b929a640 1 --2- 172.21.15.55:0/940176773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40d32a0 0x7f11b415f150 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f11b00097a0 tx=0x7f11b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf16dad48e9963d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:04.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.603+0000 7f11a7fff640 1 -- 172.21.15.55:0/940176773 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0037070 con 0x7f11b40d32a0 2024-05-20T22:09:04.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.603+0000 7f11a7fff640 1 -- 172.21.15.55:0/940176773 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b0032440 con 0x7f11b40d32a0 2024-05-20T22:09:04.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.603+0000 7f11a7fff640 1 -- 172.21.15.55:0/940176773 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0032810 con 0x7f11b40d32a0 2024-05-20T22:09:04.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.603+0000 7f11bb4fe640 1 -- 172.21.15.55:0/940176773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40d32a0 msgr2=0x7f11b415f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:04.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.603+0000 7f11bb4fe640 1 --2- 172.21.15.55:0/940176773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40d32a0 0x7f11b415f150 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f11b00097a0 tx=0x7f11b002a180 comp rx=0 tx=0).stop 2024-05-20T22:09:04.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.603+0000 7f11bb4fe640 1 -- 172.21.15.55:0/940176773 shutdown_connections 2024-05-20T22:09:04.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.603+0000 7f11bb4fe640 1 --2- 172.21.15.55:0/940176773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d01b0 0x7f11b406b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:04.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.603+0000 7f11bb4fe640 1 --2- 172.21.15.55:0/940176773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40d32a0 0x7f11b415f150 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:04.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.604+0000 7f11bb4fe640 1 -- 172.21.15.55:0/940176773 >> 172.21.15.55:0/940176773 conn(0x7f11b40b6de0 msgr2=0x7f11b40d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:04.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.604+0000 7f11bb4fe640 1 -- 172.21.15.55:0/940176773 shutdown_connections 2024-05-20T22:09:04.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.604+0000 7f11bb4fe640 1 -- 172.21.15.55:0/940176773 wait complete. 2024-05-20T22:09:04.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.605+0000 7f11bb4fe640 1 Processor -- start 2024-05-20T22:09:04.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.605+0000 7f11bb4fe640 1 -- start start 2024-05-20T22:09:04.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11bb4fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40cf690 0x7f11b40c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:04.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11bb4fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11b40d01b0 0x7f11b414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:04.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11bb4fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d32a0 0x7f11b40ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:04.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11bb4fe640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f11b40d4820 con 0x7f11b40cf690 2024-05-20T22:09:04.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11bb4fe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11b40b4350 con 0x7f11b40d01b0 2024-05-20T22:09:04.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11bb4fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11b40c9040 con 0x7f11b40d32a0 2024-05-20T22:09:04.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b929a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40cf690 0x7f11b40c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:04.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b929a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40cf690 0x7f11b40c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47968/0 (socket says 172.21.15.55:47968) 2024-05-20T22:09:04.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b929a640 1 -- 172.21.15.55:0/4137982798 learned_addr learned my addr 172.21.15.55:0/4137982798 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:04.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b9a9b640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d32a0 0x7f11b40ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b8a99640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11b40d01b0 0x7f11b414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b929a640 1 -- 172.21.15.55:0/4137982798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d32a0 msgr2=0x7f11b40ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b929a640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d32a0 0x7f11b40ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b929a640 1 -- 172.21.15.55:0/4137982798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11b40d01b0 msgr2=0x7f11b414ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b929a640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11b40d01b0 0x7f11b414ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b929a640 1 -- 172.21.15.55:0/4137982798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b40c9850 con 0x7f11b40cf690 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11b929a640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40cf690 0x7f11b40c87d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f11b00098c0 tx=0x7f11b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11a5ffb640 1 -- 172.21.15.55:0/4137982798 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0037040 con 0x7f11b40cf690 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.607+0000 7f11a5ffb640 1 -- 172.21.15.55:0/4137982798 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b0042070 con 0x7f11b40cf690 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.608+0000 7f11bb4fe640 1 -- 172.21.15.55:0/4137982798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b40c9290 con 0x7f11b40cf690 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.608+0000 7f11a5ffb640 1 -- 172.21.15.55:0/4137982798 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0030bb0 con 0x7f11b40cf690 2024-05-20T22:09:04.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.608+0000 7f11bb4fe640 1 -- 172.21.15.55:0/4137982798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b40cbad0 con 0x7f11b40cf690 2024-05-20T22:09:04.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.609+0000 7f11bb4fe640 1 -- 172.21.15.55:0/4137982798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b4069090 con 0x7f11b40cf690 2024-05-20T22:09:04.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.609+0000 7f11a5ffb640 1 -- 172.21.15.55:0/4137982798 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f11b0047460 con 0x7f11b40cf690 2024-05-20T22:09:04.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.610+0000 7f11a5ffb640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f119806be60 0x7f119806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:04.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.610+0000 7f11a5ffb640 1 -- 172.21.15.55:0/4137982798 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f11b00abe10 con 0x7f11b40cf690 2024-05-20T22:09:04.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.610+0000 7f11b8a99640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f119806be60 0x7f119806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:04.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.610+0000 7f11b8a99640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f119806be60 0x7f119806e370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f11b406c7a0 tx=0x7f11a8007c00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:04.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.614+0000 7f11a5ffb640 1 -- 172.21.15.55:0/4137982798 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f11b0032e20 con 0x7f11b40cf690 2024-05-20T22:09:04.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:04.803+0000 7f11bb4fe640 1 -- 172.21.15.55:0/4137982798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f11b40d2e50 con 0x7f11b40cf690 2024-05-20T22:09:05.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.463+0000 7f11a5ffb640 1 -- 172.21.15.55:0/4137982798 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7f11b007a360 con 0x7f11b40cf690 2024-05-20T22:09:05.464 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-05-20T22:09:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 -- 172.21.15.55:0/4137982798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f119806be60 msgr2=0x7f119806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f119806be60 0x7f119806e370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f11b406c7a0 tx=0x7f11a8007c00 comp rx=0 tx=0).stop 2024-05-20T22:09:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 -- 172.21.15.55:0/4137982798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40cf690 msgr2=0x7f11b40c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40cf690 0x7f11b40c87d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f11b00098c0 tx=0x7f11b00046c0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 -- 172.21.15.55:0/4137982798 shutdown_connections 2024-05-20T22:09:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f119806be60 0x7f119806e370 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11b40d32a0 0x7f11b40ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11b40d01b0 0x7f11b414ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 --2- 172.21.15.55:0/4137982798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11b40cf690 0x7f11b40c87d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 -- 172.21.15.55:0/4137982798 >> 172.21.15.55:0/4137982798 conn(0x7f11b40b6de0 msgr2=0x7f11b415e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:05.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 -- 172.21.15.55:0/4137982798 shutdown_connections 2024-05-20T22:09:05.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.466+0000 7f11bb4fe640 1 -- 172.21.15.55:0/4137982798 wait complete. 2024-05-20T22:09:05.484 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T22:09:05.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.606+0000 7f3392273640 1 Processor -- start 2024-05-20T22:09:05.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.606+0000 7f3392273640 1 -- start start 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f3392273640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f338c15f150 0x7f338c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f3392273640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f338c0d3780 con 0x7f338c0d32a0 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f3392273640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f338c0d3a60 con 0x7f338c06b500 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f3392273640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f338c0d3d40 con 0x7f338c15f150 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f338bfff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f338c0d32a0 0x7f338c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51016/0 (socket says 172.21.15.55:51016) 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f338bfff640 1 -- 172.21.15.55:0/3677409467 learned_addr learned my addr 172.21.15.55:0/3677409467 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 386298414 0 0) 0x7f338c0d3780 con 0x7f338c0d32a0 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3374003520 con 0x7f338c0d32a0 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f338b7fe640 1 --2- 172.21.15.55:0/3677409467 >> v2:172.21.15.138:3300/0 conn(0x7f338c15f150 0x7f338c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2003279399 0 0) 0x7f3374003520 con 0x7f338c0d32a0 2024-05-20T22:09:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 >> v2:172.21.15.138:3300/0 conn(0x7f338c15f150 msgr2=0x7f338c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f338affd640 1 --2- 172.21.15.55:0/3677409467 >> v2:172.21.15.138:3300/0 conn(0x7f338c15f150 0x7f338c0cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 >> v1:172.21.15.138:6789/0 conn(0x7f338c06b500 legacy=0x7f338c06b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.608+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c0d4020 con 0x7f338c0d32a0 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f337c001670 con 0x7f338c0d32a0 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 >> v1:172.21.15.55:6789/0 conn(0x7f338c0d32a0 legacy=0x7f338c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338affd640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33740039b0 0x7f3374003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338affd640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3374004800 0x7f3374004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338affd640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33740051c0 0x7f337400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f338c0d3d40 con 0x7f3374004800 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f337400c160 con 0x7f33740039b0 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f337400c360 con 0x7f33740051c0 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338b7fe640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3374004800 0x7f3374004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:05.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338bfff640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33740039b0 0x7f3374003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:05.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f3390810640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33740051c0 0x7f337400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:05.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338b7fe640 1 -- 172.21.15.55:0/3677409467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33740051c0 msgr2=0x7f337400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:05.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338b7fe640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33740051c0 0x7f337400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338b7fe640 1 -- 172.21.15.55:0/3677409467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33740039b0 msgr2=0x7f3374003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:05.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338b7fe640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33740039b0 0x7f3374003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f338b7fe640 1 -- 172.21.15.55:0/3677409467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3374003520 con 0x7f3374004800 2024-05-20T22:09:05.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f3390810640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33740051c0 0x7f337400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:05.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.609+0000 7f3390810640 1 -- 172.21.15.55:0/3677409467 reap_dead start 2024-05-20T22:09:05.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.610+0000 7f338b7fe640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3374004800 0x7f3374004c10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3380009ac0 tx=0x7f3374005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd68aadc8fc76bd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:05.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.610+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380036070 con 0x7f3374004800 2024-05-20T22:09:05.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.610+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3380031400 con 0x7f3374004800 2024-05-20T22:09:05.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.610+0000 7f338affd640 1 -- 172.21.15.55:0/3677409467 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380009690 con 0x7f3374004800 2024-05-20T22:09:05.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.611+0000 7f3392273640 1 -- 172.21.15.55:0/3677409467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3374004800 msgr2=0x7f3374004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:05.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.611+0000 7f3392273640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3374004800 0x7f3374004c10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3380009ac0 tx=0x7f3374005830 comp rx=0 tx=0).stop 2024-05-20T22:09:05.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.611+0000 7f3392273640 1 -- 172.21.15.55:0/3677409467 shutdown_connections 2024-05-20T22:09:05.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.611+0000 7f3392273640 1 --2- 172.21.15.55:0/3677409467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3374004800 0x7f3374004c10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.611+0000 7f3392273640 1 -- 172.21.15.55:0/3677409467 >> 172.21.15.55:0/3677409467 conn(0x7f338c0b6de0 msgr2=0x7f338c0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:05.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.611+0000 7f3392273640 1 -- 172.21.15.55:0/3677409467 shutdown_connections 2024-05-20T22:09:05.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.611+0000 7f3392273640 1 -- 172.21.15.55:0/3677409467 wait complete. 2024-05-20T22:09:05.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.612+0000 7f3392273640 1 Processor -- start 2024-05-20T22:09:05.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.613+0000 7f3392273640 1 -- start start 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f3392273640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3374004800 0x7f338c158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f3392273640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f338c1595c0 0x7f338c161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f3392273640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f338c1f5160 0x7f338c161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f3392273640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f338c0d4820 con 0x7f338c1595c0 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f3392273640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f338c15f560 con 0x7f338c1f5160 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f3392273640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f338c1f5c50 con 0x7f3374004800 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f338c1595c0 0x7f338c161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f338c1595c0 0x7f338c161dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47992/0 (socket says 172.21.15.55:47992) 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f338b7fe640 1 -- 172.21.15.55:0/2289035594 learned_addr learned my addr 172.21.15.55:0/2289035594 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f338bfff640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3374004800 0x7f338c158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f3390810640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f338c1f5160 0x7f338c161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f338b7fe640 1 -- 172.21.15.55:0/2289035594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3374004800 msgr2=0x7f338c158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:05.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f338b7fe640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3374004800 0x7f338c158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f338b7fe640 1 -- 172.21.15.55:0/2289035594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f338c1f5160 msgr2=0x7f338c161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.614+0000 7f338b7fe640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f338c1f5160 0x7f338c161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.615+0000 7f338b7fe640 1 -- 172.21.15.55:0/2289035594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c1625d0 con 0x7f338c1595c0 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.615+0000 7f338bfff640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3374004800 0x7f338c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.615+0000 7f3390810640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f338c1f5160 0x7f338c161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.615+0000 7f338b7fe640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f338c1595c0 0x7f338c161dc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3380034040 tx=0x7f3380004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.615+0000 7f3388ff9640 1 -- 172.21.15.55:0/2289035594 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338003f020 con 0x7f338c1595c0 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.615+0000 7f3388ff9640 1 -- 172.21.15.55:0/2289035594 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3380037040 con 0x7f338c1595c0 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.615+0000 7f3392273640 1 -- 172.21.15.55:0/2289035594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338c162300 con 0x7f338c1595c0 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.615+0000 7f3388ff9640 1 -- 172.21.15.55:0/2289035594 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380036070 con 0x7f338c1595c0 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.615+0000 7f3392273640 1 -- 172.21.15.55:0/2289035594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338c160570 con 0x7f338c1595c0 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.616+0000 7f3392273640 1 -- 172.21.15.55:0/2289035594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f338c069090 con 0x7f338c1595c0 2024-05-20T22:09:05.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.617+0000 7f3388ff9640 1 -- 172.21.15.55:0/2289035594 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3380042020 con 0x7f338c1595c0 2024-05-20T22:09:05.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.617+0000 7f3388ff9640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f337806be60 0x7f337806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:05.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.617+0000 7f338bfff640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f337806be60 0x7f337806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:05.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.617+0000 7f3388ff9640 1 -- 172.21.15.55:0/2289035594 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7f33800ab2d0 con 0x7f338c1595c0 2024-05-20T22:09:05.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.618+0000 7f338bfff640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f337806be60 0x7f337806e370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f337c007db0 tx=0x7f337c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:05.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.622+0000 7f3388ff9640 1 -- 172.21.15.55:0/2289035594 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3380049030 con 0x7f338c1595c0 2024-05-20T22:09:05.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:05.810+0000 7f3392273640 1 -- 172.21.15.55:0/2289035594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f338c0d2e50 con 0x7f338c1595c0 2024-05-20T22:09:06.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.465+0000 7f3388ff9640 1 -- 172.21.15.55:0/2289035594 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f3380079aa0 con 0x7f338c1595c0 2024-05-20T22:09:06.466 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-05-20T22:09:06.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 -- 172.21.15.55:0/2289035594 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f337806be60 msgr2=0x7f337806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:06.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f337806be60 0x7f337806e370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f337c007db0 tx=0x7f337c009040 comp rx=0 tx=0).stop 2024-05-20T22:09:06.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 -- 172.21.15.55:0/2289035594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f338c1595c0 msgr2=0x7f338c161dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:06.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f338c1595c0 0x7f338c161dc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3380034040 tx=0x7f3380004730 comp rx=0 tx=0).stop 2024-05-20T22:09:06.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 -- 172.21.15.55:0/2289035594 shutdown_connections 2024-05-20T22:09:06.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f337806be60 0x7f337806e370 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:06.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f338c1f5160 0x7f338c161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:06.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f338c1595c0 0x7f338c161dc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:06.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 --2- 172.21.15.55:0/2289035594 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3374004800 0x7f338c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:06.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 -- 172.21.15.55:0/2289035594 >> 172.21.15.55:0/2289035594 conn(0x7f338c0b6de0 msgr2=0x7f338c0b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:06.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 -- 172.21.15.55:0/2289035594 shutdown_connections 2024-05-20T22:09:06.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.468+0000 7f3392273640 1 -- 172.21.15.55:0/2289035594 wait complete. 2024-05-20T22:09:06.486 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-20T22:09:06.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.611+0000 7f1e967e3640 1 Processor -- start 2024-05-20T22:09:06.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.611+0000 7f1e967e3640 1 -- start start 2024-05-20T22:09:06.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e967e3640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1e9015f150 0x7f1e900cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:06.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e967e3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9006b500 0x7f1e9006b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:06.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e967e3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e900d3780 con 0x7f1e900d32a0 2024-05-20T22:09:06.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e967e3640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f1e900d3a60 con 0x7f1e9015f150 2024-05-20T22:09:06.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e967e3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1e900d3d40 con 0x7f1e9006b500 2024-05-20T22:09:06.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e8f7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1e9015f150 0x7f1e900cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:06.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e8f7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1e9015f150 0x7f1e900cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48004/0 (socket says 172.21.15.55:48004) 2024-05-20T22:09:06.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e8f7fe640 1 -- 172.21.15.55:0/2402044254 learned_addr learned my addr 172.21.15.55:0/2402044254 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:06.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e94d80640 1 --2- 172.21.15.55:0/2402044254 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9006b500 0x7f1e9006b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:06.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e8f7fe640 1 -- 172.21.15.55:0/2402044254 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9006b500 msgr2=0x7f1e9006b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:06.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e8f7fe640 1 --2- 172.21.15.55:0/2402044254 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9006b500 0x7f1e9006b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:06.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.612+0000 7f1e8f7fe640 1 -- 172.21.15.55:0/2402044254 >> v1:172.21.15.138:6789/0 conn(0x7f1e900d32a0 legacy=0x7f1e900cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:06.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.613+0000 7f1e8f7fe640 1 -- 172.21.15.55:0/2402044254 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e900d4020 con 0x7f1e9015f150 2024-05-20T22:09:06.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.613+0000 7f1e8f7fe640 1 --2- 172.21.15.55:0/2402044254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9015f150 0x7f1e900cfda0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009870 tx=0x7f1e8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d969b811f52308 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:06.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.613+0000 7f1e8effd640 1 -- 172.21.15.55:0/2402044254 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84037070 con 0x7f1e9015f150 2024-05-20T22:09:06.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.613+0000 7f1e8effd640 1 -- 172.21.15.55:0/2402044254 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e84032440 con 0x7f1e9015f150 2024-05-20T22:09:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.613+0000 7f1e8effd640 1 -- 172.21.15.55:0/2402044254 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84032810 con 0x7f1e9015f150 2024-05-20T22:09:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.614+0000 7f1e967e3640 1 -- 172.21.15.55:0/2402044254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9015f150 msgr2=0x7f1e900cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.614+0000 7f1e967e3640 1 --2- 172.21.15.55:0/2402044254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9015f150 0x7f1e900cfda0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009870 tx=0x7f1e8402a270 comp rx=0 tx=0).stop 2024-05-20T22:09:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.614+0000 7f1e967e3640 1 -- 172.21.15.55:0/2402044254 shutdown_connections 2024-05-20T22:09:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.614+0000 7f1e967e3640 1 --2- 172.21.15.55:0/2402044254 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9006b500 0x7f1e9006b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.614+0000 7f1e967e3640 1 --2- 172.21.15.55:0/2402044254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9015f150 0x7f1e900cfda0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.614+0000 7f1e967e3640 1 -- 172.21.15.55:0/2402044254 >> 172.21.15.55:0/2402044254 conn(0x7f1e900b6de0 msgr2=0x7f1e900d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.614+0000 7f1e967e3640 1 -- 172.21.15.55:0/2402044254 shutdown_connections 2024-05-20T22:09:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.614+0000 7f1e967e3640 1 -- 172.21.15.55:0/2402044254 wait complete. 2024-05-20T22:09:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.616+0000 7f1e967e3640 1 Processor -- start 2024-05-20T22:09:06.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.616+0000 7f1e967e3640 1 -- start start 2024-05-20T22:09:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e967e3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9006b500 0x7f1e900ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e967e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e900d32a0 0x7f1e90155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e967e3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9015f150 0x7f1e900b98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e967e3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1e900d4820 con 0x7f1e9006b500 2024-05-20T22:09:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e967e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9015f560 con 0x7f1e900d32a0 2024-05-20T22:09:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e967e3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9015fc10 con 0x7f1e9015f150 2024-05-20T22:09:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9006b500 0x7f1e900ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9006b500 0x7f1e900ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48014/0 (socket says 172.21.15.55:48014) 2024-05-20T22:09:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8ffff640 1 -- 172.21.15.55:0/292741531 learned_addr learned my addr 172.21.15.55:0/292741531 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e94d80640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9015f150 0x7f1e900b98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8f7fe640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e900d32a0 0x7f1e90155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8ffff640 1 -- 172.21.15.55:0/292741531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9015f150 msgr2=0x7f1e900b98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8ffff640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9015f150 0x7f1e900b98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8ffff640 1 -- 172.21.15.55:0/292741531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e900d32a0 msgr2=0x7f1e90155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8ffff640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e900d32a0 0x7f1e90155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8ffff640 1 -- 172.21.15.55:0/292741531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e901e9d00 con 0x7f1e9006b500 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8f7fe640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e900d32a0 0x7f1e90155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e94d80640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9015f150 0x7f1e900b98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8ffff640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9006b500 0x7f1e900ca6e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c3c0 tx=0x7f1e8000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8cff9640 1 -- 172.21.15.55:0/292741531 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800052d0 con 0x7f1e9006b500 2024-05-20T22:09:06.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8cff9640 1 -- 172.21.15.55:0/292741531 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e80005490 con 0x7f1e9006b500 2024-05-20T22:09:06.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e967e3640 1 -- 172.21.15.55:0/292741531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e901f8b80 con 0x7f1e9006b500 2024-05-20T22:09:06.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.618+0000 7f1e8cff9640 1 -- 172.21.15.55:0/292741531 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80007260 con 0x7f1e9006b500 2024-05-20T22:09:06.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.619+0000 7f1e967e3640 1 -- 172.21.15.55:0/292741531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e901f9140 con 0x7f1e9006b500 2024-05-20T22:09:06.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.620+0000 7f1e967e3640 1 -- 172.21.15.55:0/292741531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e90069090 con 0x7f1e9006b500 2024-05-20T22:09:06.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.620+0000 7f1e8cff9640 1 -- 172.21.15.55:0/292741531 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e80007400 con 0x7f1e9006b500 2024-05-20T22:09:06.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.621+0000 7f1e8cff9640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7c06be60 0x7f1e7c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:06.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.621+0000 7f1e8f7fe640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7c06be60 0x7f1e7c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:06.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.621+0000 7f1e8cff9640 1 -- 172.21.15.55:0/292741531 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7f1e8008a400 con 0x7f1e9006b500 2024-05-20T22:09:06.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.621+0000 7f1e8f7fe640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7c06be60 0x7f1e7c06e370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1e84034000 tx=0x7f1e8402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:06.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.625+0000 7f1e8cff9640 1 -- 172.21.15.55:0/292741531 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1e80058c90 con 0x7f1e9006b500 2024-05-20T22:09:06.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:06.822+0000 7f1e967e3640 1 -- 172.21.15.55:0/292741531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1e900d2e50 con 0x7f1e9006b500 2024-05-20T22:09:07.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.482+0000 7f1e8cff9640 1 -- 172.21.15.55:0/292741531 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7f1e80058590 con 0x7f1e9006b500 2024-05-20T22:09:07.483 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data_ec' removed 2024-05-20T22:09:07.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.484+0000 7f1e967e3640 1 -- 172.21.15.55:0/292741531 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7c06be60 msgr2=0x7f1e7c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:07.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.484+0000 7f1e967e3640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7c06be60 0x7f1e7c06e370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1e84034000 tx=0x7f1e8402e000 comp rx=0 tx=0).stop 2024-05-20T22:09:07.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.484+0000 7f1e967e3640 1 -- 172.21.15.55:0/292741531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9006b500 msgr2=0x7f1e900ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:07.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.484+0000 7f1e967e3640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9006b500 0x7f1e900ca6e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c3c0 tx=0x7f1e8000c880 comp rx=0 tx=0).stop 2024-05-20T22:09:07.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.484+0000 7f1e967e3640 1 -- 172.21.15.55:0/292741531 shutdown_connections 2024-05-20T22:09:07.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.484+0000 7f1e967e3640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1e7c06be60 0x7f1e7c06e370 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:07.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.484+0000 7f1e967e3640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e9015f150 0x7f1e900b98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:07.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.484+0000 7f1e967e3640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e900d32a0 0x7f1e90155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:07.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.484+0000 7f1e967e3640 1 --2- 172.21.15.55:0/292741531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e9006b500 0x7f1e900ca6e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:07.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.484+0000 7f1e967e3640 1 -- 172.21.15.55:0/292741531 >> 172.21.15.55:0/292741531 conn(0x7f1e900b6de0 msgr2=0x7f1e900d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:07.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.485+0000 7f1e967e3640 1 -- 172.21.15.55:0/292741531 shutdown_connections 2024-05-20T22:09:07.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.485+0000 7f1e967e3640 1 -- 172.21.15.55:0/292741531 wait complete. 2024-05-20T22:09:07.502 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T22:09:07.502 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T22:09:07.502 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T22:09:07.502 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:09:07.502 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T22:09:07.502 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T22:09:07.503 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T22:09:07.503 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:09:07.503 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T22:09:07.503 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T22:09:07.503 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T22:09:07.503 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:09:07.503 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T22:09:07.504 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T22:09:07.504 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T22:09:07.504 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:09:07.514 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T22:09:07.514 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T22:09:07.515 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T22:09:07.515 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T22:09:07.516 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T22:09:07.516 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T22:09:07.518 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T22:09:07.519 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T22:09:07.558 INFO:tasks.ceph.mds.a:Started 2024-05-20T22:09:07.559 DEBUG:teuthology.parallel:result is None 2024-05-20T22:09:07.560 INFO:tasks.ceph.mds.c:Started 2024-05-20T22:09:07.560 DEBUG:teuthology.parallel:result is None 2024-05-20T22:09:07.561 INFO:tasks.ceph.mds.b:Started 2024-05-20T22:09:07.561 DEBUG:teuthology.parallel:result is None 2024-05-20T22:09:07.562 INFO:tasks.ceph.mds.d:Started 2024-05-20T22:09:07.563 DEBUG:teuthology.parallel:result is None 2024-05-20T22:09:07.563 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-20T22:09:07.654 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-20T22:09:07.655 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-20T22:09:07.655 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-05-20T22:09:07.656 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-05-20T22:09:07.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.689+0000 7f9246796640 1 Processor -- start 2024-05-20T22:09:07.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.689+0000 7f9246796640 1 -- start start 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f9246796640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92400cd520 0x7f924008c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f9246796640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f92400cf270 0x7f92400cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f923f7fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f92400d0860 0x7f92400b19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51028/0 (socket says 172.21.15.55:51028) 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f923f7fe640 1 -- 172.21.15.55:0/1273476105 learned_addr learned my addr 172.21.15.55:0/1273476105 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f9246796640 1 -- 172.21.15.55:0/1273476105 --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92400cfbc0 con 0x7f92400d0860 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f923ffff640 1 --2- 172.21.15.55:0/1273476105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92400cd520 0x7f924008c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f9244d33640 1 --2- 172.21.15.55:0/1273476105 >> v2:172.21.15.138:3300/0 conn(0x7f92400cf270 0x7f92400cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f9246796640 1 -- 172.21.15.55:0/1273476105 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f92400d1020 con 0x7f92400cf270 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f9246796640 1 -- 172.21.15.55:0/1273476105 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f92400d1300 con 0x7f92400cd520 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1779430712 0 0) 0x7f9234001110 con 0x7f92400d0860 2024-05-20T22:09:07.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.691+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9228003520 con 0x7f92400d0860 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3888246299 0 0) 0x7f9228003520 con 0x7f92400d0860 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92400cd520 msgr2=0x7f924008c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 --2- 172.21.15.55:0/1273476105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92400cd520 0x7f924008c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 >> v2:172.21.15.138:3300/0 conn(0x7f92400cf270 msgr2=0x7f92400cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 --2- 172.21.15.55:0/1273476105 >> v2:172.21.15.138:3300/0 conn(0x7f92400cf270 0x7f92400cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92400d15e0 con 0x7f92400d0860 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f9234001840 con 0x7f92400d0860 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 >> v1:172.21.15.55:6789/0 conn(0x7f92400d0860 legacy=0x7f92400b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 --2- 172.21.15.55:0/1273476105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92400cd520 0x7f92280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 --2- 172.21.15.55:0/1273476105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9228003f30 0x7f9228004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 --2- 172.21.15.55:0/1273476105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9228004d40 0x7f9228005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f92400d1020 con 0x7f9228004d40 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92400d1300 con 0x7f9228003f30 2024-05-20T22:09:07.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f922800bac0 con 0x7f92400cd520 2024-05-20T22:09:07.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f9244d33640 1 --2- 172.21.15.55:0/1273476105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9228004d40 0x7f9228005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:07.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f9244d33640 1 -- 172.21.15.55:0/1273476105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92400cd520 msgr2=0x7f92280039b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T22:09:07.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f9244d33640 1 --2- 172.21.15.55:0/1273476105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92400cd520 0x7f92280039b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:07.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f9244d33640 1 -- 172.21.15.55:0/1273476105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9228003f30 msgr2=0x7f9228004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T22:09:07.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f9244d33640 1 --2- 172.21.15.55:0/1273476105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9228003f30 0x7f9228004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:07.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.692+0000 7f9244d33640 1 -- 172.21.15.55:0/1273476105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9228003520 con 0x7f9228004d40 2024-05-20T22:09:07.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.693+0000 7f9244d33640 1 --2- 172.21.15.55:0/1273476105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9228004d40 0x7f9228005150 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9238009f90 tx=0x7f9228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ef73cfed7b3d26 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:07.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.693+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238009c10 con 0x7f9228004d40 2024-05-20T22:09:07.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.693+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9238036070 con 0x7f9228004d40 2024-05-20T22:09:07.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.693+0000 7f923effd640 1 -- 172.21.15.55:0/1273476105 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238031880 con 0x7f9228004d40 2024-05-20T22:09:07.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.694+0000 7f9246796640 1 -- 172.21.15.55:0/1273476105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9228004d40 msgr2=0x7f9228005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:07.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.694+0000 7f9246796640 1 --2- 172.21.15.55:0/1273476105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9228004d40 0x7f9228005150 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9238009f90 tx=0x7f9228005830 comp rx=0 tx=0).stop 2024-05-20T22:09:07.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.694+0000 7f9244d33640 1 -- 172.21.15.55:0/1273476105 reap_dead start 2024-05-20T22:09:07.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.694+0000 7f9246796640 1 -- 172.21.15.55:0/1273476105 shutdown_connections 2024-05-20T22:09:07.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.694+0000 7f9246796640 1 -- 172.21.15.55:0/1273476105 >> 172.21.15.55:0/1273476105 conn(0x7f92400d2860 msgr2=0x7f92400d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:07.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.694+0000 7f9246796640 1 -- 172.21.15.55:0/1273476105 shutdown_connections 2024-05-20T22:09:07.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.694+0000 7f9246796640 1 -- 172.21.15.55:0/1273476105 wait complete. 2024-05-20T22:09:07.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.696+0000 7f9246796640 1 Processor -- start 2024-05-20T22:09:07.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.696+0000 7f9246796640 1 -- start start 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.697+0000 7f9246796640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92400af430 0x7f924015de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.697+0000 7f9246796640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92401570b0 0x7f924009c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.697+0000 7f9246796640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92401574c0 0x7f92400b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.697+0000 7f9246796640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f92400d0c70 con 0x7f92401574c0 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.697+0000 7f9246796640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92400b2cb0 con 0x7f92400af430 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.697+0000 7f9246796640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f92400b1660 con 0x7f92401570b0 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9244d33640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92401574c0 0x7f92400b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9244d33640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92401574c0 0x7f92400b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48060/0 (socket says 172.21.15.55:48060) 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9244d33640 1 -- 172.21.15.55:0/150641140 learned_addr learned my addr 172.21.15.55:0/150641140 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f923f7fe640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92401570b0 0x7f924009c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f923ffff640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92400af430 0x7f924015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9244d33640 1 -- 172.21.15.55:0/150641140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92401570b0 msgr2=0x7f924009c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9244d33640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92401570b0 0x7f924009c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9244d33640 1 -- 172.21.15.55:0/150641140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92400af430 msgr2=0x7f924015de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9244d33640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92400af430 0x7f924015de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9244d33640 1 -- 172.21.15.55:0/150641140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92400b10a0 con 0x7f92401574c0 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f923f7fe640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92401570b0 0x7f924009c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9244d33640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92401574c0 0x7f92400b30d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9238009f90 tx=0x7f9238009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f923cff9640 1 -- 172.21.15.55:0/150641140 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238009c10 con 0x7f92401574c0 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f923cff9640 1 -- 172.21.15.55:0/150641140 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9238031dd0 con 0x7f92401574c0 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9246796640 1 -- 172.21.15.55:0/150641140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92400b1380 con 0x7f92401574c0 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f923cff9640 1 -- 172.21.15.55:0/150641140 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238003bd0 con 0x7f92401574c0 2024-05-20T22:09:07.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.698+0000 7f9246796640 1 -- 172.21.15.55:0/150641140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92400afd90 con 0x7f92401574c0 2024-05-20T22:09:07.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.699+0000 7f9246796640 1 -- 172.21.15.55:0/150641140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92400b4000 con 0x7f92401574c0 2024-05-20T22:09:07.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.700+0000 7f923cff9640 1 -- 172.21.15.55:0/150641140 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9238005d10 con 0x7f92401574c0 2024-05-20T22:09:07.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.701+0000 7f923cff9640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f922c06be30 0x7f922c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:07.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.701+0000 7f923cff9640 1 -- 172.21.15.55:0/150641140 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7f92380ac010 con 0x7f92401574c0 2024-05-20T22:09:07.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.701+0000 7f923ffff640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f922c06be30 0x7f922c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:07.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.701+0000 7f923ffff640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f922c06be30 0x7f922c06e340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f92400b3d40 tx=0x7f9230008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:07.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.706+0000 7f923cff9640 1 -- 172.21.15.55:0/150641140 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f92400b4000 con 0x7f92401574c0 2024-05-20T22:09:07.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:07.904+0000 7f9246796640 1 -- 172.21.15.55:0/150641140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9240069090 con 0x7f92401574c0 2024-05-20T22:09:08.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.519+0000 7f923cff9640 1 -- 172.21.15.55:0/150641140 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f923804a060 con 0x7f92401574c0 2024-05-20T22:09:08.521 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2024-05-20T22:09:08.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 -- 172.21.15.55:0/150641140 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f922c06be30 msgr2=0x7f922c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:08.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f922c06be30 0x7f922c06e340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f92400b3d40 tx=0x7f9230008040 comp rx=0 tx=0).stop 2024-05-20T22:09:08.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 -- 172.21.15.55:0/150641140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92401574c0 msgr2=0x7f92400b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:08.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92401574c0 0x7f92400b30d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9238009f90 tx=0x7f9238009750 comp rx=0 tx=0).stop 2024-05-20T22:09:08.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 -- 172.21.15.55:0/150641140 shutdown_connections 2024-05-20T22:09:08.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f922c06be30 0x7f922c06e340 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92401574c0 0x7f92400b30d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92401570b0 0x7f924009c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 --2- 172.21.15.55:0/150641140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92400af430 0x7f924015de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 -- 172.21.15.55:0/150641140 >> 172.21.15.55:0/150641140 conn(0x7f92400d2860 msgr2=0x7f92400b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:08.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 -- 172.21.15.55:0/150641140 shutdown_connections 2024-05-20T22:09:08.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.522+0000 7f9246796640 1 -- 172.21.15.55:0/150641140 wait complete. 2024-05-20T22:09:08.538 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-20T22:09:08.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.657+0000 7f0b723e0640 1 Processor -- start 2024-05-20T22:09:08.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.657+0000 7f0b723e0640 1 -- start start 2024-05-20T22:09:08.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.658+0000 7f0b723e0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b6c0b3b30 0x7f0b6c0b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:08.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.658+0000 7f0b723e0640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0b6c06c790 0x7f0b6c0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.658+0000 7f0b723e0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b6c0d3dd0 con 0x7f0b6c15f100 2024-05-20T22:09:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.658+0000 7f0b723e0640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f0b6c0d40b0 con 0x7f0b6c06c790 2024-05-20T22:09:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.658+0000 7f0b723e0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c0d4390 con 0x7f0b6c0b3b30 2024-05-20T22:09:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.658+0000 7f0b7097d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0b6c06c790 0x7f0b6c0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.658+0000 7f0b7097d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0b6c06c790 0x7f0b6c0cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48066/0 (socket says 172.21.15.55:48066) 2024-05-20T22:09:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.658+0000 7f0b7097d640 1 -- 172.21.15.55:0/4161810056 learned_addr learned my addr 172.21.15.55:0/4161810056 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:08.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.658+0000 7f0b6bfff640 1 --2- 172.21.15.55:0/4161810056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b6c0b3b30 0x7f0b6c0b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:08.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.659+0000 7f0b7097d640 1 -- 172.21.15.55:0/4161810056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b6c0b3b30 msgr2=0x7f0b6c0b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:08.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.659+0000 7f0b7097d640 1 --2- 172.21.15.55:0/4161810056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b6c0b3b30 0x7f0b6c0b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.659+0000 7f0b7097d640 1 -- 172.21.15.55:0/4161810056 >> v1:172.21.15.138:6789/0 conn(0x7f0b6c15f100 legacy=0x7f0b6c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.659+0000 7f0b7097d640 1 -- 172.21.15.55:0/4161810056 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c06b5a0 con 0x7f0b6c06c790 2024-05-20T22:09:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.659+0000 7f0b7097d640 1 --2- 172.21.15.55:0/4161810056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c06c790 0x7f0b6c0cf740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009e50 tx=0x7f0b6402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28a44f8dd4725e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.659+0000 7f0b6affd640 1 -- 172.21.15.55:0/4161810056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64036070 con 0x7f0b6c06c790 2024-05-20T22:09:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.660+0000 7f0b6affd640 1 -- 172.21.15.55:0/4161810056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b64031600 con 0x7f0b6c06c790 2024-05-20T22:09:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.660+0000 7f0b6affd640 1 -- 172.21.15.55:0/4161810056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b640319d0 con 0x7f0b6c06c790 2024-05-20T22:09:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.660+0000 7f0b723e0640 1 -- 172.21.15.55:0/4161810056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c06c790 msgr2=0x7f0b6c0cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.660+0000 7f0b723e0640 1 --2- 172.21.15.55:0/4161810056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c06c790 0x7f0b6c0cf740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009e50 tx=0x7f0b6402a710 comp rx=0 tx=0).stop 2024-05-20T22:09:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.660+0000 7f0b723e0640 1 -- 172.21.15.55:0/4161810056 shutdown_connections 2024-05-20T22:09:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.660+0000 7f0b723e0640 1 --2- 172.21.15.55:0/4161810056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c06c790 0x7f0b6c0cf740 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.660+0000 7f0b723e0640 1 --2- 172.21.15.55:0/4161810056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b6c0b3b30 0x7f0b6c0b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.660+0000 7f0b723e0640 1 -- 172.21.15.55:0/4161810056 >> 172.21.15.55:0/4161810056 conn(0x7f0b6c0d51d0 msgr2=0x7f0b6c0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:08.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.660+0000 7f0b723e0640 1 -- 172.21.15.55:0/4161810056 shutdown_connections 2024-05-20T22:09:08.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.660+0000 7f0b723e0640 1 -- 172.21.15.55:0/4161810056 wait complete. 2024-05-20T22:09:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.662+0000 7f0b723e0640 1 Processor -- start 2024-05-20T22:09:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.662+0000 7f0b723e0640 1 -- start start 2024-05-20T22:09:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.663+0000 7f0b723e0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b6c06c790 0x7f0b6c146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.663+0000 7f0b723e0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b6c0b3b30 0x7f0b6c1f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:08.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b723e0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c15f100 0x7f0b6c1f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:08.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b723e0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c06cba0 con 0x7f0b6c15f100 2024-05-20T22:09:08.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b723e0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c06bb60 con 0x7f0b6c0b3b30 2024-05-20T22:09:08.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b723e0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c06cd20 con 0x7f0b6c06c790 2024-05-20T22:09:08.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b7097d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c15f100 0x7f0b6c1f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:08.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b7097d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c15f100 0x7f0b6c1f92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49174/0 (socket says 172.21.15.55:49174) 2024-05-20T22:09:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b7097d640 1 -- 172.21.15.55:0/818402588 learned_addr learned my addr 172.21.15.55:0/818402588 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b6bfff640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b6c06c790 0x7f0b6c146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b6b7fe640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b6c0b3b30 0x7f0b6c1f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b7097d640 1 -- 172.21.15.55:0/818402588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b6c06c790 msgr2=0x7f0b6c146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b7097d640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b6c06c790 0x7f0b6c146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b7097d640 1 -- 172.21.15.55:0/818402588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b6c0b3b30 msgr2=0x7f0b6c1f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:08.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b7097d640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b6c0b3b30 0x7f0b6c1f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b7097d640 1 -- 172.21.15.55:0/818402588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c156680 con 0x7f0b6c15f100 2024-05-20T22:09:08.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b7097d640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c15f100 0x7f0b6c1f92f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0b64034040 tx=0x7f0b64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:08.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b68ff9640 1 -- 172.21.15.55:0/818402588 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64036050 con 0x7f0b6c15f100 2024-05-20T22:09:08.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b68ff9640 1 -- 172.21.15.55:0/818402588 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b64042070 con 0x7f0b6c15f100 2024-05-20T22:09:08.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b723e0640 1 -- 172.21.15.55:0/818402588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c156960 con 0x7f0b6c15f100 2024-05-20T22:09:08.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b68ff9640 1 -- 172.21.15.55:0/818402588 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64009510 con 0x7f0b6c15f100 2024-05-20T22:09:08.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.664+0000 7f0b723e0640 1 -- 172.21.15.55:0/818402588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c156f20 con 0x7f0b6c15f100 2024-05-20T22:09:08.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.666+0000 7f0b723e0640 1 -- 172.21.15.55:0/818402588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6c069090 con 0x7f0b6c15f100 2024-05-20T22:09:08.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.666+0000 7f0b68ff9640 1 -- 172.21.15.55:0/818402588 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b64007bf0 con 0x7f0b6c15f100 2024-05-20T22:09:08.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.666+0000 7f0b68ff9640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b5806be30 0x7f0b5806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:08.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.667+0000 7f0b68ff9640 1 -- 172.21.15.55:0/818402588 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f0b640abaa0 con 0x7f0b6c15f100 2024-05-20T22:09:08.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.667+0000 7f0b6bfff640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b5806be30 0x7f0b5806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:08.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.667+0000 7f0b6bfff640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b5806be30 0x7f0b5806e340 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c06d0c0 tx=0x7f0b5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:08.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.671+0000 7f0b68ff9640 1 -- 172.21.15.55:0/818402588 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0b64045360 con 0x7f0b6c15f100 2024-05-20T22:09:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.923+0000 7f0b723e0640 1 -- 172.21.15.55:0/818402588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0b6c0d2ee0 con 0x7f0b6c15f100 2024-05-20T22:09:08.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.923+0000 7f0b68ff9640 1 -- 172.21.15.55:0/818402588 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v5) ==== 80+0+6645 (secure 0 0 0) 0x7f0b6407a6a0 con 0x7f0b6c15f100 2024-05-20T22:09:08.924 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:08.924 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:09:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2024-05-20T22:09:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2024-05-20T22:09:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmnF7NJRAAIHaudq8A1NdLVw4llE1GKA==", 2024-05-20T22:09:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:09:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-20T22:09:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-20T22:09:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.925 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmNDXWMhAAQU8KHijh4cVysa71xLvMjg==", 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmKe+YLRAAagJbIVjyJ+R1oZEBhZyGiQ==", 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.927 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtm6mtqOhAA6mTeQoWYP+yUvHt/szLnlw==", 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2024-05-20T22:09:08.928 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB8yUtm5SdYLhAAleTxIMKS9IsQjv4FusrryA==", 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB/yUtm7zGqLhAAU1uR214d/bSdRlcl5frlYA==", 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:09:08.929 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCCyUtmwBfXLxAAwRbdPpoNxTIoCSDsNdZOKQ==", 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.930 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCFyUtmTcaaMBAAQCJ6epS4s2FPsSvRBFdcLg==", 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.931 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2024-05-20T22:09:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCLyUtmNsNGNRAAQA8pn/2GQ4FPNxu3zsS9JQ==", 2024-05-20T22:09:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:09:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:09:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2024-05-20T22:09:08.932 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCOyUtmdmFLNBAAUH+yBiv2QSbfBqob0d6X8Q==", 2024-05-20T22:09:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:09:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:09:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2024-05-20T22:09:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCRyUtmVZGkMhAAmI118TqDPXtUvsH8We4hyg==", 2024-05-20T22:09:08.933 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCUyUtmNmlRMRAAUL0mlbzjAXGdx7oOZeKQ/w==", 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:09:08.934 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB2yUtm592RAxAAiaDYBHgkpVfsQHm+jLOy3g==", 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-20T22:09:08.935 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB2yUtmNXZWCxAArgL/Kf6oP/vHiIhC/4f/kg==", 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.936 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB2yUtmDwN6BxAAuS4b7g8QxESuhrW9d39wMQ==", 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.937 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2024-05-20T22:09:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB2yUtmIsoxDxAAdHDRYPZ5HeYfqye4wIR2wQ==", 2024-05-20T22:09:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.938 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:09:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-20T22:09:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-20T22:09:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-20T22:09:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2024-05-20T22:09:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmTdmWBxAA2yyudY+ao2TWVP52VQjobg==", 2024-05-20T22:09:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.942 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-mds", 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCfyUtmQUcZNhAAxROvUz2CzX9JLjktr0diKQ==", 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-mds" 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.943 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-mgr", 2024-05-20T22:09:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCfyUtm8FIZNhAAjWsTx/+F1hN+qcVIbtkRjg==", 2024-05-20T22:09:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-20T22:09:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.944 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-osd", 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCfyUtmi1sZNhAATVw+bySfN6FA3oJRbKxfdg==", 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-osd" 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-rbd", 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCfyUtmI2QZNhAAtgtP9Ge6HVkDu84lNMCasA==", 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-20T22:09:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCfyUtm/mwZNhAAFoM52BhVM/HsaQwcOsoDLw==", 2024-05-20T22:09:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-20T22:09:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-rgw", 2024-05-20T22:09:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCfyUtm8nUZNhAAUpEyLexPPCTlusXgDFt1xA==", 2024-05-20T22:09:08.946 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmwQ/lIRAAuSUOiqQA/GU0trg5dUB8Sg==", 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:08.947 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2024-05-20T22:09:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmXhIUHRAAzFxrH0wPo95yWm4KoEFLvw==", 2024-05-20T22:09:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:09:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-20T22:09:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-20T22:09:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:09:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:08.948 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-05-20T22:09:08.948 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 -- 172.21.15.55:0/818402588 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b5806be30 msgr2=0x7f0b5806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b5806be30 0x7f0b5806e340 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c06d0c0 tx=0x7f0b5c008040 comp rx=0 tx=0).stop 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 -- 172.21.15.55:0/818402588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c15f100 msgr2=0x7f0b6c1f92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c15f100 0x7f0b6c1f92f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0b64034040 tx=0x7f0b64004a40 comp rx=0 tx=0).stop 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 -- 172.21.15.55:0/818402588 shutdown_connections 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b5806be30 0x7f0b5806e340 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b6c15f100 0x7f0b6c1f92f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b6c0b3b30 0x7f0b6c1f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 --2- 172.21.15.55:0/818402588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b6c06c790 0x7f0b6c146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 -- 172.21.15.55:0/818402588 >> 172.21.15.55:0/818402588 conn(0x7f0b6c0d51d0 msgr2=0x7f0b6c0b8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 -- 172.21.15.55:0/818402588 shutdown_connections 2024-05-20T22:09:08.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:08.925+0000 7f0b723e0640 1 -- 172.21.15.55:0/818402588 wait complete. 2024-05-20T22:09:08.951 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-20T22:09:09.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.066+0000 7fb6f6765640 1 Processor -- start 2024-05-20T22:09:09.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.066+0000 7fb6f6765640 1 -- start start 2024-05-20T22:09:09.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.067+0000 7fb6f6765640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb6f00d32a0 0x7fb6f015f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.067+0000 7fb6f6765640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d01b0 0x7fb6f006b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.067+0000 7fb6f6765640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f00d3780 con 0x7fb6f00cf690 2024-05-20T22:09:09.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.067+0000 7fb6f6765640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f00d3a60 con 0x7fb6f00d32a0 2024-05-20T22:09:09.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.067+0000 7fb6f6765640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f00d3d40 con 0x7fb6f00d01b0 2024-05-20T22:09:09.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.068+0000 7fb6f4d02640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d01b0 0x7fb6f006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.068+0000 7fb6effff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb6f00d32a0 0x7fb6f015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.068+0000 7fb6e7fff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb6f00cf690 0x7fb6f00cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:37516/0 (socket says 172.21.15.55:37516) 2024-05-20T22:09:09.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.068+0000 7fb6e7fff640 1 -- 172.21.15.55:0/1842423472 learned_addr learned my addr 172.21.15.55:0/1842423472 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:09.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.068+0000 7fb6effff640 1 -- 172.21.15.55:0/1842423472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d01b0 msgr2=0x7fb6f006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:09.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.068+0000 7fb6effff640 1 --2- 172.21.15.55:0/1842423472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d01b0 0x7fb6f006b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.068+0000 7fb6effff640 1 -- 172.21.15.55:0/1842423472 >> v1:172.21.15.138:6789/0 conn(0x7fb6f00cf690 legacy=0x7fb6f00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.068+0000 7fb6effff640 1 -- 172.21.15.55:0/1842423472 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f00d4020 con 0x7fb6f00d32a0 2024-05-20T22:09:09.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.068+0000 7fb6ef7fe640 1 -- 172.21.15.55:0/1842423472 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356200167 0 0) 0x7fb6f00d3780 con 0x7fb6f00cf690 2024-05-20T22:09:09.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.068+0000 7fb6f4d02640 1 --2- 172.21.15.55:0/1842423472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d01b0 0x7fb6f006b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:09:09.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.069+0000 7fb6effff640 1 --2- 172.21.15.55:0/1842423472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00d32a0 0x7fb6f015f150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00097a0 tx=0x7fb6e002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e7d9d5f180f212 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:09.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.069+0000 7fb6ef7fe640 1 -- 172.21.15.55:0/1842423472 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0037070 con 0x7fb6f00d32a0 2024-05-20T22:09:09.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.069+0000 7fb6ef7fe640 1 -- 172.21.15.55:0/1842423472 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e0032440 con 0x7fb6f00d32a0 2024-05-20T22:09:09.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.069+0000 7fb6ef7fe640 1 -- 172.21.15.55:0/1842423472 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0032810 con 0x7fb6f00d32a0 2024-05-20T22:09:09.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.070+0000 7fb6f6765640 1 -- 172.21.15.55:0/1842423472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00d32a0 msgr2=0x7fb6f015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.070+0000 7fb6f6765640 1 --2- 172.21.15.55:0/1842423472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00d32a0 0x7fb6f015f150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00097a0 tx=0x7fb6e002a180 comp rx=0 tx=0).stop 2024-05-20T22:09:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.070+0000 7fb6f6765640 1 -- 172.21.15.55:0/1842423472 shutdown_connections 2024-05-20T22:09:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.070+0000 7fb6f6765640 1 --2- 172.21.15.55:0/1842423472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d01b0 0x7fb6f006b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.070+0000 7fb6f6765640 1 --2- 172.21.15.55:0/1842423472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00d32a0 0x7fb6f015f150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.070+0000 7fb6f6765640 1 -- 172.21.15.55:0/1842423472 >> 172.21.15.55:0/1842423472 conn(0x7fb6f00b6de0 msgr2=0x7fb6f00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.070+0000 7fb6f6765640 1 -- 172.21.15.55:0/1842423472 shutdown_connections 2024-05-20T22:09:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.070+0000 7fb6f6765640 1 -- 172.21.15.55:0/1842423472 wait complete. 2024-05-20T22:09:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.072+0000 7fb6f6765640 1 Processor -- start 2024-05-20T22:09:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.072+0000 7fb6f6765640 1 -- start start 2024-05-20T22:09:09.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6f6765640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00cf690 0x7fb6f00ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6f6765640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6f00d01b0 0x7fb6f014cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6f6765640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d32a0 0x7fb6f00ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6f6765640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00d4820 con 0x7fb6f00d01b0 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6f6765640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00b4350 con 0x7fb6f00cf690 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6f6765640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f014dd20 con 0x7fb6f00d32a0 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6e7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6f00d01b0 0x7fb6f014cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6e7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6f00d01b0 0x7fb6f014cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49186/0 (socket says 172.21.15.55:49186) 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6e7fff640 1 -- 172.21.15.55:0/707098835 learned_addr learned my addr 172.21.15.55:0/707098835 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6effff640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00cf690 0x7fb6f00ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6f4d02640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d32a0 0x7fb6f00ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6e7fff640 1 -- 172.21.15.55:0/707098835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d32a0 msgr2=0x7fb6f00ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6e7fff640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d32a0 0x7fb6f00ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6e7fff640 1 -- 172.21.15.55:0/707098835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00cf690 msgr2=0x7fb6f00ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6e7fff640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00cf690 0x7fb6f00ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.073+0000 7fb6e7fff640 1 -- 172.21.15.55:0/707098835 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f01601f0 con 0x7fb6f00d01b0 2024-05-20T22:09:09.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.074+0000 7fb6e7fff640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6f00d01b0 0x7fb6f014cfa0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800c460 tx=0x7fb6d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:09.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.074+0000 7fb6ed7fa640 1 -- 172.21.15.55:0/707098835 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8018070 con 0x7fb6f00d01b0 2024-05-20T22:09:09.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.074+0000 7fb6ed7fa640 1 -- 172.21.15.55:0/707098835 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d8007500 con 0x7fb6f00d01b0 2024-05-20T22:09:09.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.074+0000 7fb6f6765640 1 -- 172.21.15.55:0/707098835 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f01604d0 con 0x7fb6f00d01b0 2024-05-20T22:09:09.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.074+0000 7fb6ed7fa640 1 -- 172.21.15.55:0/707098835 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8004a20 con 0x7fb6f00d01b0 2024-05-20T22:09:09.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.074+0000 7fb6f6765640 1 -- 172.21.15.55:0/707098835 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f0160a90 con 0x7fb6f00d01b0 2024-05-20T22:09:09.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.075+0000 7fb6f6765640 1 -- 172.21.15.55:0/707098835 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f015ff60 con 0x7fb6f00d01b0 2024-05-20T22:09:09.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.076+0000 7fb6ed7fa640 1 -- 172.21.15.55:0/707098835 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6d8004bc0 con 0x7fb6f00d01b0 2024-05-20T22:09:09.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.076+0000 7fb6ed7fa640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6d006be30 0x7fb6d006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.076+0000 7fb6ed7fa640 1 -- 172.21.15.55:0/707098835 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fb6d808a0f0 con 0x7fb6f00d01b0 2024-05-20T22:09:09.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.076+0000 7fb6effff640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6d006be30 0x7fb6d006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.077+0000 7fb6effff640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6d006be30 0x7fb6d006e340 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0034000 tx=0x7fb6e002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:09.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.081+0000 7fb6ed7fa640 1 -- 172.21.15.55:0/707098835 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb6d8058b50 con 0x7fb6f00d01b0 2024-05-20T22:09:09.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.334+0000 7fb6f6765640 1 -- 172.21.15.55:0/707098835 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fb6f0069090 con 0x7fb6f00d01b0 2024-05-20T22:09:09.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.337+0000 7fb6ed7fa640 1 -- 172.21.15.55:0/707098835 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fb6d8058450 con 0x7fb6f00d01b0 2024-05-20T22:09:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 -- 172.21.15.55:0/707098835 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6d006be30 msgr2=0x7fb6d006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6d006be30 0x7fb6d006e340 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0034000 tx=0x7fb6e002e000 comp rx=0 tx=0).stop 2024-05-20T22:09:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 -- 172.21.15.55:0/707098835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6f00d01b0 msgr2=0x7fb6f014cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6f00d01b0 0x7fb6f014cfa0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800c460 tx=0x7fb6d800c920 comp rx=0 tx=0).stop 2024-05-20T22:09:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 -- 172.21.15.55:0/707098835 shutdown_connections 2024-05-20T22:09:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb6d006be30 0x7fb6d006e340 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6f00d32a0 0x7fb6f00ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6f00d01b0 0x7fb6f014cfa0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 --2- 172.21.15.55:0/707098835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6f00cf690 0x7fb6f00ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 -- 172.21.15.55:0/707098835 >> 172.21.15.55:0/707098835 conn(0x7fb6f00b6de0 msgr2=0x7fb6f00b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:09.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 -- 172.21.15.55:0/707098835 shutdown_connections 2024-05-20T22:09:09.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.339+0000 7fb6f6765640 1 -- 172.21.15.55:0/707098835 wait complete. 2024-05-20T22:09:09.351 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-20T22:09:09.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.467+0000 7f873f74c640 1 Processor -- start 2024-05-20T22:09:09.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.467+0000 7f873f74c640 1 -- start start 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.468+0000 7f873f74c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f873808c960 0x7f87380b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.468+0000 7f873f74c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f87380cf270 0x7f87380cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.468+0000 7f873f74c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873808cd70 con 0x7f87380cd520 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.468+0000 7f873f74c640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f87380cfbc0 con 0x7f873808c960 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.468+0000 7f873f74c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f87380b1f30 con 0x7f87380cf270 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.468+0000 7f873cce7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f873808c960 0x7f87380b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.468+0000 7f873cce7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f873808c960 0x7f87380b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49196/0 (socket says 172.21.15.55:49196) 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.468+0000 7f873cce7640 1 -- 172.21.15.55:0/3589351801 learned_addr learned my addr 172.21.15.55:0/3589351801 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.469+0000 7f873cce7640 1 -- 172.21.15.55:0/3589351801 >> v2:172.21.15.138:3300/0 conn(0x7f87380cf270 msgr2=0x7f87380cf680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.469+0000 7f873cce7640 1 --2- 172.21.15.55:0/3589351801 >> v2:172.21.15.138:3300/0 conn(0x7f87380cf270 0x7f87380cf680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.469+0000 7f873cce7640 1 -- 172.21.15.55:0/3589351801 >> v1:172.21.15.138:6789/0 conn(0x7f87380cd520 legacy=0x7f87380d0860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T22:09:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.469+0000 7f873cce7640 1 -- 172.21.15.55:0/3589351801 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87380d1020 con 0x7f873808c960 2024-05-20T22:09:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.469+0000 7f873cce7640 1 --2- 172.21.15.55:0/3589351801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 0x7f87380b19f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f87380d1b60 tx=0x7f872802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1e3cdc65e3bfdd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.470+0000 7f872ffff640 1 -- 172.21.15.55:0/3589351801 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728037070 con 0x7f873808c960 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.470+0000 7f872ffff640 1 -- 172.21.15.55:0/3589351801 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8728032440 con 0x7f873808c960 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.470+0000 7f872ffff640 1 -- 172.21.15.55:0/3589351801 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728032810 con 0x7f873808c960 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.470+0000 7f873f74c640 1 -- 172.21.15.55:0/3589351801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 msgr2=0x7f87380b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.470+0000 7f873f74c640 1 --2- 172.21.15.55:0/3589351801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 0x7f87380b19f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f87380d1b60 tx=0x7f872802a1c0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.470+0000 7f873f74c640 1 -- 172.21.15.55:0/3589351801 shutdown_connections 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.470+0000 7f873f74c640 1 --2- 172.21.15.55:0/3589351801 >> v2:172.21.15.138:3300/0 conn(0x7f87380cf270 0x7f87380cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.470+0000 7f873f74c640 1 --2- 172.21.15.55:0/3589351801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 0x7f87380b19f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.470+0000 7f873f74c640 1 -- 172.21.15.55:0/3589351801 >> 172.21.15.55:0/3589351801 conn(0x7f87380d2860 msgr2=0x7f87380d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.470+0000 7f873f74c640 1 -- 172.21.15.55:0/3589351801 shutdown_connections 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.471+0000 7f873f74c640 1 -- 172.21.15.55:0/3589351801 wait complete. 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.472+0000 7f873f74c640 1 Processor -- start 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.472+0000 7f873f74c640 1 -- start start 2024-05-20T22:09:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873f74c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 0x7f87381573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873f74c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87380cd520 0x7f87380cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873f74c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87380cf270 0x7f873815b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873f74c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f87380d18c0 con 0x7f873808c960 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873f74c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87380d15e0 con 0x7f87380cf270 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873f74c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f87380b2cb0 con 0x7f87380cd520 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873d4e8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 0x7f87381573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873d4e8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 0x7f87381573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49206/0 (socket says 172.21.15.55:49206) 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873d4e8640 1 -- 172.21.15.55:0/2144614454 learned_addr learned my addr 172.21.15.55:0/2144614454 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873dce9640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87380cf270 0x7f873815b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873cce7640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87380cd520 0x7f87380cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873d4e8640 1 -- 172.21.15.55:0/2144614454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87380cd520 msgr2=0x7f87380cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873d4e8640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87380cd520 0x7f87380cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873d4e8640 1 -- 172.21.15.55:0/2144614454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87380cf270 msgr2=0x7f873815b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873d4e8640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87380cf270 0x7f873815b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873d4e8640 1 -- 172.21.15.55:0/2144614454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87381f6e40 con 0x7f873808c960 2024-05-20T22:09:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873d4e8640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 0x7f87381573e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f873000b410 tx=0x7f873000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f872dffb640 1 -- 172.21.15.55:0/2144614454 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730015020 con 0x7f873808c960 2024-05-20T22:09:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f872dffb640 1 -- 172.21.15.55:0/2144614454 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8730011830 con 0x7f873808c960 2024-05-20T22:09:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.474+0000 7f873f74c640 1 -- 172.21.15.55:0/2144614454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87381f7120 con 0x7f873808c960 2024-05-20T22:09:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.475+0000 7f872dffb640 1 -- 172.21.15.55:0/2144614454 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730011c20 con 0x7f873808c960 2024-05-20T22:09:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.475+0000 7f873f74c640 1 -- 172.21.15.55:0/2144614454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87381f76e0 con 0x7f873808c960 2024-05-20T22:09:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.476+0000 7f873f74c640 1 -- 172.21.15.55:0/2144614454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8738069090 con 0x7f873808c960 2024-05-20T22:09:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.476+0000 7f872dffb640 1 -- 172.21.15.55:0/2144614454 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8730018020 con 0x7f873808c960 2024-05-20T22:09:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.477+0000 7f872dffb640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f871c06be60 0x7f871c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.477+0000 7f872dffb640 1 -- 172.21.15.55:0/2144614454 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f873008ad40 con 0x7f873808c960 2024-05-20T22:09:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.477+0000 7f873cce7640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f871c06be60 0x7f871c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.477+0000 7f873cce7640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f871c06be60 0x7f871c06e370 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8728034000 tx=0x7f872802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.481+0000 7f872dffb640 1 -- 172.21.15.55:0/2144614454 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f873001f020 con 0x7f873808c960 2024-05-20T22:09:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.737+0000 7f873f74c640 1 -- 172.21.15.55:0/2144614454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f87380b27c0 con 0x7f873808c960 2024-05-20T22:09:09.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.740+0000 7f872dffb640 1 -- 172.21.15.55:0/2144614454 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f87300172c0 con 0x7f873808c960 2024-05-20T22:09:09.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 -- 172.21.15.55:0/2144614454 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f871c06be60 msgr2=0x7f871c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f871c06be60 0x7f871c06e370 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8728034000 tx=0x7f872802e000 comp rx=0 tx=0).stop 2024-05-20T22:09:09.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 -- 172.21.15.55:0/2144614454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 msgr2=0x7f87381573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 0x7f87381573e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f873000b410 tx=0x7f873000b8d0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 -- 172.21.15.55:0/2144614454 shutdown_connections 2024-05-20T22:09:09.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f871c06be60 0x7f871c06e370 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87380cf270 0x7f873815b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87380cd520 0x7f87380cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 --2- 172.21.15.55:0/2144614454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f873808c960 0x7f87381573e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 -- 172.21.15.55:0/2144614454 >> 172.21.15.55:0/2144614454 conn(0x7f87380d2860 msgr2=0x7f87380b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 -- 172.21.15.55:0/2144614454 shutdown_connections 2024-05-20T22:09:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.742+0000 7f873f74c640 1 -- 172.21.15.55:0/2144614454 wait complete. 2024-05-20T22:09:09.754 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-20T22:09:09.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.873+0000 7fd0da39d640 1 Processor -- start 2024-05-20T22:09:09.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.873+0000 7fd0da39d640 1 -- start start 2024-05-20T22:09:09.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.874+0000 7fd0da39d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0d40cfad0 0x7fd0d40b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.874+0000 7fd0da39d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0d406d0b0 0x7fd0d40d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.874+0000 7fd0da39d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d40b37b0 con 0x7fd0d40b2470 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.874+0000 7fd0da39d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d40b2880 con 0x7fd0d40cfad0 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.874+0000 7fd0da39d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d406d8b0 con 0x7fd0d406d0b0 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.874+0000 7fd0cbfff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0d40cfad0 0x7fd0d40b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.874+0000 7fd0cbfff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0d40cfad0 0x7fd0d40b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49208/0 (socket says 172.21.15.55:49208) 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.874+0000 7fd0cbfff640 1 -- 172.21.15.55:0/914649416 learned_addr learned my addr 172.21.15.55:0/914649416 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.875+0000 7fd0d893a640 1 --2- 172.21.15.55:0/914649416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0d406d0b0 0x7fd0d40d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.875+0000 7fd0cbfff640 1 -- 172.21.15.55:0/914649416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0d406d0b0 msgr2=0x7fd0d40d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.875+0000 7fd0cbfff640 1 --2- 172.21.15.55:0/914649416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0d406d0b0 0x7fd0d40d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.875+0000 7fd0cbfff640 1 -- 172.21.15.55:0/914649416 >> v1:172.21.15.138:6789/0 conn(0x7fd0d40b2470 legacy=0x7fd0d40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.875+0000 7fd0cbfff640 1 -- 172.21.15.55:0/914649416 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d406db90 con 0x7fd0d40cfad0 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.876+0000 7fd0cbfff640 1 --2- 172.21.15.55:0/914649416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40cfad0 0x7fd0d40b1eb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009870 tx=0x7fd0bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef7154823520af0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.876+0000 7fd0d37fe640 1 -- 172.21.15.55:0/914649416 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc037070 con 0x7fd0d40cfad0 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.876+0000 7fd0d37fe640 1 -- 172.21.15.55:0/914649416 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0bc032440 con 0x7fd0d40cfad0 2024-05-20T22:09:09.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.876+0000 7fd0d37fe640 1 -- 172.21.15.55:0/914649416 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc032810 con 0x7fd0d40cfad0 2024-05-20T22:09:09.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.876+0000 7fd0da39d640 1 -- 172.21.15.55:0/914649416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40cfad0 msgr2=0x7fd0d40b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.876+0000 7fd0da39d640 1 --2- 172.21.15.55:0/914649416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40cfad0 0x7fd0d40b1eb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009870 tx=0x7fd0bc02a270 comp rx=0 tx=0).stop 2024-05-20T22:09:09.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.877+0000 7fd0da39d640 1 -- 172.21.15.55:0/914649416 shutdown_connections 2024-05-20T22:09:09.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.877+0000 7fd0da39d640 1 --2- 172.21.15.55:0/914649416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0d406d0b0 0x7fd0d40d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.877+0000 7fd0da39d640 1 --2- 172.21.15.55:0/914649416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40cfad0 0x7fd0d40b1eb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.877+0000 7fd0da39d640 1 -- 172.21.15.55:0/914649416 >> 172.21.15.55:0/914649416 conn(0x7fd0d409be10 msgr2=0x7fd0d40d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:09.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.877+0000 7fd0da39d640 1 -- 172.21.15.55:0/914649416 shutdown_connections 2024-05-20T22:09:09.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.877+0000 7fd0da39d640 1 -- 172.21.15.55:0/914649416 wait complete. 2024-05-20T22:09:09.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.878+0000 7fd0da39d640 1 Processor -- start 2024-05-20T22:09:09.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.878+0000 7fd0da39d640 1 -- start start 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0da39d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d406d0b0 0x7fd0d415d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0da39d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40b2470 0x7fd0d41f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0da39d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0d409fc10 0x7fd0d41f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0da39d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d406d4c0 con 0x7fd0d40b2470 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0da39d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d40b3a30 con 0x7fd0d406d0b0 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0da39d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d40b3f50 con 0x7fd0d409fc10 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40b2470 0x7fd0d41f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40b2470 0x7fd0d41f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49220/0 (socket says 172.21.15.55:49220) 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0cbfff640 1 -- 172.21.15.55:0/249378175 learned_addr learned my addr 172.21.15.55:0/249378175 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0d3fff640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d406d0b0 0x7fd0d415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0d893a640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0d409fc10 0x7fd0d41f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0cbfff640 1 -- 172.21.15.55:0/249378175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0d409fc10 msgr2=0x7fd0d41f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0cbfff640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0d409fc10 0x7fd0d41f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0cbfff640 1 -- 172.21.15.55:0/249378175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d406d0b0 msgr2=0x7fd0d415d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0cbfff640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d406d0b0 0x7fd0d415d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0cbfff640 1 -- 172.21.15.55:0/249378175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d41f61f0 con 0x7fd0d40b2470 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.880+0000 7fd0cbfff640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40b2470 0x7fd0d41f5cb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc000c00 tx=0x7fd0bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.881+0000 7fd0d17fa640 1 -- 172.21.15.55:0/249378175 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc041020 con 0x7fd0d40b2470 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.881+0000 7fd0d17fa640 1 -- 172.21.15.55:0/249378175 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0bc0306c0 con 0x7fd0d40b2470 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.881+0000 7fd0da39d640 1 -- 172.21.15.55:0/249378175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d409e640 con 0x7fd0d40b2470 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.881+0000 7fd0d17fa640 1 -- 172.21.15.55:0/249378175 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc037040 con 0x7fd0d40b2470 2024-05-20T22:09:09.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.881+0000 7fd0da39d640 1 -- 172.21.15.55:0/249378175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d409ee50 con 0x7fd0d40b2470 2024-05-20T22:09:09.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.882+0000 7fd0da39d640 1 -- 172.21.15.55:0/249378175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d4069090 con 0x7fd0d40b2470 2024-05-20T22:09:09.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.882+0000 7fd0d17fa640 1 -- 172.21.15.55:0/249378175 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0bc044020 con 0x7fd0d40b2470 2024-05-20T22:09:09.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.883+0000 7fd0d17fa640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0b406be60 0x7fd0b406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:09.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.883+0000 7fd0d17fa640 1 -- 172.21.15.55:0/249378175 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fd0bc0ab4b0 con 0x7fd0d40b2470 2024-05-20T22:09:09.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.883+0000 7fd0d3fff640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0b406be60 0x7fd0b406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:09.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.884+0000 7fd0d3fff640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0b406be60 0x7fd0b406e370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4000f60 tx=0x7fd0c4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:09.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:09.888+0000 7fd0d17fa640 1 -- 172.21.15.55:0/249378175 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd0bc04b030 con 0x7fd0d40b2470 2024-05-20T22:09:10.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.150+0000 7fd0da39d640 1 -- 172.21.15.55:0/249378175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fd0d40d35a0 con 0x7fd0d40b2470 2024-05-20T22:09:10.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.158+0000 7fd0d17fa640 1 -- 172.21.15.55:0/249378175 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fd0bc079f90 con 0x7fd0d40b2470 2024-05-20T22:09:10.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 -- 172.21.15.55:0/249378175 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0b406be60 msgr2=0x7fd0b406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0b406be60 0x7fd0b406e370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4000f60 tx=0x7fd0c4009040 comp rx=0 tx=0).stop 2024-05-20T22:09:10.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 -- 172.21.15.55:0/249378175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40b2470 msgr2=0x7fd0d41f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40b2470 0x7fd0d41f5cb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc000c00 tx=0x7fd0bc0046c0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 -- 172.21.15.55:0/249378175 shutdown_connections 2024-05-20T22:09:10.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd0b406be60 0x7fd0b406e370 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0d409fc10 0x7fd0d41f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0d40b2470 0x7fd0d41f5cb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 --2- 172.21.15.55:0/249378175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0d406d0b0 0x7fd0d415d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 -- 172.21.15.55:0/249378175 >> 172.21.15.55:0/249378175 conn(0x7fd0d409be10 msgr2=0x7fd0d4071380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:10.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 -- 172.21.15.55:0/249378175 shutdown_connections 2024-05-20T22:09:10.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.160+0000 7fd0da39d640 1 -- 172.21.15.55:0/249378175 wait complete. 2024-05-20T22:09:10.172 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-20T22:09:10.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.287+0000 7f159fd37640 1 Processor -- start 2024-05-20T22:09:10.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.287+0000 7f159fd37640 1 -- start start 2024-05-20T22:09:10.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159fd37640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f15980d32a0 0x7f159815f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159fd37640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15980d01b0 0x7f159806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159fd37640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15980d3780 con 0x7f15980cf690 2024-05-20T22:09:10.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159fd37640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f15980d3a60 con 0x7f15980d32a0 2024-05-20T22:09:10.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159fd37640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f15980d3d40 con 0x7f15980d01b0 2024-05-20T22:09:10.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159dad3640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f15980d32a0 0x7f159815f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159d2d2640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f15980cf690 0x7f15980cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54204/0 (socket says 172.21.15.55:54204) 2024-05-20T22:09:10.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159d2d2640 1 -- 172.21.15.55:0/3369865049 learned_addr learned my addr 172.21.15.55:0/3369865049 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:10.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159e2d4640 1 --2- 172.21.15.55:0/3369865049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15980d01b0 0x7f159806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159dad3640 1 -- 172.21.15.55:0/3369865049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15980d01b0 msgr2=0x7f159806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:10.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159dad3640 1 --2- 172.21.15.55:0/3369865049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15980d01b0 0x7f159806b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159dad3640 1 -- 172.21.15.55:0/3369865049 >> v1:172.21.15.55:6789/0 conn(0x7f15980cf690 legacy=0x7f15980cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159cad1640 1 -- 172.21.15.55:0/3369865049 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2438099346 0 0) 0x7f15980d3780 con 0x7f15980cf690 2024-05-20T22:09:10.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.289+0000 7f159dad3640 1 -- 172.21.15.55:0/3369865049 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15980d4020 con 0x7f15980d32a0 2024-05-20T22:09:10.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.290+0000 7f159dad3640 1 --2- 172.21.15.55:0/3369865049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 0x7f159815f150 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f15940097a0 tx=0x7f159402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c256468fd30f2bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:10.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.290+0000 7f159cad1640 1 -- 172.21.15.55:0/3369865049 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594037070 con 0x7f15980d32a0 2024-05-20T22:09:10.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.290+0000 7f159cad1640 1 -- 172.21.15.55:0/3369865049 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1594032440 con 0x7f15980d32a0 2024-05-20T22:09:10.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.290+0000 7f159cad1640 1 -- 172.21.15.55:0/3369865049 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594032810 con 0x7f15980d32a0 2024-05-20T22:09:10.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.291+0000 7f159fd37640 1 -- 172.21.15.55:0/3369865049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 msgr2=0x7f159815f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.291+0000 7f159fd37640 1 --2- 172.21.15.55:0/3369865049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 0x7f159815f150 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f15940097a0 tx=0x7f159402a180 comp rx=0 tx=0).stop 2024-05-20T22:09:10.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.291+0000 7f159fd37640 1 -- 172.21.15.55:0/3369865049 shutdown_connections 2024-05-20T22:09:10.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.291+0000 7f159fd37640 1 --2- 172.21.15.55:0/3369865049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15980d01b0 0x7f159806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.291+0000 7f159fd37640 1 --2- 172.21.15.55:0/3369865049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 0x7f159815f150 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.291+0000 7f159fd37640 1 -- 172.21.15.55:0/3369865049 >> 172.21.15.55:0/3369865049 conn(0x7f15980b6de0 msgr2=0x7f15980d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:10.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.291+0000 7f159fd37640 1 -- 172.21.15.55:0/3369865049 shutdown_connections 2024-05-20T22:09:10.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.291+0000 7f159fd37640 1 -- 172.21.15.55:0/3369865049 wait complete. 2024-05-20T22:09:10.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.293+0000 7f159fd37640 1 Processor -- start 2024-05-20T22:09:10.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.293+0000 7f159fd37640 1 -- start start 2024-05-20T22:09:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159fd37640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15980cf690 0x7f15980cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159fd37640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15980d01b0 0x7f1598150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159fd37640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 0x7f15980cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159fd37640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15980d4820 con 0x7f15980d32a0 2024-05-20T22:09:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159fd37640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15980b4350 con 0x7f15980d01b0 2024-05-20T22:09:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159fd37640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f15980c9e80 con 0x7f15980cf690 2024-05-20T22:09:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159e2d4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 0x7f15980cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159e2d4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 0x7f15980cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49246/0 (socket says 172.21.15.55:49246) 2024-05-20T22:09:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159e2d4640 1 -- 172.21.15.55:0/2273291527 learned_addr learned my addr 172.21.15.55:0/2273291527 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:10.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159dad3640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15980cf690 0x7f15980cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159d2d2640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15980d01b0 0x7f1598150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159e2d4640 1 -- 172.21.15.55:0/2273291527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15980cf690 msgr2=0x7f15980cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159e2d4640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15980cf690 0x7f15980cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159e2d4640 1 -- 172.21.15.55:0/2273291527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15980d01b0 msgr2=0x7f1598150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159e2d4640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15980d01b0 0x7f1598150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.294+0000 7f159e2d4640 1 -- 172.21.15.55:0/2273291527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15980cc1f0 con 0x7f15980d32a0 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.295+0000 7f159e2d4640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 0x7f15980cac50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1590007d40 tx=0x7f159000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.295+0000 7f158e7fc640 1 -- 172.21.15.55:0/2273291527 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590014420 con 0x7f15980d32a0 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.295+0000 7f158e7fc640 1 -- 172.21.15.55:0/2273291527 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1590019070 con 0x7f15980d32a0 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.295+0000 7f158e7fc640 1 -- 172.21.15.55:0/2273291527 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590014820 con 0x7f15980d32a0 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.295+0000 7f159fd37640 1 -- 172.21.15.55:0/2273291527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15980cc4d0 con 0x7f15980d32a0 2024-05-20T22:09:10.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.295+0000 7f159fd37640 1 -- 172.21.15.55:0/2273291527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15980cca90 con 0x7f15980d32a0 2024-05-20T22:09:10.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.296+0000 7f159fd37640 1 -- 172.21.15.55:0/2273291527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1598069090 con 0x7f15980d32a0 2024-05-20T22:09:10.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.296+0000 7f158e7fc640 1 -- 172.21.15.55:0/2273291527 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15900149c0 con 0x7f15980d32a0 2024-05-20T22:09:10.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.297+0000 7f158e7fc640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f157c06be30 0x7f157c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.297+0000 7f159dad3640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f157c06be30 0x7f157c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.297+0000 7f158e7fc640 1 -- 172.21.15.55:0/2273291527 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f159008b470 con 0x7f15980d32a0 2024-05-20T22:09:10.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.298+0000 7f159dad3640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f157c06be30 0x7f157c06e340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1594034000 tx=0x7f159402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:10.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.302+0000 7f158e7fc640 1 -- 172.21.15.55:0/2273291527 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1590059fd0 con 0x7f15980d32a0 2024-05-20T22:09:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.557+0000 7f159fd37640 1 -- 172.21.15.55:0/2273291527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f15980cbf80 con 0x7f15980d32a0 2024-05-20T22:09:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.560+0000 7f158e7fc640 1 -- 172.21.15.55:0/2273291527 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f15900598d0 con 0x7f15980d32a0 2024-05-20T22:09:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 -- 172.21.15.55:0/2273291527 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f157c06be30 msgr2=0x7f157c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f157c06be30 0x7f157c06e340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1594034000 tx=0x7f159402e000 comp rx=0 tx=0).stop 2024-05-20T22:09:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 -- 172.21.15.55:0/2273291527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 msgr2=0x7f15980cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 0x7f15980cac50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1590007d40 tx=0x7f159000e4b0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 -- 172.21.15.55:0/2273291527 shutdown_connections 2024-05-20T22:09:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f157c06be30 0x7f157c06e340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15980d32a0 0x7f15980cac50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15980d01b0 0x7f1598150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 --2- 172.21.15.55:0/2273291527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15980cf690 0x7f15980cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 -- 172.21.15.55:0/2273291527 >> 172.21.15.55:0/2273291527 conn(0x7f15980b6de0 msgr2=0x7f159815ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 -- 172.21.15.55:0/2273291527 shutdown_connections 2024-05-20T22:09:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.562+0000 7f159fd37640 1 -- 172.21.15.55:0/2273291527 wait complete. 2024-05-20T22:09:10.574 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-20T22:09:10.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.688+0000 7fdcf7f1a640 1 Processor -- start 2024-05-20T22:09:10.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.689+0000 7fdcf7f1a640 1 -- start start 2024-05-20T22:09:10.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.690+0000 7fdcf7f1a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdcf015f150 0x7fdcf00cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.690+0000 7fdcf7f1a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf00d3780 con 0x7fdcf006b500 2024-05-20T22:09:10.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.690+0000 7fdcf7f1a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf00d3a60 con 0x7fdcf00d32a0 2024-05-20T22:09:10.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.690+0000 7fdcf7f1a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf00d3d40 con 0x7fdcf015f150 2024-05-20T22:09:10.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.690+0000 7fdcf54b5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdcf015f150 0x7fdcf00cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.690+0000 7fdcf64b7640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdcf006b500 0x7fdcf006b910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54214/0 (socket says 172.21.15.55:54214) 2024-05-20T22:09:10.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.690+0000 7fdcf54b5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdcf015f150 0x7fdcf00cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49260/0 (socket says 172.21.15.55:49260) 2024-05-20T22:09:10.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.690+0000 7fdcf64b7640 1 -- 172.21.15.55:0/2556793154 learned_addr learned my addr 172.21.15.55:0/2556793154 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:10.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.691+0000 7fdcf54b5640 1 -- 172.21.15.55:0/2556793154 >> v1:172.21.15.138:6789/0 conn(0x7fdcf00d32a0 legacy=0x7fdcf00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:10.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.691+0000 7fdcf54b5640 1 -- 172.21.15.55:0/2556793154 >> v1:172.21.15.55:6789/0 conn(0x7fdcf006b500 legacy=0x7fdcf006b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.691+0000 7fdcf54b5640 1 -- 172.21.15.55:0/2556793154 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00d4020 con 0x7fdcf015f150 2024-05-20T22:09:10.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.691+0000 7fdcf4cb4640 1 -- 172.21.15.55:0/2556793154 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2443397609 0 0) 0x7fdcf00d3780 con 0x7fdcf006b500 2024-05-20T22:09:10.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.691+0000 7fdcf54b5640 1 --2- 172.21.15.55:0/2556793154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf015f150 0x7fdcf00cfda0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdce40097a0 tx=0x7fdce402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f4108589026364 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:10.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.692+0000 7fdcf4cb4640 1 -- 172.21.15.55:0/2556793154 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4037070 con 0x7fdcf015f150 2024-05-20T22:09:10.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.692+0000 7fdcf4cb4640 1 -- 172.21.15.55:0/2556793154 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce4032440 con 0x7fdcf015f150 2024-05-20T22:09:10.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.692+0000 7fdcf4cb4640 1 -- 172.21.15.55:0/2556793154 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4032810 con 0x7fdcf015f150 2024-05-20T22:09:10.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.692+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/2556793154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf015f150 msgr2=0x7fdcf00cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.692+0000 7fdcf7f1a640 1 --2- 172.21.15.55:0/2556793154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf015f150 0x7fdcf00cfda0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdce40097a0 tx=0x7fdce402a1e0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.692+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/2556793154 shutdown_connections 2024-05-20T22:09:10.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.692+0000 7fdcf7f1a640 1 --2- 172.21.15.55:0/2556793154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf015f150 0x7fdcf00cfda0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.692+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/2556793154 >> 172.21.15.55:0/2556793154 conn(0x7fdcf00b6de0 msgr2=0x7fdcf00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:10.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.692+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/2556793154 shutdown_connections 2024-05-20T22:09:10.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.692+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/2556793154 wait complete. 2024-05-20T22:09:10.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.694+0000 7fdcf7f1a640 1 Processor -- start 2024-05-20T22:09:10.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.694+0000 7fdcf7f1a640 1 -- start start 2024-05-20T22:09:10.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.695+0000 7fdcf7f1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf006b500 0x7fdcf0153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.695+0000 7fdcf7f1a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf00d32a0 0x7fdcf0161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf7f1a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf015f150 0x7fdcf0161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf7f1a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00d4820 con 0x7fdcf00d32a0 2024-05-20T22:09:10.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf7f1a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf015f560 con 0x7fdcf006b500 2024-05-20T22:09:10.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf7f1a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf00ccb00 con 0x7fdcf015f150 2024-05-20T22:09:10.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf54b5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf00d32a0 0x7fdcf0161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf54b5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf00d32a0 0x7fdcf0161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49270/0 (socket says 172.21.15.55:49270) 2024-05-20T22:09:10.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf54b5640 1 -- 172.21.15.55:0/1294002063 learned_addr learned my addr 172.21.15.55:0/1294002063 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf64b7640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf015f150 0x7fdcf0161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf5cb6640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf006b500 0x7fdcf0153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf54b5640 1 -- 172.21.15.55:0/1294002063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf015f150 msgr2=0x7fdcf0161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf54b5640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf015f150 0x7fdcf0161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf54b5640 1 -- 172.21.15.55:0/1294002063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf006b500 msgr2=0x7fdcf0153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf54b5640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf006b500 0x7fdcf0153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf54b5640 1 -- 172.21.15.55:0/1294002063 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00cb5f0 con 0x7fdcf00d32a0 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf54b5640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf00d32a0 0x7fdcf0161000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdce40097a0 tx=0x7fdce40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcee7fc640 1 -- 172.21.15.55:0/1294002063 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4037040 con 0x7fdcf00d32a0 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcee7fc640 1 -- 172.21.15.55:0/1294002063 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce4004250 con 0x7fdcf00d32a0 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcee7fc640 1 -- 172.21.15.55:0/1294002063 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4032e20 con 0x7fdcf00d32a0 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/1294002063 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf00c8a20 con 0x7fdcf00d32a0 2024-05-20T22:09:10.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.696+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/1294002063 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf00c8fe0 con 0x7fdcf00d32a0 2024-05-20T22:09:10.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.697+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/1294002063 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf01f9a80 con 0x7fdcf00d32a0 2024-05-20T22:09:10.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.698+0000 7fdcee7fc640 1 -- 172.21.15.55:0/1294002063 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdce4030770 con 0x7fdcf00d32a0 2024-05-20T22:09:10.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.699+0000 7fdcee7fc640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcdc06be30 0x7fdcdc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:10.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.699+0000 7fdcee7fc640 1 -- 172.21.15.55:0/1294002063 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fdce40abf40 con 0x7fdcf00d32a0 2024-05-20T22:09:10.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.699+0000 7fdcf5cb6640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcdc06be30 0x7fdcdc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:10.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.699+0000 7fdcf5cb6640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcdc06be30 0x7fdcdc06e340 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009f20 tx=0x7fdce80094d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:10.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.703+0000 7fdcee7fc640 1 -- 172.21.15.55:0/1294002063 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdce407aab0 con 0x7fdcf00d32a0 2024-05-20T22:09:10.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.959+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/1294002063 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fdcf0069090 con 0x7fdcf00d32a0 2024-05-20T22:09:10.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.966+0000 7fdcee7fc640 1 -- 172.21.15.55:0/1294002063 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v10) ==== 97+0+0 (secure 0 0 0) 0x7fdce407a3b0 con 0x7fdcf00d32a0 2024-05-20T22:09:10.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/1294002063 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcdc06be30 msgr2=0x7fdcdc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcdc06be30 0x7fdcdc06e340 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009f20 tx=0x7fdce80094d0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/1294002063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf00d32a0 msgr2=0x7fdcf0161000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:10.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf00d32a0 0x7fdcf0161000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdce40097a0 tx=0x7fdce40046c0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/1294002063 shutdown_connections 2024-05-20T22:09:10.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdcdc06be30 0x7fdcdc06e340 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf015f150 0x7fdcf0161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcf00d32a0 0x7fdcf0161000 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 --2- 172.21.15.55:0/1294002063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf006b500 0x7fdcf0153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:10.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/1294002063 >> 172.21.15.55:0/1294002063 conn(0x7fdcf00b6de0 msgr2=0x7fdcf006b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:10.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/1294002063 shutdown_connections 2024-05-20T22:09:10.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:10.968+0000 7fdcf7f1a640 1 -- 172.21.15.55:0/1294002063 wait complete. 2024-05-20T22:09:10.980 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-20T22:09:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.095+0000 7fccd17ce640 1 Processor -- start 2024-05-20T22:09:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.095+0000 7fccd17ce640 1 -- start start 2024-05-20T22:09:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fccd17ce640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcccc0d32a0 0x7fcccc15f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fccd17ce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcccc0d01b0 0x7fcccc06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fccd17ce640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcccc0d3780 con 0x7fcccc0cf690 2024-05-20T22:09:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fccd17ce640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fcccc0d3a60 con 0x7fcccc0d32a0 2024-05-20T22:09:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fccd17ce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcccc0d3d40 con 0x7fcccc0d01b0 2024-05-20T22:09:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fcccaffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcccc0d32a0 0x7fcccc15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fcccaffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcccc0d32a0 0x7fcccc15f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49282/0 (socket says 172.21.15.55:49282) 2024-05-20T22:09:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fcccaffd640 1 -- 172.21.15.55:0/2184735449 learned_addr learned my addr 172.21.15.55:0/2184735449 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:11.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fcccb7fe640 1 --2- 172.21.15.55:0/2184735449 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcccc0d01b0 0x7fcccc06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fcccaffd640 1 -- 172.21.15.55:0/2184735449 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcccc0d01b0 msgr2=0x7fcccc06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:11.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fcccaffd640 1 --2- 172.21.15.55:0/2184735449 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcccc0d01b0 0x7fcccc06b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fcccaffd640 1 -- 172.21.15.55:0/2184735449 >> v1:172.21.15.138:6789/0 conn(0x7fcccc0cf690 legacy=0x7fcccc0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:11.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.097+0000 7fcccaffd640 1 -- 172.21.15.55:0/2184735449 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcccc0d4020 con 0x7fcccc0d32a0 2024-05-20T22:09:11.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.098+0000 7fcccaffd640 1 --2- 172.21.15.55:0/2184735449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d32a0 0x7fcccc15f150 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009870 tx=0x7fccbc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295379eaa984eaf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:11.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.098+0000 7fccc9ffb640 1 -- 172.21.15.55:0/2184735449 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc037070 con 0x7fcccc0d32a0 2024-05-20T22:09:11.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.098+0000 7fccc9ffb640 1 -- 172.21.15.55:0/2184735449 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccbc032440 con 0x7fcccc0d32a0 2024-05-20T22:09:11.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.098+0000 7fccc9ffb640 1 -- 172.21.15.55:0/2184735449 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc032810 con 0x7fcccc0d32a0 2024-05-20T22:09:11.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.099+0000 7fccd17ce640 1 -- 172.21.15.55:0/2184735449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d32a0 msgr2=0x7fcccc15f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.099+0000 7fccd17ce640 1 --2- 172.21.15.55:0/2184735449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d32a0 0x7fcccc15f150 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009870 tx=0x7fccbc02a210 comp rx=0 tx=0).stop 2024-05-20T22:09:11.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.099+0000 7fccd17ce640 1 -- 172.21.15.55:0/2184735449 shutdown_connections 2024-05-20T22:09:11.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.099+0000 7fccd17ce640 1 --2- 172.21.15.55:0/2184735449 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcccc0d01b0 0x7fcccc06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.099+0000 7fccd17ce640 1 --2- 172.21.15.55:0/2184735449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d32a0 0x7fcccc15f150 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.099+0000 7fccd17ce640 1 -- 172.21.15.55:0/2184735449 >> 172.21.15.55:0/2184735449 conn(0x7fcccc0b6de0 msgr2=0x7fcccc0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:11.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.099+0000 7fccd17ce640 1 -- 172.21.15.55:0/2184735449 shutdown_connections 2024-05-20T22:09:11.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.099+0000 7fccd17ce640 1 -- 172.21.15.55:0/2184735449 wait complete. 2024-05-20T22:09:11.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.101+0000 7fccd17ce640 1 Processor -- start 2024-05-20T22:09:11.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.101+0000 7fccd17ce640 1 -- start start 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccd17ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcccc0cf690 0x7fcccc0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccd17ce640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d01b0 0x7fcccc159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccd17ce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcccc0d32a0 0x7fcccc0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccd17ce640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc0d4820 con 0x7fcccc0d01b0 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccd17ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc0b4350 con 0x7fcccc0cf690 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccd17ce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcccc1f6460 con 0x7fcccc0d32a0 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccca7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d01b0 0x7fcccc159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccca7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d01b0 0x7fcccc159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49288/0 (socket says 172.21.15.55:49288) 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccca7fc640 1 -- 172.21.15.55:0/3309051196 learned_addr learned my addr 172.21.15.55:0/3309051196 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccca7fc640 1 -- 172.21.15.55:0/3309051196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcccc0d32a0 msgr2=0x7fcccc0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccca7fc640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcccc0d32a0 0x7fcccc0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccca7fc640 1 -- 172.21.15.55:0/3309051196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcccc0cf690 msgr2=0x7fcccc0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccca7fc640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcccc0cf690 0x7fcccc0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.102+0000 7fccca7fc640 1 -- 172.21.15.55:0/3309051196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcccc0c9010 con 0x7fcccc0d01b0 2024-05-20T22:09:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.103+0000 7fccca7fc640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d01b0 0x7fcccc159410 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b420 tx=0x7fccc000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.103+0000 7fccb37fe640 1 -- 172.21.15.55:0/3309051196 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0011830 con 0x7fcccc0d01b0 2024-05-20T22:09:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.103+0000 7fccb37fe640 1 -- 172.21.15.55:0/3309051196 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc0016070 con 0x7fcccc0d01b0 2024-05-20T22:09:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.103+0000 7fccd17ce640 1 -- 172.21.15.55:0/3309051196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcccc0c92f0 con 0x7fcccc0d01b0 2024-05-20T22:09:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.103+0000 7fccb37fe640 1 -- 172.21.15.55:0/3309051196 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0011c10 con 0x7fcccc0d01b0 2024-05-20T22:09:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.103+0000 7fccd17ce640 1 -- 172.21.15.55:0/3309051196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcccc0c98b0 con 0x7fcccc0d01b0 2024-05-20T22:09:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.104+0000 7fccd17ce640 1 -- 172.21.15.55:0/3309051196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcccc069090 con 0x7fcccc0d01b0 2024-05-20T22:09:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.104+0000 7fccb37fe640 1 -- 172.21.15.55:0/3309051196 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fccc001b460 con 0x7fcccc0d01b0 2024-05-20T22:09:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.105+0000 7fccb37fe640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccb806be30 0x7fccb806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.105+0000 7fccb37fe640 1 -- 172.21.15.55:0/3309051196 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fccc008a800 con 0x7fcccc0d01b0 2024-05-20T22:09:11.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.105+0000 7fcccaffd640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccb806be30 0x7fccb806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.105+0000 7fcccaffd640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccb806be30 0x7fccb806e340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fccbc034000 tx=0x7fccbc02c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.109+0000 7fccb37fe640 1 -- 172.21.15.55:0/3309051196 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fccc0020020 con 0x7fcccc0d01b0 2024-05-20T22:09:11.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.371+0000 7fccd17ce640 1 -- 172.21.15.55:0/3309051196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fcccc0c8da0 con 0x7fcccc0d01b0 2024-05-20T22:09:11.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.378+0000 7fccb37fe640 1 -- 172.21.15.55:0/3309051196 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7fccc00592e0 con 0x7fcccc0d01b0 2024-05-20T22:09:11.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 -- 172.21.15.55:0/3309051196 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccb806be30 msgr2=0x7fccb806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccb806be30 0x7fccb806e340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fccbc034000 tx=0x7fccbc02c000 comp rx=0 tx=0).stop 2024-05-20T22:09:11.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 -- 172.21.15.55:0/3309051196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d01b0 msgr2=0x7fcccc159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d01b0 0x7fcccc159410 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b420 tx=0x7fccc000b8e0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 -- 172.21.15.55:0/3309051196 shutdown_connections 2024-05-20T22:09:11.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fccb806be30 0x7fccb806e340 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcccc0d32a0 0x7fcccc0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcccc0d01b0 0x7fcccc159410 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 --2- 172.21.15.55:0/3309051196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcccc0cf690 0x7fcccc0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 -- 172.21.15.55:0/3309051196 >> 172.21.15.55:0/3309051196 conn(0x7fcccc0b6de0 msgr2=0x7fcccc162ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:11.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 -- 172.21.15.55:0/3309051196 shutdown_connections 2024-05-20T22:09:11.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.380+0000 7fccd17ce640 1 -- 172.21.15.55:0/3309051196 wait complete. 2024-05-20T22:09:11.392 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:09:11.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.506+0000 7fbad80f0640 1 Processor -- start 2024-05-20T22:09:11.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.507+0000 7fbad80f0640 1 -- start start 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.508+0000 7fbad80f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbad00af980 0x7fbad00cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.508+0000 7fbad80f0640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbad00cfda0 0x7fbad0160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.508+0000 7fbad80f0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad01613f0 con 0x7fbad0160fe0 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.508+0000 7fbad80f0640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fbad00cbc60 con 0x7fbad00cfda0 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.508+0000 7fbad80f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbad00cbf40 con 0x7fbad00af980 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.508+0000 7fbad568b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbad00cfda0 0x7fbad0160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.508+0000 7fbad568b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbad00cfda0 0x7fbad0160aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49298/0 (socket says 172.21.15.55:49298) 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.508+0000 7fbad568b640 1 -- 172.21.15.55:0/3936395466 learned_addr learned my addr 172.21.15.55:0/3936395466 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.508+0000 7fbad5e8c640 1 --2- 172.21.15.55:0/3936395466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbad00af980 0x7fbad00cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.509+0000 7fbad568b640 1 -- 172.21.15.55:0/3936395466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbad00af980 msgr2=0x7fbad00cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.509+0000 7fbad568b640 1 --2- 172.21.15.55:0/3936395466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbad00af980 0x7fbad00cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.509+0000 7fbad568b640 1 -- 172.21.15.55:0/3936395466 >> v1:172.21.15.138:6789/0 conn(0x7fbad0160fe0 legacy=0x7fbad015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.509+0000 7fbad568b640 1 -- 172.21.15.55:0/3936395466 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad00cc220 con 0x7fbad00cfda0 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.509+0000 7fbad568b640 1 --2- 172.21.15.55:0/3936395466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 0x7fbad0160aa0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009870 tx=0x7fbac402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6457a9e3de04f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad4e8a640 1 -- 172.21.15.55:0/3936395466 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4037070 con 0x7fbad00cfda0 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad4e8a640 1 -- 172.21.15.55:0/3936395466 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac4032440 con 0x7fbad00cfda0 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad4e8a640 1 -- 172.21.15.55:0/3936395466 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4032810 con 0x7fbad00cfda0 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad80f0640 1 -- 172.21.15.55:0/3936395466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 msgr2=0x7fbad0160aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad80f0640 1 --2- 172.21.15.55:0/3936395466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 0x7fbad0160aa0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009870 tx=0x7fbac402a1c0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad80f0640 1 -- 172.21.15.55:0/3936395466 shutdown_connections 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad80f0640 1 --2- 172.21.15.55:0/3936395466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 0x7fbad0160aa0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad80f0640 1 --2- 172.21.15.55:0/3936395466 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbad00af980 0x7fbad00cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad80f0640 1 -- 172.21.15.55:0/3936395466 >> 172.21.15.55:0/3936395466 conn(0x7fbad00cd4d0 msgr2=0x7fbad00cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad80f0640 1 -- 172.21.15.55:0/3936395466 shutdown_connections 2024-05-20T22:09:11.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.510+0000 7fbad80f0640 1 -- 172.21.15.55:0/3936395466 wait complete. 2024-05-20T22:09:11.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.512+0000 7fbad80f0640 1 Processor -- start 2024-05-20T22:09:11.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.512+0000 7fbad80f0640 1 -- start start 2024-05-20T22:09:11.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.513+0000 7fbad80f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbad00af980 0x7fbad01e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.513+0000 7fbad80f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 0x7fbad01510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.513+0000 7fbad80f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbad0160fe0 0x7fbad01f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad80f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbad00d0330 con 0x7fbad00cfda0 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad80f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbad00d01b0 con 0x7fbad0160fe0 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad80f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbad006cad0 con 0x7fbad00af980 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad568b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 0x7fbad01510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad568b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 0x7fbad01510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49306/0 (socket says 172.21.15.55:49306) 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad568b640 1 -- 172.21.15.55:0/85723545 learned_addr learned my addr 172.21.15.55:0/85723545 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad5e8c640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbad00af980 0x7fbad01e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad668d640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbad0160fe0 0x7fbad01f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad568b640 1 -- 172.21.15.55:0/85723545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbad00af980 msgr2=0x7fbad01e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad568b640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbad00af980 0x7fbad01e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad568b640 1 -- 172.21.15.55:0/85723545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbad0160fe0 msgr2=0x7fbad01f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad568b640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbad0160fe0 0x7fbad01f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad568b640 1 -- 172.21.15.55:0/85723545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad01e7c40 con 0x7fbad00cfda0 2024-05-20T22:09:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad668d640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbad0160fe0 0x7fbad01f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad568b640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 0x7fbad01510f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbac4000c00 tx=0x7fbac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbace7fc640 1 -- 172.21.15.55:0/85723545 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4041020 con 0x7fbad00cfda0 2024-05-20T22:09:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbace7fc640 1 -- 172.21.15.55:0/85723545 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac40306c0 con 0x7fbad00cfda0 2024-05-20T22:09:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbace7fc640 1 -- 172.21.15.55:0/85723545 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4037040 con 0x7fbad00cfda0 2024-05-20T22:09:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad80f0640 1 -- 172.21.15.55:0/85723545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad00b3670 con 0x7fbad00cfda0 2024-05-20T22:09:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.514+0000 7fbad80f0640 1 -- 172.21.15.55:0/85723545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad00b3c30 con 0x7fbad00cfda0 2024-05-20T22:09:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.515+0000 7fbad80f0640 1 -- 172.21.15.55:0/85723545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad0069090 con 0x7fbad00cfda0 2024-05-20T22:09:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.516+0000 7fbace7fc640 1 -- 172.21.15.55:0/85723545 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbac4044020 con 0x7fbad00cfda0 2024-05-20T22:09:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.516+0000 7fbace7fc640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbabc06be30 0x7fbabc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.517+0000 7fbace7fc640 1 -- 172.21.15.55:0/85723545 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fbac40ab470 con 0x7fbad00cfda0 2024-05-20T22:09:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.517+0000 7fbad5e8c640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbabc06be30 0x7fbabc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.517+0000 7fbad5e8c640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbabc06be30 0x7fbabc06e340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbad00b22e0 tx=0x7fbac0009830 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:11.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.521+0000 7fbace7fc640 1 -- 172.21.15.55:0/85723545 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbac404b030 con 0x7fbad00cfda0 2024-05-20T22:09:11.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.705+0000 7fbad80f0640 1 -- 172.21.15.55:0/85723545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbad01e79d0 con 0x7fbad00cfda0 2024-05-20T22:09:11.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.706+0000 7fbace7fc640 1 -- 172.21.15.55:0/85723545 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13164 (secure 0 0 0) 0x7fbac407ee20 con 0x7fbad00cfda0 2024-05-20T22:09:11.707 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:11.707 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":34,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:09:08.541098+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.708+0000 7fbad80f0640 1 -- 172.21.15.55:0/85723545 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbabc06be30 msgr2=0x7fbabc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.708+0000 7fbad80f0640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbabc06be30 0x7fbabc06e340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbad00b22e0 tx=0x7fbac0009830 comp rx=0 tx=0).stop 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.708+0000 7fbad80f0640 1 -- 172.21.15.55:0/85723545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 msgr2=0x7fbad01510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.708+0000 7fbad80f0640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 0x7fbad01510f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbac4000c00 tx=0x7fbac40046c0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.709+0000 7fbad80f0640 1 -- 172.21.15.55:0/85723545 shutdown_connections 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.709+0000 7fbad80f0640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fbabc06be30 0x7fbabc06e340 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.709+0000 7fbad80f0640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbad0160fe0 0x7fbad01f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.709+0000 7fbad80f0640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbad00cfda0 0x7fbad01510f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.709+0000 7fbad80f0640 1 --2- 172.21.15.55:0/85723545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbad00af980 0x7fbad01e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.709+0000 7fbad80f0640 1 -- 172.21.15.55:0/85723545 >> 172.21.15.55:0/85723545 conn(0x7fbad00cd4d0 msgr2=0x7fbad015e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.709+0000 7fbad80f0640 1 -- 172.21.15.55:0/85723545 shutdown_connections 2024-05-20T22:09:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.709+0000 7fbad80f0640 1 -- 172.21.15.55:0/85723545 wait complete. 2024-05-20T22:09:11.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T22:09:11.721 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:09:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.837+0000 7f75844f7640 1 Processor -- start 2024-05-20T22:09:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.837+0000 7f75844f7640 1 -- start start 2024-05-20T22:09:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f75844f7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f757c0d32a0 0x7f757c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f75844f7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f757c0cf690 0x7f757c0cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f75844f7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c0cffe0 0x7f757c162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f75844f7640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f757c0d3780 con 0x7f757c0d32a0 2024-05-20T22:09:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f75844f7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f757c0d3a60 con 0x7f757c0cf690 2024-05-20T22:09:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f75844f7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f757c0d3d40 con 0x7f757c0cffe0 2024-05-20T22:09:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f7582293640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f757c0d32a0 0x7f757c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f7582293640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f757c0d32a0 0x7f757c15f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49318/0 (socket says 172.21.15.55:49318) 2024-05-20T22:09:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f7582293640 1 -- 172.21.15.55:0/3189460670 learned_addr learned my addr 172.21.15.55:0/3189460670 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f7581a92640 1 --2- 172.21.15.55:0/3189460670 >> v2:172.21.15.138:3300/0 conn(0x7f757c0cf690 0x7f757c0cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f7582293640 1 -- 172.21.15.55:0/3189460670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c0cffe0 msgr2=0x7f757c162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f7582293640 1 --2- 172.21.15.55:0/3189460670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c0cffe0 0x7f757c162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f7582293640 1 -- 172.21.15.55:0/3189460670 >> v2:172.21.15.138:3300/0 conn(0x7f757c0cf690 msgr2=0x7f757c0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f7582293640 1 --2- 172.21.15.55:0/3189460670 >> v2:172.21.15.138:3300/0 conn(0x7f757c0cf690 0x7f757c0cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.839+0000 7f7582293640 1 -- 172.21.15.55:0/3189460670 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c0d4020 con 0x7f757c0d32a0 2024-05-20T22:09:11.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.840+0000 7f7582293640 1 --2- 172.21.15.55:0/3189460670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0d32a0 0x7f757c15f150 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f757c08e150 tx=0x7f757402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6bee6232321e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:11.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.840+0000 7f7581291640 1 -- 172.21.15.55:0/3189460670 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574037070 con 0x7f757c0d32a0 2024-05-20T22:09:11.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.840+0000 7f7581291640 1 -- 172.21.15.55:0/3189460670 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7574032440 con 0x7f757c0d32a0 2024-05-20T22:09:11.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.840+0000 7f7581291640 1 -- 172.21.15.55:0/3189460670 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574032810 con 0x7f757c0d32a0 2024-05-20T22:09:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.841+0000 7f75844f7640 1 -- 172.21.15.55:0/3189460670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0d32a0 msgr2=0x7f757c15f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.841+0000 7f75844f7640 1 --2- 172.21.15.55:0/3189460670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0d32a0 0x7f757c15f150 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f757c08e150 tx=0x7f757402a200 comp rx=0 tx=0).stop 2024-05-20T22:09:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.841+0000 7f75844f7640 1 -- 172.21.15.55:0/3189460670 shutdown_connections 2024-05-20T22:09:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.841+0000 7f75844f7640 1 --2- 172.21.15.55:0/3189460670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c0cffe0 0x7f757c162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.841+0000 7f75844f7640 1 --2- 172.21.15.55:0/3189460670 >> v2:172.21.15.138:3300/0 conn(0x7f757c0cf690 0x7f757c0cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.841+0000 7f75844f7640 1 --2- 172.21.15.55:0/3189460670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0d32a0 0x7f757c15f150 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.841+0000 7f75844f7640 1 -- 172.21.15.55:0/3189460670 >> 172.21.15.55:0/3189460670 conn(0x7f757c0b6de0 msgr2=0x7f757c0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.841+0000 7f75844f7640 1 -- 172.21.15.55:0/3189460670 shutdown_connections 2024-05-20T22:09:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.841+0000 7f75844f7640 1 -- 172.21.15.55:0/3189460670 wait complete. 2024-05-20T22:09:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.843+0000 7f75844f7640 1 Processor -- start 2024-05-20T22:09:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.843+0000 7f75844f7640 1 -- start start 2024-05-20T22:09:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f75844f7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c0cf690 0x7f757c14dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f75844f7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0cffe0 0x7f757c1f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f75844f7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0d32a0 0x7f757c1f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f75844f7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f757c0d4820 con 0x7f757c0cffe0 2024-05-20T22:09:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f75844f7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f757c0d03f0 con 0x7f757c0d32a0 2024-05-20T22:09:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f75844f7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f757c15a790 con 0x7f757c0cf690 2024-05-20T22:09:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f7581a92640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0cffe0 0x7f757c1f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f7581a92640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0cffe0 0x7f757c1f5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49322/0 (socket says 172.21.15.55:49322) 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f7581a92640 1 -- 172.21.15.55:0/1920922604 learned_addr learned my addr 172.21.15.55:0/1920922604 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f7582293640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c0cf690 0x7f757c14dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.844+0000 7f7582a94640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0d32a0 0x7f757c1f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f7581a92640 1 -- 172.21.15.55:0/1920922604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c0cf690 msgr2=0x7f757c14dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f7581a92640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c0cf690 0x7f757c14dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f7581a92640 1 -- 172.21.15.55:0/1920922604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0d32a0 msgr2=0x7f757c1f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f7581a92640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0d32a0 0x7f757c1f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f7581a92640 1 -- 172.21.15.55:0/1920922604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c0cc130 con 0x7f757c0cffe0 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f7582a94640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0d32a0 0x7f757c1f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f7581a92640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0cffe0 0x7f757c1f5a70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f757000c820 tx=0x7f757000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f757affd640 1 -- 172.21.15.55:0/1920922604 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570015070 con 0x7f757c0cffe0 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f757affd640 1 -- 172.21.15.55:0/1920922604 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7570004020 con 0x7f757c0cffe0 2024-05-20T22:09:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f757affd640 1 -- 172.21.15.55:0/1920922604 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700043f0 con 0x7f757c0cffe0 2024-05-20T22:09:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f75844f7640 1 -- 172.21.15.55:0/1920922604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757c0cc410 con 0x7f757c0cffe0 2024-05-20T22:09:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.845+0000 7f75844f7640 1 -- 172.21.15.55:0/1920922604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757c0cc9d0 con 0x7f757c0cffe0 2024-05-20T22:09:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.846+0000 7f75844f7640 1 -- 172.21.15.55:0/1920922604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757c0cbea0 con 0x7f757c0cffe0 2024-05-20T22:09:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.847+0000 7f757affd640 1 -- 172.21.15.55:0/1920922604 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7570007830 con 0x7f757c0cffe0 2024-05-20T22:09:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.847+0000 7f757affd640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f756806be30 0x7f756806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.847+0000 7f757affd640 1 -- 172.21.15.55:0/1920922604 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f757008a9e0 con 0x7f757c0cffe0 2024-05-20T22:09:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.847+0000 7f7582293640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f756806be30 0x7f756806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.848+0000 7f7582293640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f756806be30 0x7f756806e340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7574034000 tx=0x7f757402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:11.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:11.852+0000 7f757affd640 1 -- 172.21.15.55:0/1920922604 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7570058f00 con 0x7f757c0cffe0 2024-05-20T22:09:12.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.040+0000 7f75844f7640 1 -- 172.21.15.55:0/1920922604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f757c069090 con 0x7f757c0cffe0 2024-05-20T22:09:12.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.040+0000 7f757affd640 1 -- 172.21.15.55:0/1920922604 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f757005d200 con 0x7f757c0cffe0 2024-05-20T22:09:12.041 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:09:12.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 -- 172.21.15.55:0/1920922604 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f756806be30 msgr2=0x7f756806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f756806be30 0x7f756806e340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7574034000 tx=0x7f757402e000 comp rx=0 tx=0).stop 2024-05-20T22:09:12.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 -- 172.21.15.55:0/1920922604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0cffe0 msgr2=0x7f757c1f5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0cffe0 0x7f757c1f5a70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f757000c820 tx=0x7f757000cce0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 -- 172.21.15.55:0/1920922604 shutdown_connections 2024-05-20T22:09:12.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f756806be30 0x7f756806e340 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757c0d32a0 0x7f757c1f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f757c0cffe0 0x7f757c1f5a70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 --2- 172.21.15.55:0/1920922604 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f757c0cf690 0x7f757c14dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 -- 172.21.15.55:0/1920922604 >> 172.21.15.55:0/1920922604 conn(0x7f757c0b6de0 msgr2=0x7f757c0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:12.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 -- 172.21.15.55:0/1920922604 shutdown_connections 2024-05-20T22:09:12.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.042+0000 7f75844f7640 1 -- 172.21.15.55:0/1920922604 wait complete. 2024-05-20T22:09:12.054 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T22:09:12.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.167+0000 7f4101185640 1 Processor -- start 2024-05-20T22:09:12.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.168+0000 7f4101185640 1 -- start start 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.169+0000 7f4101185640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f40fc15f100 0x7f40fc0b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.169+0000 7f4101185640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40fc0d3dd0 con 0x7f40fc06c790 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.169+0000 7f4101185640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40fc0d40b0 con 0x7f40fc0b3b30 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.169+0000 7f4101185640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f40fc0d4390 con 0x7f40fc15f100 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.169+0000 7f40fb59e640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f40fc06c790 0x7f40fc0cf740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54220/0 (socket says 172.21.15.55:54220) 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.169+0000 7f40fb59e640 1 -- 172.21.15.55:0/3238036967 learned_addr learned my addr 172.21.15.55:0/3238036967 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.169+0000 7f40fa59c640 1 --2- 172.21.15.55:0/3238036967 >> v2:172.21.15.138:3300/0 conn(0x7f40fc15f100 0x7f40fc0b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.169+0000 7f40f9d9b640 1 -- 172.21.15.55:0/3238036967 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 752311039 0 0) 0x7f40fc0d3dd0 con 0x7f40fc06c790 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.169+0000 7f40f9d9b640 1 -- 172.21.15.55:0/3238036967 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40e4003520 con 0x7f40fc06c790 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.170+0000 7f40f9d9b640 1 -- 172.21.15.55:0/3238036967 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3329794859 0 0) 0x7f40fc0d40b0 con 0x7f40fc0b3b30 2024-05-20T22:09:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.170+0000 7f40f9d9b640 1 -- 172.21.15.55:0/3238036967 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40fc0d3dd0 con 0x7f40fc0b3b30 2024-05-20T22:09:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.170+0000 7f40f9d9b640 1 -- 172.21.15.55:0/3238036967 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3445826184 0 0) 0x7f40e4003520 con 0x7f40fc06c790 2024-05-20T22:09:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.170+0000 7f40fa59c640 1 -- 172.21.15.55:0/3238036967 >> v1:172.21.15.138:6789/0 conn(0x7f40fc0b3b30 legacy=0x7f40fc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.170+0000 7f40fa59c640 1 -- 172.21.15.55:0/3238036967 >> v1:172.21.15.55:6789/0 conn(0x7f40fc06c790 legacy=0x7f40fc0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.170+0000 7f40fa59c640 1 -- 172.21.15.55:0/3238036967 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc06b5a0 con 0x7f40fc15f100 2024-05-20T22:09:12.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.170+0000 7f40fa59c640 1 --2- 172.21.15.55:0/3238036967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40fc15f100 0x7f40fc0b7fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f40f00097a0 tx=0x7f40f002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b460c4392864a9b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:12.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.171+0000 7f40f9d9b640 1 -- 172.21.15.55:0/3238036967 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0037070 con 0x7f40fc15f100 2024-05-20T22:09:12.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.171+0000 7f40f9d9b640 1 -- 172.21.15.55:0/3238036967 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f002ac50 con 0x7f40fc15f100 2024-05-20T22:09:12.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.171+0000 7f40f9d9b640 1 -- 172.21.15.55:0/3238036967 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40019f0 con 0x7f40fc15f100 2024-05-20T22:09:12.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.171+0000 7f4101185640 1 -- 172.21.15.55:0/3238036967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40fc15f100 msgr2=0x7f40fc0b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.171+0000 7f4101185640 1 --2- 172.21.15.55:0/3238036967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40fc15f100 0x7f40fc0b7fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f40f00097a0 tx=0x7f40f002a1c0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.171+0000 7f4101185640 1 -- 172.21.15.55:0/3238036967 shutdown_connections 2024-05-20T22:09:12.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.171+0000 7f4101185640 1 --2- 172.21.15.55:0/3238036967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40fc15f100 0x7f40fc0b7fe0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.171+0000 7f4101185640 1 -- 172.21.15.55:0/3238036967 >> 172.21.15.55:0/3238036967 conn(0x7f40fc0d51d0 msgr2=0x7f40fc0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:12.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.171+0000 7f4101185640 1 -- 172.21.15.55:0/3238036967 shutdown_connections 2024-05-20T22:09:12.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.171+0000 7f4101185640 1 -- 172.21.15.55:0/3238036967 wait complete. 2024-05-20T22:09:12.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.174+0000 7f4101185640 1 Processor -- start 2024-05-20T22:09:12.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.174+0000 7f4101185640 1 -- start start 2024-05-20T22:09:12.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f4101185640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40fc06c790 0x7f40fc156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f4101185640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40fc0b3b30 0x7f40fc1f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f4101185640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40fc15f100 0x7f40fc158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f4101185640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc06cba0 con 0x7f40fc15f100 2024-05-20T22:09:12.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f4101185640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc06bb60 con 0x7f40fc0b3b30 2024-05-20T22:09:12.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f4101185640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f40fc06cd20 con 0x7f40fc06c790 2024-05-20T22:09:12.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f40fb59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40fc15f100 0x7f40fc158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f40fb59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40fc15f100 0x7f40fc158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49338/0 (socket says 172.21.15.55:49338) 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f40fb59e640 1 -- 172.21.15.55:0/3141010969 learned_addr learned my addr 172.21.15.55:0/3141010969 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f40fa59c640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40fc0b3b30 0x7f40fc1f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f40fad9d640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40fc06c790 0x7f40fc156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f40fb59e640 1 -- 172.21.15.55:0/3141010969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40fc06c790 msgr2=0x7f40fc156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f40fb59e640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40fc06c790 0x7f40fc156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f40fb59e640 1 -- 172.21.15.55:0/3141010969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40fc0b3b30 msgr2=0x7f40fc1f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f40fb59e640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40fc0b3b30 0x7f40fc1f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.175+0000 7f40fb59e640 1 -- 172.21.15.55:0/3141010969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc1f9970 con 0x7f40fc15f100 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.176+0000 7f40fb59e640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40fc15f100 0x7f40fc158e50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f40f4001350 tx=0x7f40f400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.176+0000 7f40e37fe640 1 -- 172.21.15.55:0/3141010969 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4017070 con 0x7f40fc15f100 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.176+0000 7f40e37fe640 1 -- 172.21.15.55:0/3141010969 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f400ad40 con 0x7f40fc15f100 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.176+0000 7f40e37fe640 1 -- 172.21.15.55:0/3141010969 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4012600 con 0x7f40fc15f100 2024-05-20T22:09:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.176+0000 7f4101185640 1 -- 172.21.15.55:0/3141010969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40fc1574a0 con 0x7f40fc15f100 2024-05-20T22:09:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.176+0000 7f4101185640 1 -- 172.21.15.55:0/3141010969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40fc157a60 con 0x7f40fc15f100 2024-05-20T22:09:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.177+0000 7f4101185640 1 -- 172.21.15.55:0/3141010969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40fc069090 con 0x7f40fc15f100 2024-05-20T22:09:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.178+0000 7f40e37fe640 1 -- 172.21.15.55:0/3141010969 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40f401a070 con 0x7f40fc15f100 2024-05-20T22:09:12.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.178+0000 7f40e37fe640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f40e806be60 0x7f40e806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.178+0000 7f40e37fe640 1 -- 172.21.15.55:0/3141010969 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f40f408aaf0 con 0x7f40fc15f100 2024-05-20T22:09:12.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.178+0000 7f40fad9d640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f40e806be60 0x7f40e806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.179+0000 7f40fad9d640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f40e806be60 0x7f40e806e370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0011e0 tx=0x7f40ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:12.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.183+0000 7f40e37fe640 1 -- 172.21.15.55:0/3141010969 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f40f4058f70 con 0x7f40fc15f100 2024-05-20T22:09:12.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.370+0000 7f4101185640 1 -- 172.21.15.55:0/3141010969 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f40fc0d2ee0 con 0x7f40fc15f100 2024-05-20T22:09:12.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.370+0000 7f40e37fe640 1 -- 172.21.15.55:0/3141010969 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7f40f405d270 con 0x7f40fc15f100 2024-05-20T22:09:12.372 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:12.372 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-05-20T22:09:12.372 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:09:12.372 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-05-20T22:09:12.372 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-05-20T22:09:12.372 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:12.373 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-05-20T22:09:12.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 -- 172.21.15.55:0/3141010969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f40e806be60 msgr2=0x7f40e806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f40e806be60 0x7f40e806e370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0011e0 tx=0x7f40ec009040 comp rx=0 tx=0).stop 2024-05-20T22:09:12.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 -- 172.21.15.55:0/3141010969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40fc15f100 msgr2=0x7f40fc158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40fc15f100 0x7f40fc158e50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f40f4001350 tx=0x7f40f400a430 comp rx=0 tx=0).stop 2024-05-20T22:09:12.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 -- 172.21.15.55:0/3141010969 shutdown_connections 2024-05-20T22:09:12.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f40e806be60 0x7f40e806e370 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40fc15f100 0x7f40fc158e50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40fc0b3b30 0x7f40fc1f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 --2- 172.21.15.55:0/3141010969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40fc06c790 0x7f40fc156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 -- 172.21.15.55:0/3141010969 >> 172.21.15.55:0/3141010969 conn(0x7f40fc0d51d0 msgr2=0x7f40fc15a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:12.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 -- 172.21.15.55:0/3141010969 shutdown_connections 2024-05-20T22:09:12.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.373+0000 7f4101185640 1 -- 172.21.15.55:0/3141010969 wait complete. 2024-05-20T22:09:12.385 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T22:09:12.386 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T22:09:12.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.503+0000 7fadc821b640 1 Processor -- start 2024-05-20T22:09:12.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.503+0000 7fadc821b640 1 -- start start 2024-05-20T22:09:12.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc821b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fadc00b2470 0x7fadc00b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc821b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00cfad0 0x7fadc00d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc821b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc00b37b0 con 0x7fadc006d0b0 2024-05-20T22:09:12.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc821b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fadc00b2880 con 0x7fadc00b2470 2024-05-20T22:09:12.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc821b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fadc006d8b0 con 0x7fadc00cfad0 2024-05-20T22:09:12.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc67b8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00cfad0 0x7fadc00d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc5fb7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fadc00b2470 0x7fadc00b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc57b6640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fadc006d0b0 0x7fadc00cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:37586/0 (socket says 172.21.15.55:37586) 2024-05-20T22:09:12.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc5fb7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fadc00b2470 0x7fadc00b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:46486/0 (socket says 172.21.15.55:46486) 2024-05-20T22:09:12.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc67b8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00cfad0 0x7fadc00d0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:47908/0 (socket says 172.21.15.55:47908) 2024-05-20T22:09:12.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.505+0000 7fadc57b6640 1 -- 172.21.15.55:0/778145833 learned_addr learned my addr 172.21.15.55:0/778145833 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:12.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.506+0000 7fadc5fb7640 1 -- 172.21.15.55:0/778145833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00cfad0 msgr2=0x7fadc00d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:12.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.506+0000 7fadc5fb7640 1 --2- 172.21.15.55:0/778145833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00cfad0 0x7fadc00d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.506+0000 7fadc5fb7640 1 -- 172.21.15.55:0/778145833 >> v1:172.21.15.138:6789/0 conn(0x7fadc006d0b0 legacy=0x7fadc00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.506+0000 7fadc5fb7640 1 -- 172.21.15.55:0/778145833 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc006db90 con 0x7fadc00b2470 2024-05-20T22:09:12.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.507+0000 7fadc5fb7640 1 --2- 172.21.15.55:0/778145833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc00b2470 0x7fadc00b1eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009d90 tx=0x7fadb002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8764d1b53a7693be server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:12.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.507+0000 7fadc4fb5640 1 -- 172.21.15.55:0/778145833 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0007740 con 0x7fadc00b2470 2024-05-20T22:09:12.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.507+0000 7fadc4fb5640 1 -- 172.21.15.55:0/778145833 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb00078e0 con 0x7fadc00b2470 2024-05-20T22:09:12.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.507+0000 7fadc4fb5640 1 -- 172.21.15.55:0/778145833 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0007cb0 con 0x7fadc00b2470 2024-05-20T22:09:12.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.507+0000 7fadc821b640 1 -- 172.21.15.55:0/778145833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc00b2470 msgr2=0x7fadc00b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.507+0000 7fadc821b640 1 --2- 172.21.15.55:0/778145833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc00b2470 0x7fadc00b1eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009d90 tx=0x7fadb002a660 comp rx=0 tx=0).stop 2024-05-20T22:09:12.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.507+0000 7fadc821b640 1 -- 172.21.15.55:0/778145833 shutdown_connections 2024-05-20T22:09:12.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.507+0000 7fadc821b640 1 --2- 172.21.15.55:0/778145833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00cfad0 0x7fadc00d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.507+0000 7fadc821b640 1 --2- 172.21.15.55:0/778145833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc00b2470 0x7fadc00b1eb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.507+0000 7fadc821b640 1 -- 172.21.15.55:0/778145833 >> 172.21.15.55:0/778145833 conn(0x7fadc009be10 msgr2=0x7fadc00d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:12.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.508+0000 7fadc821b640 1 -- 172.21.15.55:0/778145833 shutdown_connections 2024-05-20T22:09:12.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.508+0000 7fadc821b640 1 -- 172.21.15.55:0/778145833 wait complete. 2024-05-20T22:09:12.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.509+0000 7fadc821b640 1 Processor -- start 2024-05-20T22:09:12.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.509+0000 7fadc821b640 1 -- start start 2024-05-20T22:09:12.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc821b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc006d0b0 0x7fadc0154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc821b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00b2470 0x7fadc01f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc821b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc01614e0 0x7fadc01618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc821b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fadc006d4c0 con 0x7fadc006d0b0 2024-05-20T22:09:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc821b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fadc00b3a30 con 0x7fadc01614e0 2024-05-20T22:09:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc821b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fadc00b3f50 con 0x7fadc00b2470 2024-05-20T22:09:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc5fb7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc006d0b0 0x7fadc0154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc5fb7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc006d0b0 0x7fadc0154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49350/0 (socket says 172.21.15.55:49350) 2024-05-20T22:09:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc5fb7640 1 -- 172.21.15.55:0/125068632 learned_addr learned my addr 172.21.15.55:0/125068632 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc57b6640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00b2470 0x7fadc01f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc67b8640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc01614e0 0x7fadc01618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc5fb7640 1 -- 172.21.15.55:0/125068632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00b2470 msgr2=0x7fadc01f8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc5fb7640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00b2470 0x7fadc01f8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc5fb7640 1 -- 172.21.15.55:0/125068632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc01614e0 msgr2=0x7fadc01618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:12.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc5fb7640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc01614e0 0x7fadc01618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:12.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc5fb7640 1 -- 172.21.15.55:0/125068632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc0161e30 con 0x7fadc006d0b0 2024-05-20T22:09:12.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadc5fb7640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc006d0b0 0x7fadc0154060 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009d90 tx=0x7fadb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:12.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.511+0000 7fadbe7fc640 1 -- 172.21.15.55:0/125068632 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb003f070 con 0x7fadc006d0b0 2024-05-20T22:09:12.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.512+0000 7fadbe7fc640 1 -- 172.21.15.55:0/125068632 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb0031c40 con 0x7fadc006d0b0 2024-05-20T22:09:12.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.512+0000 7fadc821b640 1 -- 172.21.15.55:0/125068632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc009ce60 con 0x7fadc006d0b0 2024-05-20T22:09:12.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.512+0000 7fadbe7fc640 1 -- 172.21.15.55:0/125068632 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0009450 con 0x7fadc006d0b0 2024-05-20T22:09:12.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.512+0000 7fadc821b640 1 -- 172.21.15.55:0/125068632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc009eb00 con 0x7fadc006d0b0 2024-05-20T22:09:12.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.513+0000 7fadc821b640 1 -- 172.21.15.55:0/125068632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc0069090 con 0x7fadc006d0b0 2024-05-20T22:09:12.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.513+0000 7fadbe7fc640 1 -- 172.21.15.55:0/125068632 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadb0044460 con 0x7fadc006d0b0 2024-05-20T22:09:12.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.514+0000 7fadbe7fc640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fadac06be30 0x7fadac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:12.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.514+0000 7fadbe7fc640 1 -- 172.21.15.55:0/125068632 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fadb00ac5c0 con 0x7fadc006d0b0 2024-05-20T22:09:12.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.514+0000 7fadc57b6640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fadac06be30 0x7fadac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:12.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.514+0000 7fadc57b6640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fadac06be30 0x7fadac06e340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fadc00b33f0 tx=0x7fadb4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:12.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.518+0000 7fadbe7fc640 1 -- 172.21.15.55:0/125068632 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fadc0161e30 con 0x7fadc006d0b0 2024-05-20T22:09:12.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:12.702+0000 7fadc821b640 1 -- 172.21.15.55:0/125068632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fadc00d35a0 con 0x7fadc006d0b0 2024-05-20T22:09:13.328 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:09:13.331 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:09:13.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.554+0000 7fadbe7fc640 1 -- 172.21.15.55:0/125068632 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7fadb007aa50 con 0x7fadc006d0b0 2024-05-20T22:09:13.555 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-05-20T22:09:13.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 -- 172.21.15.55:0/125068632 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fadac06be30 msgr2=0x7fadac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:13.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fadac06be30 0x7fadac06e340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fadc00b33f0 tx=0x7fadb4009040 comp rx=0 tx=0).stop 2024-05-20T22:09:13.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 -- 172.21.15.55:0/125068632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc006d0b0 msgr2=0x7fadc0154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:13.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc006d0b0 0x7fadc0154060 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009d90 tx=0x7fadb0004a40 comp rx=0 tx=0).stop 2024-05-20T22:09:13.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 -- 172.21.15.55:0/125068632 shutdown_connections 2024-05-20T22:09:13.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fadac06be30 0x7fadac06e340 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:13.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc01614e0 0x7fadc01618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:13.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fadc00b2470 0x7fadc01f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:13.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 --2- 172.21.15.55:0/125068632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc006d0b0 0x7fadc0154060 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:13.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 -- 172.21.15.55:0/125068632 >> 172.21.15.55:0/125068632 conn(0x7fadc009be10 msgr2=0x7fadc008bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:13.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 -- 172.21.15.55:0/125068632 shutdown_connections 2024-05-20T22:09:13.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.557+0000 7fadc821b640 1 -- 172.21.15.55:0/125068632 wait complete. 2024-05-20T22:09:13.576 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T22:09:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.704+0000 7f4e6cf77640 1 Processor -- start 2024-05-20T22:09:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.704+0000 7f4e6cf77640 1 -- start start 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.705+0000 7f4e6cf77640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e680b7ae0 0x7f4e680d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.705+0000 7f4e6cf77640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4e6809abd0 0x7f4e6809afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.705+0000 7f4e6cf77640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e680b80d0 con 0x7f4e680ccf50 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.705+0000 7f4e6cf77640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f4e680b83b0 con 0x7f4e6809abd0 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.705+0000 7f4e6cf77640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4e6809ba70 con 0x7f4e680b7ae0 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.705+0000 7f4e65d9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4e6809abd0 0x7f4e6809afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.705+0000 7f4e66d9d640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4e680ccf50 0x7f4e680cd360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54232/0 (socket says 172.21.15.55:54232) 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.705+0000 7f4e66d9d640 1 -- 172.21.15.55:0/2815659114 learned_addr learned my addr 172.21.15.55:0/2815659114 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.706+0000 7f4e6659c640 1 --2- 172.21.15.55:0/2815659114 >> v2:172.21.15.138:3300/0 conn(0x7f4e680b7ae0 0x7f4e680d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.706+0000 7f4e65d9b640 1 -- 172.21.15.55:0/2815659114 >> v2:172.21.15.138:3300/0 conn(0x7f4e680b7ae0 msgr2=0x7f4e680d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.706+0000 7f4e65d9b640 1 --2- 172.21.15.55:0/2815659114 >> v2:172.21.15.138:3300/0 conn(0x7f4e680b7ae0 0x7f4e680d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:13.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.706+0000 7f4e65d9b640 1 -- 172.21.15.55:0/2815659114 >> v1:172.21.15.55:6789/0 conn(0x7f4e680ccf50 legacy=0x7f4e680cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:13.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.706+0000 7f4e65d9b640 1 -- 172.21.15.55:0/2815659114 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e680d18b0 con 0x7f4e6809abd0 2024-05-20T22:09:13.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.706+0000 7f4e6559a640 1 -- 172.21.15.55:0/2815659114 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1073630097 0 0) 0x7f4e680b80d0 con 0x7f4e680ccf50 2024-05-20T22:09:13.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.707+0000 7f4e65d9b640 1 --2- 172.21.15.55:0/2815659114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 0x7f4e6809afe0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0097a0 tx=0x7f4e5c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76117a7092b6b75f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:13.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.707+0000 7f4e6559a640 1 -- 172.21.15.55:0/2815659114 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c037070 con 0x7f4e6809abd0 2024-05-20T22:09:13.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.707+0000 7f4e6559a640 1 -- 172.21.15.55:0/2815659114 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5c02ae50 con 0x7f4e6809abd0 2024-05-20T22:09:13.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.707+0000 7f4e6559a640 1 -- 172.21.15.55:0/2815659114 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c032660 con 0x7f4e6809abd0 2024-05-20T22:09:13.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.708+0000 7f4e6cf77640 1 -- 172.21.15.55:0/2815659114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 msgr2=0x7f4e6809afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:13.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.708+0000 7f4e6cf77640 1 --2- 172.21.15.55:0/2815659114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 0x7f4e6809afe0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0097a0 tx=0x7f4e5c02a140 comp rx=0 tx=0).stop 2024-05-20T22:09:13.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.708+0000 7f4e6cf77640 1 -- 172.21.15.55:0/2815659114 shutdown_connections 2024-05-20T22:09:13.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.708+0000 7f4e6cf77640 1 --2- 172.21.15.55:0/2815659114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 0x7f4e6809afe0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:13.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.708+0000 7f4e6cf77640 1 --2- 172.21.15.55:0/2815659114 >> v2:172.21.15.138:3300/0 conn(0x7f4e680b7ae0 0x7f4e680d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:13.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.708+0000 7f4e6cf77640 1 -- 172.21.15.55:0/2815659114 >> 172.21.15.55:0/2815659114 conn(0x7f4e680b43f0 msgr2=0x7f4e680b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:13.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.708+0000 7f4e6cf77640 1 -- 172.21.15.55:0/2815659114 shutdown_connections 2024-05-20T22:09:13.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.708+0000 7f4e6cf77640 1 -- 172.21.15.55:0/2815659114 wait complete. 2024-05-20T22:09:13.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.710+0000 7f4e6cf77640 1 Processor -- start 2024-05-20T22:09:13.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.710+0000 7f4e6cf77640 1 -- start start 2024-05-20T22:09:13.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6cf77640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 0x7f4e68154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:13.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6cf77640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e680b7ae0 0x7f4e681609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:13.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6cf77640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e680ccf50 0x7f4e68161de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:13.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6cf77640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e680b39b0 con 0x7f4e6809abd0 2024-05-20T22:09:13.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6cf77640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e680b3830 con 0x7f4e680ccf50 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6cf77640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4e680cda70 con 0x7f4e680b7ae0 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 0x7f4e68154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 0x7f4e68154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49378/0 (socket says 172.21.15.55:49378) 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6659c640 1 -- 172.21.15.55:0/3155294655 learned_addr learned my addr 172.21.15.55:0/3155294655 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e65d9b640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e680b7ae0 0x7f4e681609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e66d9d640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e680ccf50 0x7f4e68161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6659c640 1 -- 172.21.15.55:0/3155294655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e680b7ae0 msgr2=0x7f4e681609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6659c640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e680b7ae0 0x7f4e681609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6659c640 1 -- 172.21.15.55:0/3155294655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e680ccf50 msgr2=0x7f4e68161de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6659c640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e680ccf50 0x7f4e68161de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.711+0000 7f4e6659c640 1 -- 172.21.15.55:0/3155294655 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e680c9bc0 con 0x7f4e6809abd0 2024-05-20T22:09:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.712+0000 7f4e6659c640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 0x7f4e68154da0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800e870 tx=0x7f4e5800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.712+0000 7f4e56ffd640 1 -- 172.21.15.55:0/3155294655 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58017070 con 0x7f4e6809abd0 2024-05-20T22:09:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.712+0000 7f4e56ffd640 1 -- 172.21.15.55:0/3155294655 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5800c940 con 0x7f4e6809abd0 2024-05-20T22:09:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.712+0000 7f4e6cf77640 1 -- 172.21.15.55:0/3155294655 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e680c9ea0 con 0x7f4e6809abd0 2024-05-20T22:09:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.712+0000 7f4e56ffd640 1 -- 172.21.15.55:0/3155294655 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5800cd10 con 0x7f4e6809abd0 2024-05-20T22:09:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.712+0000 7f4e6cf77640 1 -- 172.21.15.55:0/3155294655 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e680ca460 con 0x7f4e6809abd0 2024-05-20T22:09:13.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.713+0000 7f4e6cf77640 1 -- 172.21.15.55:0/3155294655 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e6808bd10 con 0x7f4e6809abd0 2024-05-20T22:09:13.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.714+0000 7f4e56ffd640 1 -- 172.21.15.55:0/3155294655 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e5801a070 con 0x7f4e6809abd0 2024-05-20T22:09:13.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.715+0000 7f4e56ffd640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e4806be30 0x7f4e4806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.715+0000 7f4e56ffd640 1 -- 172.21.15.55:0/3155294655 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7f4e5808b050 con 0x7f4e6809abd0 2024-05-20T22:09:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.715+0000 7f4e65d9b640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e4806be30 0x7f4e4806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.715+0000 7f4e65d9b640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e4806be30 0x7f4e4806e340 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c034000 tx=0x7f4e5c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:13.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.719+0000 7f4e56ffd640 1 -- 172.21.15.55:0/3155294655 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4e6808bd10 con 0x7f4e6809abd0 2024-05-20T22:09:13.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:13.904+0000 7f4e6cf77640 1 -- 172.21.15.55:0/3155294655 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4e68069090 con 0x7f4e6809abd0 2024-05-20T22:09:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.566+0000 7f4e56ffd640 1 -- 172.21.15.55:0/3155294655 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7f4e58059a50 con 0x7f4e6809abd0 2024-05-20T22:09:14.567 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-05-20T22:09:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 -- 172.21.15.55:0/3155294655 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e4806be30 msgr2=0x7f4e4806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e4806be30 0x7f4e4806e340 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c034000 tx=0x7f4e5c02e000 comp rx=0 tx=0).stop 2024-05-20T22:09:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 -- 172.21.15.55:0/3155294655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 msgr2=0x7f4e68154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 0x7f4e68154da0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800e870 tx=0x7f4e5800ed30 comp rx=0 tx=0).stop 2024-05-20T22:09:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 -- 172.21.15.55:0/3155294655 shutdown_connections 2024-05-20T22:09:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e4806be30 0x7f4e4806e340 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e680ccf50 0x7f4e68161de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e680b7ae0 0x7f4e681609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 --2- 172.21.15.55:0/3155294655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e6809abd0 0x7f4e68154da0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 -- 172.21.15.55:0/3155294655 >> 172.21.15.55:0/3155294655 conn(0x7f4e680b43f0 msgr2=0x7f4e68071f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 -- 172.21.15.55:0/3155294655 shutdown_connections 2024-05-20T22:09:14.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.568+0000 7f4e6cf77640 1 -- 172.21.15.55:0/3155294655 wait complete. 2024-05-20T22:09:14.586 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T22:09:14.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.712+0000 7f6cc39aa640 1 Processor -- start 2024-05-20T22:09:14.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.712+0000 7f6cc39aa640 1 -- start start 2024-05-20T22:09:14.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc39aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cbc0cd520 0x7f6cbc08c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:14.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc39aa640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6cbc0d0860 0x7f6cbc0b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc39aa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cbc0b1f30 con 0x7f6cbc0cf270 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc39aa640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f6cbc0d1020 con 0x7f6cbc0d0860 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc39aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc0d1300 con 0x7f6cbc0cd520 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc0f45640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6cbc0d0860 0x7f6cbc0b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc0f45640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6cbc0d0860 0x7f6cbc0b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49388/0 (socket says 172.21.15.55:49388) 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc0f45640 1 -- 172.21.15.55:0/3786856733 learned_addr learned my addr 172.21.15.55:0/3786856733 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc1746640 1 --2- 172.21.15.55:0/3786856733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cbc0cd520 0x7f6cbc08c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc0f45640 1 -- 172.21.15.55:0/3786856733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cbc0cd520 msgr2=0x7f6cbc08c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc0f45640 1 --2- 172.21.15.55:0/3786856733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cbc0cd520 0x7f6cbc08c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc0f45640 1 -- 172.21.15.55:0/3786856733 >> v1:172.21.15.138:6789/0 conn(0x7f6cbc0cf270 legacy=0x7f6cbc0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.714+0000 7f6cc0f45640 1 -- 172.21.15.55:0/3786856733 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc0d15e0 con 0x7f6cbc0d0860 2024-05-20T22:09:14.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.715+0000 7f6cc0f45640 1 --2- 172.21.15.55:0/3786856733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0d0860 0x7f6cbc0b19f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009870 tx=0x7f6cac02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d055d402244fb4c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:14.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.715+0000 7f6cb3fff640 1 -- 172.21.15.55:0/3786856733 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac037070 con 0x7f6cbc0d0860 2024-05-20T22:09:14.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.715+0000 7f6cb3fff640 1 -- 172.21.15.55:0/3786856733 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cac032440 con 0x7f6cbc0d0860 2024-05-20T22:09:14.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.715+0000 7f6cb3fff640 1 -- 172.21.15.55:0/3786856733 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac032810 con 0x7f6cbc0d0860 2024-05-20T22:09:14.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.716+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3786856733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0d0860 msgr2=0x7f6cbc0b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:14.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.716+0000 7f6cc39aa640 1 --2- 172.21.15.55:0/3786856733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0d0860 0x7f6cbc0b19f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009870 tx=0x7f6cac02a1c0 comp rx=0 tx=0).stop 2024-05-20T22:09:14.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.716+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3786856733 shutdown_connections 2024-05-20T22:09:14.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.716+0000 7f6cc39aa640 1 --2- 172.21.15.55:0/3786856733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0d0860 0x7f6cbc0b19f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:14.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.716+0000 7f6cc39aa640 1 --2- 172.21.15.55:0/3786856733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cbc0cd520 0x7f6cbc08c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:14.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.716+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3786856733 >> 172.21.15.55:0/3786856733 conn(0x7f6cbc0d2860 msgr2=0x7f6cbc0d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:14.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.716+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3786856733 shutdown_connections 2024-05-20T22:09:14.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.716+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3786856733 wait complete. 2024-05-20T22:09:14.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.718+0000 7f6cc39aa640 1 Processor -- start 2024-05-20T22:09:14.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.718+0000 7f6cc39aa640 1 -- start start 2024-05-20T22:09:14.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.719+0000 7f6cc39aa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0cd520 0x7f6cbc1f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.719+0000 7f6cc39aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc0cf270 0x7f6cbc151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.719+0000 7f6cc39aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cbc0d0860 0x7f6cbc158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.719+0000 7f6cc39aa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0d0c70 con 0x7f6cbc0cd520 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.719+0000 7f6cc39aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0b2cb0 con 0x7f6cbc0cf270 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.719+0000 7f6cc39aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc15f620 con 0x7f6cbc0d0860 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.719+0000 7f6cc1746640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0cd520 0x7f6cbc1f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.719+0000 7f6cc1746640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0cd520 0x7f6cbc1f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49396/0 (socket says 172.21.15.55:49396) 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.719+0000 7f6cc1746640 1 -- 172.21.15.55:0/3559113105 learned_addr learned my addr 172.21.15.55:0/3559113105 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc0f45640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc0cf270 0x7f6cbc151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc1f47640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cbc0d0860 0x7f6cbc158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc1746640 1 -- 172.21.15.55:0/3559113105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cbc0d0860 msgr2=0x7f6cbc158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc1746640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cbc0d0860 0x7f6cbc158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:14.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc1746640 1 -- 172.21.15.55:0/3559113105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc0cf270 msgr2=0x7f6cbc151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:14.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc1746640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc0cf270 0x7f6cbc151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:14.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc1746640 1 -- 172.21.15.55:0/3559113105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc1f6b20 con 0x7f6cbc0cd520 2024-05-20T22:09:14.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc0f45640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc0cf270 0x7f6cbc151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:14.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc1746640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0cd520 0x7f6cbc1f3530 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800e8a0 tx=0x7f6cb800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:14.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cb1ffb640 1 -- 172.21.15.55:0/3559113105 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8017070 con 0x7f6cbc0cd520 2024-05-20T22:09:14.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cb1ffb640 1 -- 172.21.15.55:0/3559113105 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb800c8e0 con 0x7f6cbc0cd520 2024-05-20T22:09:14.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cb1ffb640 1 -- 172.21.15.55:0/3559113105 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800ccd0 con 0x7f6cbc0cd520 2024-05-20T22:09:14.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3559113105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc1f3d60 con 0x7f6cbc0cd520 2024-05-20T22:09:14.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.720+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3559113105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc1f4320 con 0x7f6cbc0cd520 2024-05-20T22:09:14.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.721+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3559113105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cbc069090 con 0x7f6cbc0cd520 2024-05-20T22:09:14.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.722+0000 7f6cb1ffb640 1 -- 172.21.15.55:0/3559113105 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6cb801a070 con 0x7f6cbc0cd520 2024-05-20T22:09:14.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.723+0000 7f6cb1ffb640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ca006be30 0x7f6ca006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:14.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.723+0000 7f6cb1ffb640 1 -- 172.21.15.55:0/3559113105 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7f6cb808af40 con 0x7f6cbc0cd520 2024-05-20T22:09:14.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.723+0000 7f6cc0f45640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ca006be30 0x7f6ca006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:14.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.723+0000 7f6cc0f45640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ca006be30 0x7f6ca006e340 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6cac034000 tx=0x7f6cac02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:14.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.732+0000 7f6cb1ffb640 1 -- 172.21.15.55:0/3559113105 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6cb8059690 con 0x7f6cbc0cd520 2024-05-20T22:09:14.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:14.956+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3559113105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6cbc0b11d0 con 0x7f6cbc0cd520 2024-05-20T22:09:15.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.580+0000 7f6cb1ffb640 1 -- 172.21.15.55:0/3559113105 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f6cb8058f90 con 0x7f6cbc0cd520 2024-05-20T22:09:15.583 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 5 and data pool 6 2024-05-20T22:09:15.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.592+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3559113105 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ca006be30 msgr2=0x7f6ca006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:15.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.592+0000 7f6cc39aa640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ca006be30 0x7f6ca006e340 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6cac034000 tx=0x7f6cac02e000 comp rx=0 tx=0).stop 2024-05-20T22:09:15.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.592+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3559113105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0cd520 msgr2=0x7f6cbc1f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:15.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.592+0000 7f6cc39aa640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0cd520 0x7f6cbc1f3530 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800e8a0 tx=0x7f6cb800ed60 comp rx=0 tx=0).stop 2024-05-20T22:09:15.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.592+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3559113105 shutdown_connections 2024-05-20T22:09:15.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.592+0000 7f6cc39aa640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ca006be30 0x7f6ca006e340 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:15.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.592+0000 7f6cc39aa640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6cbc0d0860 0x7f6cbc158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:15.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.592+0000 7f6cc39aa640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc0cf270 0x7f6cbc151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:15.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.592+0000 7f6cc39aa640 1 --2- 172.21.15.55:0/3559113105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cbc0cd520 0x7f6cbc1f3530 secure :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800e8a0 tx=0x7f6cb800ed60 comp rx=0 tx=0).stop 2024-05-20T22:09:15.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.592+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3559113105 >> 172.21.15.55:0/3559113105 conn(0x7f6cbc0d2860 msgr2=0x7f6cbc071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:15.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.599+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3559113105 shutdown_connections 2024-05-20T22:09:15.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.599+0000 7f6cc39aa640 1 -- 172.21.15.55:0/3559113105 wait complete. 2024-05-20T22:09:15.617 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:09:15.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.737+0000 7fd86b585640 1 Processor -- start 2024-05-20T22:09:15.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.737+0000 7fd86b585640 1 -- start start 2024-05-20T22:09:15.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.738+0000 7fd86b585640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8641611b0 0x7fd864167e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:15.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.738+0000 7fd86b585640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8640cbc60 con 0x7fd8640cb640 2024-05-20T22:09:15.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.738+0000 7fd86b585640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8640cbf40 con 0x7fd8640af980 2024-05-20T22:09:15.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.738+0000 7fd86b585640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fd8640cc220 con 0x7fd8641611b0 2024-05-20T22:09:15.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.739+0000 7fd869b22640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8641611b0 0x7fd864167e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:15.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.739+0000 7fd868b20640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd8640cb640 0x7fd864160aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54234/0 (socket says 172.21.15.55:54234) 2024-05-20T22:09:15.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.739+0000 7fd868b20640 1 -- 172.21.15.55:0/1876624286 learned_addr learned my addr 172.21.15.55:0/1876624286 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:15.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.739+0000 7fd869b22640 1 -- 172.21.15.55:0/1876624286 >> v1:172.21.15.138:6789/0 conn(0x7fd8640af980 legacy=0x7fd8640cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:15.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.739+0000 7fd869b22640 1 -- 172.21.15.55:0/1876624286 >> v1:172.21.15.55:6789/0 conn(0x7fd8640cb640 legacy=0x7fd864160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:15.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.739+0000 7fd85bfff640 1 -- 172.21.15.55:0/1876624286 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2291577444 0 0) 0x7fd8640cbc60 con 0x7fd8640cb640 2024-05-20T22:09:15.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.739+0000 7fd869b22640 1 -- 172.21.15.55:0/1876624286 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8640cc500 con 0x7fd8641611b0 2024-05-20T22:09:15.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.740+0000 7fd869b22640 1 --2- 172.21.15.55:0/1876624286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641611b0 0x7fd864167e30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009d90 tx=0x7fd85c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf166304512751b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:15.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.740+0000 7fd85bfff640 1 -- 172.21.15.55:0/1876624286 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c031600 con 0x7fd8641611b0 2024-05-20T22:09:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.740+0000 7fd85bfff640 1 -- 172.21.15.55:0/1876624286 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c0317a0 con 0x7fd8641611b0 2024-05-20T22:09:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.740+0000 7fd85bfff640 1 -- 172.21.15.55:0/1876624286 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c031b70 con 0x7fd8641611b0 2024-05-20T22:09:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.741+0000 7fd86b585640 1 -- 172.21.15.55:0/1876624286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641611b0 msgr2=0x7fd864167e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.741+0000 7fd86b585640 1 --2- 172.21.15.55:0/1876624286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641611b0 0x7fd864167e30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009d90 tx=0x7fd85c02a610 comp rx=0 tx=0).stop 2024-05-20T22:09:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.741+0000 7fd86b585640 1 -- 172.21.15.55:0/1876624286 shutdown_connections 2024-05-20T22:09:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.741+0000 7fd86b585640 1 --2- 172.21.15.55:0/1876624286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641611b0 0x7fd864167e30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.741+0000 7fd86b585640 1 -- 172.21.15.55:0/1876624286 >> 172.21.15.55:0/1876624286 conn(0x7fd8640cd4d0 msgr2=0x7fd8640cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.741+0000 7fd86b585640 1 -- 172.21.15.55:0/1876624286 shutdown_connections 2024-05-20T22:09:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.741+0000 7fd86b585640 1 -- 172.21.15.55:0/1876624286 wait complete. 2024-05-20T22:09:15.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.742+0000 7fd86b585640 1 Processor -- start 2024-05-20T22:09:15.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.743+0000 7fd86b585640 1 -- start start 2024-05-20T22:09:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd86b585640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8640af980 0x7fd8640cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd86b585640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8640cb640 0x7fd864156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd86b585640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641573e0 0x7fd864157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd86b585640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd864168370 con 0x7fd8641573e0 2024-05-20T22:09:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd86b585640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd86406cd20 con 0x7fd8640af980 2024-05-20T22:09:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd86b585640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd8641684f0 con 0x7fd8640cb640 2024-05-20T22:09:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd869b22640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641573e0 0x7fd864157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd869b22640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641573e0 0x7fd864157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49404/0 (socket says 172.21.15.55:49404) 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd869b22640 1 -- 172.21.15.55:0/1564274876 learned_addr learned my addr 172.21.15.55:0/1564274876 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd869321640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8640af980 0x7fd8640cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd868b20640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8640cb640 0x7fd864156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd869b22640 1 -- 172.21.15.55:0/1564274876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8640cb640 msgr2=0x7fd864156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd869b22640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8640cb640 0x7fd864156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd869b22640 1 -- 172.21.15.55:0/1564274876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8640af980 msgr2=0x7fd8640cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd869b22640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8640af980 0x7fd8640cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.744+0000 7fd869b22640 1 -- 172.21.15.55:0/1564274876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd864159940 con 0x7fd8641573e0 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.745+0000 7fd869b22640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641573e0 0x7fd864157ab0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd85c02ac00 tx=0x7fd85c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.745+0000 7fd859ffb640 1 -- 172.21.15.55:0/1564274876 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c040070 con 0x7fd8641573e0 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.745+0000 7fd859ffb640 1 -- 172.21.15.55:0/1564274876 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c030b60 con 0x7fd8641573e0 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.745+0000 7fd859ffb640 1 -- 172.21.15.55:0/1564274876 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c045460 con 0x7fd8641573e0 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.745+0000 7fd86b585640 1 -- 172.21.15.55:0/1564274876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd864159e40 con 0x7fd8641573e0 2024-05-20T22:09:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.745+0000 7fd86b585640 1 -- 172.21.15.55:0/1564274876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86415a400 con 0x7fd8641573e0 2024-05-20T22:09:15.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.746+0000 7fd86b585640 1 -- 172.21.15.55:0/1564274876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd864069090 con 0x7fd8641573e0 2024-05-20T22:09:15.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.747+0000 7fd859ffb640 1 -- 172.21.15.55:0/1564274876 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd85c039040 con 0x7fd8641573e0 2024-05-20T22:09:15.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.747+0000 7fd859ffb640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd84806be30 0x7fd84806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:15.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.747+0000 7fd859ffb640 1 -- 172.21.15.55:0/1564274876 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fd85c0ac730 con 0x7fd8641573e0 2024-05-20T22:09:15.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.747+0000 7fd869321640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd84806be30 0x7fd84806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:15.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.748+0000 7fd869321640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd84806be30 0x7fd84806e340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd860006fd0 tx=0x7fd860009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:15.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.752+0000 7fd859ffb640 1 -- 172.21.15.55:0/1564274876 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd85c07a930 con 0x7fd8641573e0 2024-05-20T22:09:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.936+0000 7fd86b585640 1 -- 172.21.15.55:0/1564274876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8640ceed0 con 0x7fd8641573e0 2024-05-20T22:09:15.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.937+0000 7fd859ffb640 1 -- 172.21.15.55:0/1564274876 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16955 (secure 0 0 0) 0x7fd85c07ec30 con 0x7fd8641573e0 2024-05-20T22:09:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:15.938 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":37,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:09:14.957299+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:09:12.734355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T22:09:13.936532+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:09:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 -- 172.21.15.55:0/1564274876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd84806be30 msgr2=0x7fd84806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd84806be30 0x7fd84806e340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd860006fd0 tx=0x7fd860009060 comp rx=0 tx=0).stop 2024-05-20T22:09:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 -- 172.21.15.55:0/1564274876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641573e0 msgr2=0x7fd864157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641573e0 0x7fd864157ab0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd85c02ac00 tx=0x7fd85c004a40 comp rx=0 tx=0).stop 2024-05-20T22:09:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 -- 172.21.15.55:0/1564274876 shutdown_connections 2024-05-20T22:09:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd84806be30 0x7fd84806e340 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8641573e0 0x7fd864157ab0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8640cb640 0x7fd864156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 --2- 172.21.15.55:0/1564274876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8640af980 0x7fd8640cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 -- 172.21.15.55:0/1564274876 >> 172.21.15.55:0/1564274876 conn(0x7fd8640cd4d0 msgr2=0x7fd864072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 -- 172.21.15.55:0/1564274876 shutdown_connections 2024-05-20T22:09:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:15.939+0000 7fd86b585640 1 -- 172.21.15.55:0/1564274876 wait complete. 2024-05-20T22:09:15.951 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:09:16.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.065+0000 7f392f32f640 1 Processor -- start 2024-05-20T22:09:16.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.066+0000 7f392f32f640 1 -- start start 2024-05-20T22:09:16.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392f32f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f392809a990 0x7f39280d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392f32f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 0x7f39280d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392f32f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39280ae690 0x7f39280aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392f32f640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f39280d13f0 con 0x7f392809a990 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392f32f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f39280af740 con 0x7f39280ae690 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392f32f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39280afa20 con 0x7f392809ada0 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392d0cb640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f392809a990 0x7f39280d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392d0cb640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f392809a990 0x7f39280d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49406/0 (socket says 172.21.15.55:49406) 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392d0cb640 1 -- 172.21.15.55:0/1983770268 learned_addr learned my addr 172.21.15.55:0/1983770268 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392d8cc640 1 --2- 172.21.15.55:0/1983770268 >> v2:172.21.15.138:3300/0 conn(0x7f39280ae690 0x7f39280aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.067+0000 7f392c8ca640 1 --2- 172.21.15.55:0/1983770268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 0x7f39280d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.068+0000 7f392d0cb640 1 -- 172.21.15.55:0/1983770268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 msgr2=0x7f39280d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.068+0000 7f392d0cb640 1 --2- 172.21.15.55:0/1983770268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 0x7f39280d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.068+0000 7f392d0cb640 1 -- 172.21.15.55:0/1983770268 >> v2:172.21.15.138:3300/0 conn(0x7f39280ae690 msgr2=0x7f39280aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.068+0000 7f392d0cb640 1 --2- 172.21.15.55:0/1983770268 >> v2:172.21.15.138:3300/0 conn(0x7f39280ae690 0x7f39280aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.068+0000 7f392d0cb640 1 -- 172.21.15.55:0/1983770268 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39280afd00 con 0x7f392809a990 2024-05-20T22:09:16.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.068+0000 7f392d0cb640 1 --2- 172.21.15.55:0/1983770268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 0x7f39280d0970 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f392806e420 tx=0x7f392402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c3d9c0030c96aa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f391ffff640 1 -- 172.21.15.55:0/1983770268 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924037070 con 0x7f392809a990 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f391ffff640 1 -- 172.21.15.55:0/1983770268 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3924032440 con 0x7f392809a990 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f391ffff640 1 -- 172.21.15.55:0/1983770268 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924032810 con 0x7f392809a990 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f392f32f640 1 -- 172.21.15.55:0/1983770268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 msgr2=0x7f39280d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f392f32f640 1 --2- 172.21.15.55:0/1983770268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 0x7f39280d0970 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f392806e420 tx=0x7f392402a1e0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f392f32f640 1 -- 172.21.15.55:0/1983770268 shutdown_connections 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f392f32f640 1 --2- 172.21.15.55:0/1983770268 >> v2:172.21.15.138:3300/0 conn(0x7f39280ae690 0x7f39280aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f392f32f640 1 --2- 172.21.15.55:0/1983770268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 0x7f39280d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f392f32f640 1 --2- 172.21.15.55:0/1983770268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 0x7f39280d0970 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f392f32f640 1 -- 172.21.15.55:0/1983770268 >> 172.21.15.55:0/1983770268 conn(0x7f39280b6540 msgr2=0x7f39280b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f392f32f640 1 -- 172.21.15.55:0/1983770268 shutdown_connections 2024-05-20T22:09:16.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.069+0000 7f392f32f640 1 -- 172.21.15.55:0/1983770268 wait complete. 2024-05-20T22:09:16.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.071+0000 7f392f32f640 1 Processor -- start 2024-05-20T22:09:16.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.071+0000 7f392f32f640 1 -- start start 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.072+0000 7f392f32f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 0x7f39281522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.072+0000 7f392f32f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 0x7f3928157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392f32f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39280ae690 0x7f39281582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392f32f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f392806e5b0 con 0x7f392809a990 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392f32f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f392815f640 con 0x7f39280ae690 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392f32f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f392806e730 con 0x7f392809ada0 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d0cb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 0x7f39281522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d0cb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 0x7f39281522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49422/0 (socket says 172.21.15.55:49422) 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d0cb640 1 -- 172.21.15.55:0/2007022978 learned_addr learned my addr 172.21.15.55:0/2007022978 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d8cc640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39280ae690 0x7f39281582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392c8ca640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 0x7f3928157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d0cb640 1 -- 172.21.15.55:0/2007022978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 msgr2=0x7f3928157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:16.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d0cb640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 0x7f3928157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d0cb640 1 -- 172.21.15.55:0/2007022978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39280ae690 msgr2=0x7f39281582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d0cb640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39280ae690 0x7f39281582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d0cb640 1 -- 172.21.15.55:0/2007022978 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3928153380 con 0x7f392809a990 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392c8ca640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 0x7f3928157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d8cc640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39280ae690 0x7f39281582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392d0cb640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 0x7f39281522c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f392402a220 tx=0x7f39240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f391dffb640 1 -- 172.21.15.55:0/2007022978 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924041020 con 0x7f392809a990 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f391dffb640 1 -- 172.21.15.55:0/2007022978 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f392403d620 con 0x7f392809a990 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f392f32f640 1 -- 172.21.15.55:0/2007022978 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392815c9d0 con 0x7f392809a990 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.073+0000 7f391dffb640 1 -- 172.21.15.55:0/2007022978 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924037040 con 0x7f392809a990 2024-05-20T22:09:16.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.074+0000 7f392f32f640 1 -- 172.21.15.55:0/2007022978 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3928158d40 con 0x7f392809a990 2024-05-20T22:09:16.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.075+0000 7f392f32f640 1 -- 172.21.15.55:0/2007022978 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3928069090 con 0x7f392809a990 2024-05-20T22:09:16.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.075+0000 7f391dffb640 1 -- 172.21.15.55:0/2007022978 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3924044020 con 0x7f392809a990 2024-05-20T22:09:16.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.076+0000 7f391dffb640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f390c06be30 0x7f390c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.076+0000 7f391dffb640 1 -- 172.21.15.55:0/2007022978 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f39240ab800 con 0x7f392809a990 2024-05-20T22:09:16.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.076+0000 7f392c8ca640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f390c06be30 0x7f390c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.076+0000 7f392c8ca640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f390c06be30 0x7f390c06e340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3928155e50 tx=0x7f3918008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:16.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.080+0000 7f391dffb640 1 -- 172.21.15.55:0/2007022978 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f392404b030 con 0x7f392809a990 2024-05-20T22:09:16.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.264+0000 7f392f32f640 1 -- 172.21.15.55:0/2007022978 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f392806e230 con 0x7f392809a990 2024-05-20T22:09:16.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.265+0000 7f391dffb640 1 -- 172.21.15.55:0/2007022978 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16955 (secure 0 0 0) 0x7f3924079ef0 con 0x7f392809a990 2024-05-20T22:09:16.267 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:16.267 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":37,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:09:14.957299+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:09:12.734355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T22:09:13.936532+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:09:16.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 -- 172.21.15.55:0/2007022978 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f390c06be30 msgr2=0x7f390c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:16.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f390c06be30 0x7f390c06e340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3928155e50 tx=0x7f3918008040 comp rx=0 tx=0).stop 2024-05-20T22:09:16.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 -- 172.21.15.55:0/2007022978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 msgr2=0x7f39281522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:16.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 0x7f39281522c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f392402a220 tx=0x7f39240046c0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 -- 172.21.15.55:0/2007022978 shutdown_connections 2024-05-20T22:09:16.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f390c06be30 0x7f390c06e340 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39280ae690 0x7f39281582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f392809ada0 0x7f3928157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 --2- 172.21.15.55:0/2007022978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392809a990 0x7f39281522c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 -- 172.21.15.55:0/2007022978 >> 172.21.15.55:0/2007022978 conn(0x7f39280b6540 msgr2=0x7f392806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:16.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 -- 172.21.15.55:0/2007022978 shutdown_connections 2024-05-20T22:09:16.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.268+0000 7f392f32f640 1 -- 172.21.15.55:0/2007022978 wait complete. 2024-05-20T22:09:16.280 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T22:09:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.397+0000 7f0c89698640 1 Processor -- start 2024-05-20T22:09:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.397+0000 7f0c89698640 1 -- start start 2024-05-20T22:09:16.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.398+0000 7f0c89698640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c840b1eb0 0x7f0c8406d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.398+0000 7f0c89698640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c840b37b0 con 0x7f0c840b2470 2024-05-20T22:09:16.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.398+0000 7f0c89698640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c8406d8b0 con 0x7f0c840cfad0 2024-05-20T22:09:16.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.398+0000 7f0c89698640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f0c840b2880 con 0x7f0c840b1eb0 2024-05-20T22:09:16.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.399+0000 7f0c82ffd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0c840b2470 0x7f0c840cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54244/0 (socket says 172.21.15.55:54244) 2024-05-20T22:09:16.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.399+0000 7f0c82ffd640 1 -- 172.21.15.55:0/2486544171 learned_addr learned my addr 172.21.15.55:0/2486544171 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:16.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.399+0000 7f0c837fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c840b1eb0 0x7f0c8406d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.399+0000 7f0c837fe640 1 -- 172.21.15.55:0/2486544171 >> v1:172.21.15.138:6789/0 conn(0x7f0c840cfad0 legacy=0x7f0c840d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:16.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.399+0000 7f0c837fe640 1 -- 172.21.15.55:0/2486544171 >> v1:172.21.15.55:6789/0 conn(0x7f0c840b2470 legacy=0x7f0c840cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:16.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.399+0000 7f0c827fc640 1 -- 172.21.15.55:0/2486544171 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3747024785 0 0) 0x7f0c74001110 con 0x7f0c840b2470 2024-05-20T22:09:16.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.399+0000 7f0c837fe640 1 -- 172.21.15.55:0/2486544171 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8406db90 con 0x7f0c840b1eb0 2024-05-20T22:09:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.400+0000 7f0c837fe640 1 --2- 172.21.15.55:0/2486544171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c840b1eb0 0x7f0c8406d0b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009d90 tx=0x7f0c7c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518f9f94122f3db server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.400+0000 7f0c827fc640 1 -- 172.21.15.55:0/2486544171 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c007740 con 0x7f0c840b1eb0 2024-05-20T22:09:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.400+0000 7f0c827fc640 1 -- 172.21.15.55:0/2486544171 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7c0078e0 con 0x7f0c840b1eb0 2024-05-20T22:09:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.400+0000 7f0c827fc640 1 -- 172.21.15.55:0/2486544171 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c007cb0 con 0x7f0c840b1eb0 2024-05-20T22:09:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.401+0000 7f0c89698640 1 -- 172.21.15.55:0/2486544171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c840b1eb0 msgr2=0x7f0c8406d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.401+0000 7f0c89698640 1 --2- 172.21.15.55:0/2486544171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c840b1eb0 0x7f0c8406d0b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009d90 tx=0x7f0c7c02a6f0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.401+0000 7f0c89698640 1 -- 172.21.15.55:0/2486544171 shutdown_connections 2024-05-20T22:09:16.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.401+0000 7f0c89698640 1 --2- 172.21.15.55:0/2486544171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c840b1eb0 0x7f0c8406d0b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.401+0000 7f0c89698640 1 -- 172.21.15.55:0/2486544171 >> 172.21.15.55:0/2486544171 conn(0x7f0c8409be10 msgr2=0x7f0c840d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:16.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.401+0000 7f0c89698640 1 -- 172.21.15.55:0/2486544171 shutdown_connections 2024-05-20T22:09:16.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.401+0000 7f0c89698640 1 -- 172.21.15.55:0/2486544171 wait complete. 2024-05-20T22:09:16.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.403+0000 7f0c89698640 1 Processor -- start 2024-05-20T22:09:16.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.403+0000 7f0c89698640 1 -- start start 2024-05-20T22:09:16.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c89698640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c840b1eb0 0x7f0c8415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c89698640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c840b2470 0x7f0c840b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c89698640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8415e9b0 0x7f0c840b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c89698640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0c840b3a30 con 0x7f0c8415e9b0 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c89698640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c840b3f50 con 0x7f0c840b2470 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c89698640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0c841628e0 con 0x7f0c840b1eb0 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c837fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8415e9b0 0x7f0c840b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c837fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8415e9b0 0x7f0c840b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49440/0 (socket says 172.21.15.55:49440) 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c837fe640 1 -- 172.21.15.55:0/2558511047 learned_addr learned my addr 172.21.15.55:0/2558511047 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c7bfff640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c840b2470 0x7f0c840b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c82ffd640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c840b1eb0 0x7f0c8415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c837fe640 1 -- 172.21.15.55:0/2558511047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c840b1eb0 msgr2=0x7f0c8415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c837fe640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c840b1eb0 0x7f0c8415e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c837fe640 1 -- 172.21.15.55:0/2558511047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c840b2470 msgr2=0x7f0c840b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c837fe640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c840b2470 0x7f0c840b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c837fe640 1 -- 172.21.15.55:0/2558511047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c841f8830 con 0x7f0c8415e9b0 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.404+0000 7f0c7bfff640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c840b2470 0x7f0c840b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:16.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.405+0000 7f0c837fe640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8415e9b0 0x7f0c840b7800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009d90 tx=0x7f0c7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.405+0000 7f0c7b7fe640 1 -- 172.21.15.55:0/2558511047 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c040070 con 0x7f0c8415e9b0 2024-05-20T22:09:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.405+0000 7f0c7b7fe640 1 -- 172.21.15.55:0/2558511047 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7c031c90 con 0x7f0c8415e9b0 2024-05-20T22:09:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.405+0000 7f0c89698640 1 -- 172.21.15.55:0/2558511047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c841f8b10 con 0x7f0c8415e9b0 2024-05-20T22:09:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.405+0000 7f0c7b7fe640 1 -- 172.21.15.55:0/2558511047 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c03c640 con 0x7f0c8415e9b0 2024-05-20T22:09:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.405+0000 7f0c89698640 1 -- 172.21.15.55:0/2558511047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c841f90d0 con 0x7f0c8415e9b0 2024-05-20T22:09:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.406+0000 7f0c89698640 1 -- 172.21.15.55:0/2558511047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c84161ae0 con 0x7f0c8415e9b0 2024-05-20T22:09:16.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.406+0000 7f0c7b7fe640 1 -- 172.21.15.55:0/2558511047 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c7c039040 con 0x7f0c8415e9b0 2024-05-20T22:09:16.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.407+0000 7f0c7b7fe640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0c6406be30 0x7f0c6406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:16.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.407+0000 7f0c7b7fe640 1 -- 172.21.15.55:0/2558511047 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f0c7c0ac6c0 con 0x7f0c8415e9b0 2024-05-20T22:09:16.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.407+0000 7f0c82ffd640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0c6406be30 0x7f0c6406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:16.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.407+0000 7f0c82ffd640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0c6406be30 0x7f0c6406e340 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0c74000f60 tx=0x7f0c74009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:16.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.411+0000 7f0c7b7fe640 1 -- 172.21.15.55:0/2558511047 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0c7c04b340 con 0x7f0c8415e9b0 2024-05-20T22:09:16.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:16.636+0000 7f0c89698640 1 -- 172.21.15.55:0/2558511047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0c84069090 con 0x7f0c8415e9b0 2024-05-20T22:09:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.596+0000 7f0c7b7fe640 1 -- 172.21.15.55:0/2558511047 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f0c7c07af10 con 0x7f0c8415e9b0 2024-05-20T22:09:17.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 -- 172.21.15.55:0/2558511047 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0c6406be30 msgr2=0x7f0c6406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:17.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0c6406be30 0x7f0c6406e340 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0c74000f60 tx=0x7f0c74009040 comp rx=0 tx=0).stop 2024-05-20T22:09:17.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 -- 172.21.15.55:0/2558511047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8415e9b0 msgr2=0x7f0c840b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:17.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8415e9b0 0x7f0c840b7800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009d90 tx=0x7f0c7c004a40 comp rx=0 tx=0).stop 2024-05-20T22:09:17.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 -- 172.21.15.55:0/2558511047 shutdown_connections 2024-05-20T22:09:17.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0c6406be30 0x7f0c6406e340 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:17.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8415e9b0 0x7f0c840b7800 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:17.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c840b2470 0x7f0c840b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:17.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 --2- 172.21.15.55:0/2558511047 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c840b1eb0 0x7f0c8415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:17.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 -- 172.21.15.55:0/2558511047 >> 172.21.15.55:0/2558511047 conn(0x7f0c8409be10 msgr2=0x7f0c84070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:17.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 -- 172.21.15.55:0/2558511047 shutdown_connections 2024-05-20T22:09:17.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.599+0000 7f0c89698640 1 -- 172.21.15.55:0/2558511047 wait complete. 2024-05-20T22:09:17.614 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-20T22:09:17.614 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T22:09:17.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.737+0000 7fe5ff1a8640 1 Processor -- start 2024-05-20T22:09:17.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.737+0000 7fe5ff1a8640 1 -- start start 2024-05-20T22:09:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5ff1a8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5f815f150 0x7fe5f80cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5ff1a8640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe5f806b500 0x7fe5f806b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5ff1a8640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f80d3780 con 0x7fe5f80d32a0 2024-05-20T22:09:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5ff1a8640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fe5f80d3a60 con 0x7fe5f806b500 2024-05-20T22:09:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5ff1a8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f80d3d40 con 0x7fe5f815f150 2024-05-20T22:09:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5fcf44640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe5f80d32a0 0x7fe5f80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54246/0 (socket says 172.21.15.55:54246) 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5fcf44640 1 -- 172.21.15.55:0/722007225 learned_addr learned my addr 172.21.15.55:0/722007225 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5fd745640 1 --2- 172.21.15.55:0/722007225 >> v2:172.21.15.55:3300/0 conn(0x7fe5f806b500 0x7fe5f806b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5f77fe640 1 -- 172.21.15.55:0/722007225 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3724630438 0 0) 0x7fe5f80d3780 con 0x7fe5f80d32a0 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5f77fe640 1 -- 172.21.15.55:0/722007225 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5e4003520 con 0x7fe5f80d32a0 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5f7fff640 1 --2- 172.21.15.55:0/722007225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5f815f150 0x7fe5f80cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5fd745640 1 -- 172.21.15.55:0/722007225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5f815f150 msgr2=0x7fe5f80cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5fd745640 1 --2- 172.21.15.55:0/722007225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5f815f150 0x7fe5f80cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5fd745640 1 -- 172.21.15.55:0/722007225 >> v1:172.21.15.55:6789/0 conn(0x7fe5f80d32a0 legacy=0x7fe5f80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.739+0000 7fe5fd745640 1 -- 172.21.15.55:0/722007225 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f80d4020 con 0x7fe5f806b500 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.740+0000 7fe5fd745640 1 --2- 172.21.15.55:0/722007225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f806b500 0x7fe5f806b910 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00097a0 tx=0x7fe5e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25deb39fe84b34bf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.740+0000 7fe5f77fe640 1 -- 172.21.15.55:0/722007225 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0037070 con 0x7fe5f806b500 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.740+0000 7fe5f77fe640 1 -- 172.21.15.55:0/722007225 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e0032440 con 0x7fe5f806b500 2024-05-20T22:09:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.740+0000 7fe5f77fe640 1 -- 172.21.15.55:0/722007225 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0032810 con 0x7fe5f806b500 2024-05-20T22:09:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.741+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/722007225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f806b500 msgr2=0x7fe5f806b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.741+0000 7fe5ff1a8640 1 --2- 172.21.15.55:0/722007225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f806b500 0x7fe5f806b910 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00097a0 tx=0x7fe5e002a180 comp rx=0 tx=0).stop 2024-05-20T22:09:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.741+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/722007225 shutdown_connections 2024-05-20T22:09:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.741+0000 7fe5ff1a8640 1 --2- 172.21.15.55:0/722007225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5f815f150 0x7fe5f80cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.741+0000 7fe5ff1a8640 1 --2- 172.21.15.55:0/722007225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f806b500 0x7fe5f806b910 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.741+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/722007225 >> 172.21.15.55:0/722007225 conn(0x7fe5f80b6de0 msgr2=0x7fe5f80d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.741+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/722007225 shutdown_connections 2024-05-20T22:09:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.741+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/722007225 wait complete. 2024-05-20T22:09:17.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.743+0000 7fe5ff1a8640 1 Processor -- start 2024-05-20T22:09:17.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.743+0000 7fe5ff1a8640 1 -- start start 2024-05-20T22:09:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.744+0000 7fe5ff1a8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5f806b500 0x7fe5f81509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.744+0000 7fe5ff1a8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f80d32a0 0x7fe5f81616d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.744+0000 7fe5ff1a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5f815f150 0x7fe5f8161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.744+0000 7fe5ff1a8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f80d4820 con 0x7fe5f80d32a0 2024-05-20T22:09:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.744+0000 7fe5ff1a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f815f560 con 0x7fe5f815f150 2024-05-20T22:09:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.744+0000 7fe5ff1a8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f80ccf20 con 0x7fe5f806b500 2024-05-20T22:09:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.744+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f80d32a0 0x7fe5f81616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.744+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f80d32a0 0x7fe5f81616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49450/0 (socket says 172.21.15.55:49450) 2024-05-20T22:09:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.744+0000 7fe5f7fff640 1 -- 172.21.15.55:0/3353528300 learned_addr learned my addr 172.21.15.55:0/3353528300 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5fcf44640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5f806b500 0x7fe5f81509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5fd745640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5f815f150 0x7fe5f8161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5f7fff640 1 -- 172.21.15.55:0/3353528300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5f806b500 msgr2=0x7fe5f81509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5f7fff640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5f806b500 0x7fe5f81509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5f7fff640 1 -- 172.21.15.55:0/3353528300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5f815f150 msgr2=0x7fe5f8161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5f7fff640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5f815f150 0x7fe5f8161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5f7fff640 1 -- 172.21.15.55:0/3353528300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f80ca6f0 con 0x7fe5f80d32a0 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5f7fff640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f80d32a0 0x7fe5f81616d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800ccf0 tx=0x7fe5e800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5f57fa640 1 -- 172.21.15.55:0/3353528300 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8003990 con 0x7fe5f80d32a0 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5f57fa640 1 -- 172.21.15.55:0/3353528300 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e8003b30 con 0x7fe5f80d32a0 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/3353528300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f80ca9d0 con 0x7fe5f80d32a0 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5f57fa640 1 -- 172.21.15.55:0/3353528300 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8004020 con 0x7fe5f80d32a0 2024-05-20T22:09:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.745+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/3353528300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f80caf90 con 0x7fe5f80d32a0 2024-05-20T22:09:17.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.746+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/3353528300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f8069090 con 0x7fe5f80d32a0 2024-05-20T22:09:17.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.747+0000 7fe5f57fa640 1 -- 172.21.15.55:0/3353528300 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe5e80041c0 con 0x7fe5f80d32a0 2024-05-20T22:09:17.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.747+0000 7fe5f57fa640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe5d806be30 0x7fe5d806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:17.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.747+0000 7fe5f57fa640 1 -- 172.21.15.55:0/3353528300 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe5e808b4a0 con 0x7fe5f80d32a0 2024-05-20T22:09:17.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.747+0000 7fe5fcf44640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe5d806be30 0x7fe5d806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:17.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.748+0000 7fe5fcf44640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe5d806be30 0x7fe5d806e340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0005880 tx=0x7fe5f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:17.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.752+0000 7fe5f57fa640 1 -- 172.21.15.55:0/3353528300 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe5e8014030 con 0x7fe5f80d32a0 2024-05-20T22:09:17.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:17.974+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/3353528300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe5f80cec80 con 0x7fe5f80d32a0 2024-05-20T22:09:18.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.605+0000 7fe5f57fa640 1 -- 172.21.15.55:0/3353528300 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fe5e8059d50 con 0x7fe5f80d32a0 2024-05-20T22:09:18.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/3353528300 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe5d806be30 msgr2=0x7fe5d806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:18.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe5d806be30 0x7fe5d806e340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0005880 tx=0x7fe5f0009040 comp rx=0 tx=0).stop 2024-05-20T22:09:18.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/3353528300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f80d32a0 msgr2=0x7fe5f81616d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:18.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f80d32a0 0x7fe5f81616d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800ccf0 tx=0x7fe5e800a5c0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/3353528300 shutdown_connections 2024-05-20T22:09:18.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe5d806be30 0x7fe5d806e340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5f815f150 0x7fe5f8161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5f80d32a0 0x7fe5f81616d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 --2- 172.21.15.55:0/3353528300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe5f806b500 0x7fe5f81509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/3353528300 >> 172.21.15.55:0/3353528300 conn(0x7fe5f80b6de0 msgr2=0x7fe5f806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:18.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/3353528300 shutdown_connections 2024-05-20T22:09:18.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.608+0000 7fe5ff1a8640 1 -- 172.21.15.55:0/3353528300 wait complete. 2024-05-20T22:09:18.623 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T22:09:18.623 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:09:18.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.745+0000 7f4da4d38640 1 Processor -- start 2024-05-20T22:09:18.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.745+0000 7f4da4d38640 1 -- start start 2024-05-20T22:09:18.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.746+0000 7f4da4d38640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4da00ce030 0x7f4da00ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:18.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.746+0000 7f4da4d38640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da00b7c90 con 0x7f4da00b8140 2024-05-20T22:09:18.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.746+0000 7f4da4d38640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da00d1740 con 0x7f4da00cd510 2024-05-20T22:09:18.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.746+0000 7f4da4d38640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4da009bc10 con 0x7f4da00ce030 2024-05-20T22:09:18.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.746+0000 7f4d9e59c640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4da00b8140 0x7f4da00d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49608/0 (socket says 172.21.15.55:49608) 2024-05-20T22:09:18.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.746+0000 7f4d9e59c640 1 -- 172.21.15.55:0/2873988304 learned_addr learned my addr 172.21.15.55:0/2873988304 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:18.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.746+0000 7f4d9ed9d640 1 --2- 172.21.15.55:0/2873988304 >> v2:172.21.15.138:3300/0 conn(0x7f4da00ce030 0x7f4da00ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:18.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.746+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3409248160 0 0) 0x7f4da00b7c90 con 0x7f4da00b8140 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.746+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d8c003520 con 0x7f4da00b8140 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 76145236 0 0) 0x7f4d8c003520 con 0x7f4da00b8140 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 >> v2:172.21.15.138:3300/0 conn(0x7f4da00ce030 msgr2=0x7f4da00ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 --2- 172.21.15.55:0/2873988304 >> v2:172.21.15.138:3300/0 conn(0x7f4da00ce030 0x7f4da00ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 >> v1:172.21.15.138:6789/0 conn(0x7f4da00cd510 legacy=0x7f4da00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da009c130 con 0x7f4da00b8140 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f4d900017f0 con 0x7f4da00b8140 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 >> v1:172.21.15.55:6789/0 conn(0x7f4da00b8140 legacy=0x7f4da00d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d8c0039b0 0x7f4d8c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d8c004800 0x7f4d8c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d8c0051c0 0x7f4d8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4da009bc10 con 0x7f4d8c0051c0 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c00c160 con 0x7f4d8c004800 2024-05-20T22:09:18.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8c00c360 con 0x7f4d8c0039b0 2024-05-20T22:09:18.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9ed9d640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d8c0051c0 0x7f4d8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:18.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d97fff640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d8c004800 0x7f4d8c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:18.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9e59c640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d8c0039b0 0x7f4d8c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:18.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9ed9d640 1 -- 172.21.15.55:0/2873988304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d8c0039b0 msgr2=0x7f4d8c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:18.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9ed9d640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d8c0039b0 0x7f4d8c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9ed9d640 1 -- 172.21.15.55:0/2873988304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d8c004800 msgr2=0x7f4d8c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:18.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9ed9d640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d8c004800 0x7f4d8c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.747+0000 7f4d9ed9d640 1 -- 172.21.15.55:0/2873988304 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c003520 con 0x7f4d8c0051c0 2024-05-20T22:09:18.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.748+0000 7f4d9ed9d640 1 -- 172.21.15.55:0/2873988304 reap_dead start 2024-05-20T22:09:18.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.748+0000 7f4d9ed9d640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d8c0051c0 0x7f4d8c00b9c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009f90 tx=0x7f4d8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bcd0cb357004c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:18.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.748+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98009c10 con 0x7f4d8c0051c0 2024-05-20T22:09:18.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.748+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d98003b90 con 0x7f4d8c0051c0 2024-05-20T22:09:18.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.748+0000 7f4d9dd9b640 1 -- 172.21.15.55:0/2873988304 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98004020 con 0x7f4d8c0051c0 2024-05-20T22:09:18.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.749+0000 7f4da4d38640 1 -- 172.21.15.55:0/2873988304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d8c0051c0 msgr2=0x7f4d8c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:18.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.749+0000 7f4da4d38640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d8c0051c0 0x7f4d8c00b9c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009f90 tx=0x7f4d8c005830 comp rx=0 tx=0).stop 2024-05-20T22:09:18.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.749+0000 7f4da4d38640 1 -- 172.21.15.55:0/2873988304 shutdown_connections 2024-05-20T22:09:18.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.749+0000 7f4da4d38640 1 --2- 172.21.15.55:0/2873988304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d8c0051c0 0x7f4d8c00b9c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.749+0000 7f4da4d38640 1 -- 172.21.15.55:0/2873988304 >> 172.21.15.55:0/2873988304 conn(0x7f4da00b4340 msgr2=0x7f4da00b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:18.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.749+0000 7f4da4d38640 1 -- 172.21.15.55:0/2873988304 shutdown_connections 2024-05-20T22:09:18.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.749+0000 7f4da4d38640 1 -- 172.21.15.55:0/2873988304 wait complete. 2024-05-20T22:09:18.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.751+0000 7f4da4d38640 1 Processor -- start 2024-05-20T22:09:18.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.751+0000 7f4da4d38640 1 -- start start 2024-05-20T22:09:18.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.752+0000 7f4da4d38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d8c0051c0 0x7f4da01e2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:18.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.752+0000 7f4da4d38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da009ceb0 0x7f4da006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:18.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.752+0000 7f4da4d38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4da01f5320 0x7f4da009fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:18.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.752+0000 7f4da4d38640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4da00b8140 con 0x7f4da009ceb0 2024-05-20T22:09:18.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.752+0000 7f4da4d38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4da00d0730 con 0x7f4d8c0051c0 2024-05-20T22:09:18.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.752+0000 7f4da4d38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4da00b82c0 con 0x7f4da01f5320 2024-05-20T22:09:18.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d97fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da009ceb0 0x7f4da006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:18.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d97fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da009ceb0 0x7f4da006cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43164/0 (socket says 172.21.15.55:43164) 2024-05-20T22:09:18.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d97fff640 1 -- 172.21.15.55:0/299444792 learned_addr learned my addr 172.21.15.55:0/299444792 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:18.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d97fff640 1 -- 172.21.15.55:0/299444792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4da01f5320 msgr2=0x7f4da009fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d9ed9d640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4da01f5320 0x7f4da009fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d9e59c640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d8c0051c0 0x7f4da01e2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d97fff640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4da01f5320 0x7f4da009fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d97fff640 1 -- 172.21.15.55:0/299444792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d8c0051c0 msgr2=0x7f4da01e2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d97fff640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d8c0051c0 0x7f4da01e2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d97fff640 1 -- 172.21.15.55:0/299444792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da009ead0 con 0x7f4da009ceb0 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d97fff640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da009ceb0 0x7f4da006cea0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800f5c0 tx=0x7f4d8800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d96ffd640 1 -- 172.21.15.55:0/299444792 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d880052e0 con 0x7f4da009ceb0 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d96ffd640 1 -- 172.21.15.55:0/299444792 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d88018070 con 0x7f4da009ceb0 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4da4d38640 1 -- 172.21.15.55:0/299444792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da009edb0 con 0x7f4da009ceb0 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4d96ffd640 1 -- 172.21.15.55:0/299444792 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88003830 con 0x7f4da009ceb0 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.753+0000 7f4da4d38640 1 -- 172.21.15.55:0/299444792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da009f370 con 0x7f4da009ceb0 2024-05-20T22:09:18.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.755+0000 7f4da4d38640 1 -- 172.21.15.55:0/299444792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da01f5850 con 0x7f4da009ceb0 2024-05-20T22:09:18.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.755+0000 7f4d96ffd640 1 -- 172.21.15.55:0/299444792 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d880039d0 con 0x7f4da009ceb0 2024-05-20T22:09:18.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.756+0000 7f4d96ffd640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d8006be30 0x7f4d8006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:18.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.756+0000 7f4d9e59c640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d8006be30 0x7f4d8006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:18.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.756+0000 7f4d96ffd640 1 -- 172.21.15.55:0/299444792 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4d8808ae10 con 0x7f4da009ceb0 2024-05-20T22:09:18.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.756+0000 7f4d9e59c640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d8006be30 0x7f4d8006e340 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4d90000ef0 tx=0x7f4d90009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:18.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.760+0000 7f4d96ffd640 1 -- 172.21.15.55:0/299444792 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4da01f5850 con 0x7f4da009ceb0 2024-05-20T22:09:18.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.980+0000 7f4da4d38640 1 -- 172.21.15.55:0/299444792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4da0069090 con 0x7f4da009ceb0 2024-05-20T22:09:18.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.981+0000 7f4d96ffd640 1 -- 172.21.15.55:0/299444792 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f4d8808e030 con 0x7f4da009ceb0 2024-05-20T22:09:18.982 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:18.982 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/1557521986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1557521986},{"type":"v1","addr":"172.21.15.138:6834","nonce":1557521986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3642520472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3642520472},{"type":"v1","addr":"172.21.15.55:6836","nonce":3642520472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":34976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/2179562051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":2179562051},{"type":"v1","addr":"172.21.15.55:6837","nonce":2179562051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:09:14.957632+0000","modified":"2024-05-20T22:09:17.975726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34979},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34979":{"gid":34979,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/4137968836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":4137968836},{"type":"v1","addr":"172.21.15.138:6835","nonce":4137968836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34979,"qdb_cluster":[34979]},"id":2}]} 2024-05-20T22:09:18.982 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 18 2024-05-20T22:09:18.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 -- 172.21.15.55:0/299444792 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d8006be30 msgr2=0x7f4d8006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:18.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d8006be30 0x7f4d8006e340 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4d90000ef0 tx=0x7f4d90009040 comp rx=0 tx=0).stop 2024-05-20T22:09:18.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 -- 172.21.15.55:0/299444792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da009ceb0 msgr2=0x7f4da006cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:18.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da009ceb0 0x7f4da006cea0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800f5c0 tx=0x7f4d8800fa80 comp rx=0 tx=0).stop 2024-05-20T22:09:18.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 -- 172.21.15.55:0/299444792 shutdown_connections 2024-05-20T22:09:18.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d8006be30 0x7f4d8006e340 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4da01f5320 0x7f4da009fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da009ceb0 0x7f4da006cea0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 --2- 172.21.15.55:0/299444792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d8c0051c0 0x7f4da01e2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:18.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 -- 172.21.15.55:0/299444792 >> 172.21.15.55:0/299444792 conn(0x7f4da00b4340 msgr2=0x7f4da0067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:18.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 -- 172.21.15.55:0/299444792 shutdown_connections 2024-05-20T22:09:18.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:18.983+0000 7f4da4d38640 1 -- 172.21.15.55:0/299444792 wait complete. 2024-05-20T22:09:18.996 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:09:19.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.110+0000 7f5065b77640 1 Processor -- start 2024-05-20T22:09:19.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.110+0000 7f5065b77640 1 -- start start 2024-05-20T22:09:19.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f5065b77640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50600b8090 0x7f5060162b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f5065b77640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f50600b3b50 0x7f50601630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f5065b77640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50600d3ff0 con 0x7f506015f140 2024-05-20T22:09:19.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f5065b77640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f50600b43a0 con 0x7f50600b3b50 2024-05-20T22:09:19.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f5065b77640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f506006ce90 con 0x7f50600b8090 2024-05-20T22:09:19.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f505effd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f50600b3b50 0x7f50601630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f505ffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f506015f140 0x7f50600cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49614/0 (socket says 172.21.15.55:49614) 2024-05-20T22:09:19.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f505ffff640 1 -- 172.21.15.55:0/2253043557 learned_addr learned my addr 172.21.15.55:0/2253043557 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:19.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f505effd640 1 -- 172.21.15.55:0/2253043557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50600b8090 msgr2=0x7f5060162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:19.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f505effd640 1 --2- 172.21.15.55:0/2253043557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50600b8090 0x7f5060162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f505effd640 1 -- 172.21.15.55:0/2253043557 >> v1:172.21.15.55:6789/0 conn(0x7f506015f140 legacy=0x7f50600cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f505e7fc640 1 -- 172.21.15.55:0/2253043557 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860357168 0 0) 0x7f50600d3ff0 con 0x7f506015f140 2024-05-20T22:09:19.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.112+0000 7f505effd640 1 -- 172.21.15.55:0/2253043557 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50600d42d0 con 0x7f50600b3b50 2024-05-20T22:09:19.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.113+0000 7f505effd640 1 --2- 172.21.15.55:0/2253043557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b3b50 0x7f50601630b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f50540097a0 tx=0x7f505402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cd511775da3445 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:19.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.113+0000 7f505e7fc640 1 -- 172.21.15.55:0/2253043557 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054037070 con 0x7f50600b3b50 2024-05-20T22:09:19.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.113+0000 7f505e7fc640 1 -- 172.21.15.55:0/2253043557 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5054032440 con 0x7f50600b3b50 2024-05-20T22:09:19.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.113+0000 7f505e7fc640 1 -- 172.21.15.55:0/2253043557 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054032810 con 0x7f50600b3b50 2024-05-20T22:09:19.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.114+0000 7f5065b77640 1 -- 172.21.15.55:0/2253043557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b3b50 msgr2=0x7f50601630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.114+0000 7f5065b77640 1 --2- 172.21.15.55:0/2253043557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b3b50 0x7f50601630b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f50540097a0 tx=0x7f505402a180 comp rx=0 tx=0).stop 2024-05-20T22:09:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.114+0000 7f5065b77640 1 -- 172.21.15.55:0/2253043557 shutdown_connections 2024-05-20T22:09:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.114+0000 7f5065b77640 1 --2- 172.21.15.55:0/2253043557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b3b50 0x7f50601630b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.114+0000 7f5065b77640 1 --2- 172.21.15.55:0/2253043557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50600b8090 0x7f5060162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.114+0000 7f5065b77640 1 -- 172.21.15.55:0/2253043557 >> 172.21.15.55:0/2253043557 conn(0x7f50600d1bb0 msgr2=0x7f50600d4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.114+0000 7f5065b77640 1 -- 172.21.15.55:0/2253043557 shutdown_connections 2024-05-20T22:09:19.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.114+0000 7f5065b77640 1 -- 172.21.15.55:0/2253043557 wait complete. 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.116+0000 7f5065b77640 1 Processor -- start 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.116+0000 7f5065b77640 1 -- start start 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f5065b77640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50600b3b50 0x7f50600c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f5065b77640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b8090 0x7f5060159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f5065b77640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f506015f140 0x7f50601ea640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f5065b77640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f50601635f0 con 0x7f50600b8090 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f5065b77640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f506006b960 con 0x7f50600b3b50 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f5065b77640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5060163770 con 0x7f506015f140 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f505effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b8090 0x7f5060159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f505effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b8090 0x7f5060159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43184/0 (socket says 172.21.15.55:43184) 2024-05-20T22:09:19.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f505effd640 1 -- 172.21.15.55:0/3831126767 learned_addr learned my addr 172.21.15.55:0/3831126767 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f505effd640 1 -- 172.21.15.55:0/3831126767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f506015f140 msgr2=0x7f50601ea640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f505effd640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f506015f140 0x7f50601ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f505effd640 1 -- 172.21.15.55:0/3831126767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50600b3b50 msgr2=0x7f50600c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f505effd640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50600b3b50 0x7f50600c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.117+0000 7f505effd640 1 -- 172.21.15.55:0/3831126767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50600c89d0 con 0x7f50600b8090 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.118+0000 7f505effd640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b8090 0x7f5060159a30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f50540097a0 tx=0x7f50540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.118+0000 7f5047fff640 1 -- 172.21.15.55:0/3831126767 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054041070 con 0x7f50600b8090 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.118+0000 7f5047fff640 1 -- 172.21.15.55:0/3831126767 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505403c870 con 0x7f50600b8090 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.118+0000 7f5047fff640 1 -- 172.21.15.55:0/3831126767 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054037040 con 0x7f50600b8090 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.118+0000 7f5065b77640 1 -- 172.21.15.55:0/3831126767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50600c8cb0 con 0x7f50600b8090 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.118+0000 7f5065b77640 1 -- 172.21.15.55:0/3831126767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50600c9270 con 0x7f50600b8090 2024-05-20T22:09:19.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.119+0000 7f5065b77640 1 -- 172.21.15.55:0/3831126767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50600c8740 con 0x7f50600b8090 2024-05-20T22:09:19.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.120+0000 7f5047fff640 1 -- 172.21.15.55:0/3831126767 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5054044070 con 0x7f50600b8090 2024-05-20T22:09:19.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.120+0000 7f5047fff640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f504c06be30 0x7f504c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.120+0000 7f5047fff640 1 -- 172.21.15.55:0/3831126767 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f50540abbe0 con 0x7f50600b8090 2024-05-20T22:09:19.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.120+0000 7f505f7fe640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f504c06be30 0x7f504c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.121+0000 7f505f7fe640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f504c06be30 0x7f504c06e340 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f50600d12f0 tx=0x7f5050006bf0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:19.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.125+0000 7f5047fff640 1 -- 172.21.15.55:0/3831126767 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f505407a400 con 0x7f50600b8090 2024-05-20T22:09:19.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.308+0000 7f5065b77640 1 -- 172.21.15.55:0/3831126767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5060069090 con 0x7f50600b8090 2024-05-20T22:09:19.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.309+0000 7f5047fff640 1 -- 172.21.15.55:0/3831126767 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16942 (secure 0 0 0) 0x7f5054079d00 con 0x7f50600b8090 2024-05-20T22:09:19.310 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:19.311 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":38,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:09:16.568558+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:09:12.734355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T22:09:13.936532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:09:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.311+0000 7f5065b77640 1 -- 172.21.15.55:0/3831126767 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f504c06be30 msgr2=0x7f504c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.311+0000 7f5065b77640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f504c06be30 0x7f504c06e340 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f50600d12f0 tx=0x7f5050006bf0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.311+0000 7f5065b77640 1 -- 172.21.15.55:0/3831126767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b8090 msgr2=0x7f5060159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.311+0000 7f5065b77640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b8090 0x7f5060159a30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f50540097a0 tx=0x7f50540046c0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.311+0000 7f5065b77640 1 -- 172.21.15.55:0/3831126767 shutdown_connections 2024-05-20T22:09:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.311+0000 7f5065b77640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f504c06be30 0x7f504c06e340 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.311+0000 7f5065b77640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f506015f140 0x7f50601ea640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.311+0000 7f5065b77640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50600b8090 0x7f5060159a30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.311+0000 7f5065b77640 1 --2- 172.21.15.55:0/3831126767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50600b3b50 0x7f50600c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.311+0000 7f5065b77640 1 -- 172.21.15.55:0/3831126767 >> 172.21.15.55:0/3831126767 conn(0x7f50600d1bb0 msgr2=0x7f50600b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.312+0000 7f5065b77640 1 -- 172.21.15.55:0/3831126767 shutdown_connections 2024-05-20T22:09:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.312+0000 7f5065b77640 1 -- 172.21.15.55:0/3831126767 wait complete. 2024-05-20T22:09:19.324 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T22:09:19.324 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:09:19.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.438+0000 7f10a1cf0640 1 Processor -- start 2024-05-20T22:09:19.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.438+0000 7f10a1cf0640 1 -- start start 2024-05-20T22:09:19.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f10a1cf0640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f109c0b3b30 0x7f109c0b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f10a1cf0640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f109c0d3dd0 con 0x7f109c06c790 2024-05-20T22:09:19.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f10a1cf0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f109c0d40b0 con 0x7f109c15f100 2024-05-20T22:09:19.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f10a1cf0640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f109c0d4390 con 0x7f109c0b3b30 2024-05-20T22:09:19.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f109b7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f109c0b3b30 0x7f109c0b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f109bfff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f109c06c790 0x7f109c0cf740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49618/0 (socket says 172.21.15.55:49618) 2024-05-20T22:09:19.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f109bfff640 1 -- 172.21.15.55:0/2819585334 learned_addr learned my addr 172.21.15.55:0/2819585334 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:19.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f109b7fe640 1 -- 172.21.15.55:0/2819585334 >> v1:172.21.15.138:6789/0 conn(0x7f109c15f100 legacy=0x7f109c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:19.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f109b7fe640 1 -- 172.21.15.55:0/2819585334 >> v1:172.21.15.55:6789/0 conn(0x7f109c06c790 legacy=0x7f109c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f109b7fe640 1 -- 172.21.15.55:0/2819585334 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109c06b5a0 con 0x7f109c0b3b30 2024-05-20T22:09:19.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.440+0000 7f109a7fc640 1 -- 172.21.15.55:0/2819585334 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825933602 0 0) 0x7f109c0d3dd0 con 0x7f109c06c790 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.441+0000 7f109b7fe640 1 --2- 172.21.15.55:0/2819585334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c0b3b30 0x7f109c0b7fe0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f108c0097a0 tx=0x7f108c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa92243216d838c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.441+0000 7f109a7fc640 1 -- 172.21.15.55:0/2819585334 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c037070 con 0x7f109c0b3b30 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.441+0000 7f109a7fc640 1 -- 172.21.15.55:0/2819585334 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f108c032440 con 0x7f109c0b3b30 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.441+0000 7f109a7fc640 1 -- 172.21.15.55:0/2819585334 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c032810 con 0x7f109c0b3b30 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.442+0000 7f10a1cf0640 1 -- 172.21.15.55:0/2819585334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c0b3b30 msgr2=0x7f109c0b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.442+0000 7f10a1cf0640 1 --2- 172.21.15.55:0/2819585334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c0b3b30 0x7f109c0b7fe0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f108c0097a0 tx=0x7f108c02a1e0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.442+0000 7f10a1cf0640 1 -- 172.21.15.55:0/2819585334 shutdown_connections 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.442+0000 7f10a1cf0640 1 --2- 172.21.15.55:0/2819585334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c0b3b30 0x7f109c0b7fe0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.442+0000 7f10a1cf0640 1 -- 172.21.15.55:0/2819585334 >> 172.21.15.55:0/2819585334 conn(0x7f109c0d51d0 msgr2=0x7f109c0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.442+0000 7f10a1cf0640 1 -- 172.21.15.55:0/2819585334 shutdown_connections 2024-05-20T22:09:19.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.442+0000 7f10a1cf0640 1 -- 172.21.15.55:0/2819585334 wait complete. 2024-05-20T22:09:19.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.444+0000 7f10a1cf0640 1 Processor -- start 2024-05-20T22:09:19.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.444+0000 7f10a1cf0640 1 -- start start 2024-05-20T22:09:19.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.445+0000 7f10a1cf0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f109c06c790 0x7f109c151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.445+0000 7f10a1cf0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109c0b3b30 0x7f109c1f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.445+0000 7f10a1cf0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c15f100 0x7f109c1f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.445+0000 7f10a1cf0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f109c06cba0 con 0x7f109c15f100 2024-05-20T22:09:19.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.445+0000 7f10a1cf0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f109c06bb60 con 0x7f109c0b3b30 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.445+0000 7f10a1cf0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f109c06cd20 con 0x7f109c06c790 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.445+0000 7f109bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c15f100 0x7f109c1f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.445+0000 7f109bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c15f100 0x7f109c1f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43210/0 (socket says 172.21.15.55:43210) 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.445+0000 7f109bfff640 1 -- 172.21.15.55:0/375779284 learned_addr learned my addr 172.21.15.55:0/375779284 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f109b7fe640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f109c06c790 0x7f109c151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f109affd640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109c0b3b30 0x7f109c1f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f109bfff640 1 -- 172.21.15.55:0/375779284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f109c06c790 msgr2=0x7f109c151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f109bfff640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f109c06c790 0x7f109c151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f109bfff640 1 -- 172.21.15.55:0/375779284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109c0b3b30 msgr2=0x7f109c1f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f109bfff640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109c0b3b30 0x7f109c1f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f109bfff640 1 -- 172.21.15.55:0/375779284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109c1f6920 con 0x7f109c15f100 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f109bfff640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c15f100 0x7f109c1f5ea0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f109400c730 tx=0x7f109400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f1083fff640 1 -- 172.21.15.55:0/375779284 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094018070 con 0x7f109c15f100 2024-05-20T22:09:19.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f1083fff640 1 -- 172.21.15.55:0/375779284 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109400ce40 con 0x7f109c15f100 2024-05-20T22:09:19.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f10a1cf0640 1 -- 172.21.15.55:0/375779284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f109c1e8a70 con 0x7f109c15f100 2024-05-20T22:09:19.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f1083fff640 1 -- 172.21.15.55:0/375779284 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109400a860 con 0x7f109c15f100 2024-05-20T22:09:19.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.446+0000 7f10a1cf0640 1 -- 172.21.15.55:0/375779284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f109c1e9030 con 0x7f109c15f100 2024-05-20T22:09:19.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.447+0000 7f10a1cf0640 1 -- 172.21.15.55:0/375779284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f109c069090 con 0x7f109c15f100 2024-05-20T22:09:19.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.448+0000 7f1083fff640 1 -- 172.21.15.55:0/375779284 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f109401b070 con 0x7f109c15f100 2024-05-20T22:09:19.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.448+0000 7f1083fff640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f108806be60 0x7f108806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.448+0000 7f1083fff640 1 -- 172.21.15.55:0/375779284 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f109408a5b0 con 0x7f109c15f100 2024-05-20T22:09:19.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.449+0000 7f109b7fe640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f108806be60 0x7f108806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.449+0000 7f109b7fe640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f108806be60 0x7f108806e370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f108c034000 tx=0x7f108c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:19.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.453+0000 7f1083fff640 1 -- 172.21.15.55:0/375779284 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1094058dc0 con 0x7f109c15f100 2024-05-20T22:09:19.530 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-05-20T22:09:19.529+0000 7f5a3938f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T22:09:19.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.634+0000 7f10a1cf0640 1 -- 172.21.15.55:0/375779284 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f109c1f6700 con 0x7f108806be60 2024-05-20T22:09:19.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.640+0000 7f1083fff640 1 -- 172.21.15.55:0/375779284 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251888 (secure 0 0 0) 0x7f109c1f6700 con 0x7f108806be60 2024-05-20T22:09:19.644 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:19.646 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:09:19.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 -- 172.21.15.55:0/375779284 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f108806be60 msgr2=0x7f108806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f108806be60 0x7f108806e370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f108c034000 tx=0x7f108c02e000 comp rx=0 tx=0).stop 2024-05-20T22:09:19.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 -- 172.21.15.55:0/375779284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c15f100 msgr2=0x7f109c1f5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c15f100 0x7f109c1f5ea0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f109400c730 tx=0x7f109400cbf0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 -- 172.21.15.55:0/375779284 shutdown_connections 2024-05-20T22:09:19.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f108806be60 0x7f108806e370 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f109c15f100 0x7f109c1f5ea0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109c0b3b30 0x7f109c1f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 --2- 172.21.15.55:0/375779284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f109c06c790 0x7f109c151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 -- 172.21.15.55:0/375779284 >> 172.21.15.55:0/375779284 conn(0x7f109c0d51d0 msgr2=0x7f109c163500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 -- 172.21.15.55:0/375779284 shutdown_connections 2024-05-20T22:09:19.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.644+0000 7f10a1cf0640 1 -- 172.21.15.55:0/375779284 wait complete. 2024-05-20T22:09:19.656 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-05-20T22:09:18.346557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237264,"kb_used_data":4452,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737456,"statfs":{"total":773094113280,"available":772851154944,"internally_reserved":0,"allocated":4558848,"data_stored":2613339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.791004"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603789+0000","last_change":"2024-05-20T22:09:15.599140+0000","last_active":"2024-05-20T22:09:15.603789+0000","last_peered":"2024-05-20T22:09:15.603789+0000","last_clean":"2024-05-20T22:09:15.603789+0000","last_became_active":"2024-05-20T22:09:15.597445+0000","last_became_peered":"2024-05-20T22:09:15.597445+0000","last_unstale":"2024-05-20T22:09:15.603789+0000","last_undegraded":"2024-05-20T22:09:15.603789+0000","last_fullsized":"2024-05-20T22:09:15.603789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:47:45.945869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.601471+0000","last_change":"2024-05-20T22:09:15.595806+0000","last_active":"2024-05-20T22:09:15.601471+0000","last_peered":"2024-05-20T22:09:15.601471+0000","last_clean":"2024-05-20T22:09:15.601471+0000","last_became_active":"2024-05-20T22:09:15.595458+0000","last_became_peered":"2024-05-20T22:09:15.595458+0000","last_unstale":"2024-05-20T22:09:15.601471+0000","last_undegraded":"2024-05-20T22:09:15.601471+0000","last_fullsized":"2024-05-20T22:09:15.601471+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:31:49.942146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598895+0000","last_change":"2024-05-20T22:09:15.595458+0000","last_active":"2024-05-20T22:09:15.598895+0000","last_peered":"2024-05-20T22:09:15.598895+0000","last_clean":"2024-05-20T22:09:15.598895+0000","last_became_active":"2024-05-20T22:09:15.595102+0000","last_became_peered":"2024-05-20T22:09:15.595102+0000","last_unstale":"2024-05-20T22:09:15.598895+0000","last_undegraded":"2024-05-20T22:09:15.598895+0000","last_fullsized":"2024-05-20T22:09:15.598895+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:14:17.302572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599009+0000","last_change":"2024-05-20T22:09:15.595790+0000","last_active":"2024-05-20T22:09:15.599009+0000","last_peered":"2024-05-20T22:09:15.599009+0000","last_clean":"2024-05-20T22:09:15.599009+0000","last_became_active":"2024-05-20T22:09:15.595023+0000","last_became_peered":"2024-05-20T22:09:15.595023+0000","last_unstale":"2024-05-20T22:09:15.599009+0000","last_undegraded":"2024-05-20T22:09:15.599009+0000","last_fullsized":"2024-05-20T22:09:15.599009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:06:03.627688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598918+0000","last_change":"2024-05-20T22:09:15.591609+0000","last_active":"2024-05-20T22:09:15.598918+0000","last_peered":"2024-05-20T22:09:15.598918+0000","last_clean":"2024-05-20T22:09:15.598918+0000","last_became_active":"2024-05-20T22:09:15.591266+0000","last_became_peered":"2024-05-20T22:09:15.591266+0000","last_unstale":"2024-05-20T22:09:15.598918+0000","last_undegraded":"2024-05-20T22:09:15.598918+0000","last_fullsized":"2024-05-20T22:09:15.598918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:46:43.359541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.592037+0000","last_change":"2024-05-20T22:09:15.590992+0000","last_active":"2024-05-20T22:09:15.592037+0000","last_peered":"2024-05-20T22:09:15.592037+0000","last_clean":"2024-05-20T22:09:15.592037+0000","last_became_active":"2024-05-20T22:09:15.590642+0000","last_became_peered":"2024-05-20T22:09:15.590642+0000","last_unstale":"2024-05-20T22:09:15.592037+0000","last_undegraded":"2024-05-20T22:09:15.592037+0000","last_fullsized":"2024-05-20T22:09:15.592037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:43.194449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603996+0000","last_change":"2024-05-20T22:09:15.602750+0000","last_active":"2024-05-20T22:09:15.603996+0000","last_peered":"2024-05-20T22:09:15.603996+0000","last_clean":"2024-05-20T22:09:15.603996+0000","last_became_active":"2024-05-20T22:09:15.602445+0000","last_became_peered":"2024-05-20T22:09:15.602445+0000","last_unstale":"2024-05-20T22:09:15.603996+0000","last_undegraded":"2024-05-20T22:09:15.603996+0000","last_fullsized":"2024-05-20T22:09:15.603996+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:15:11.782755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603599+0000","last_change":"2024-05-20T22:09:15.596617+0000","last_active":"2024-05-20T22:09:15.603599+0000","last_peered":"2024-05-20T22:09:15.603599+0000","last_clean":"2024-05-20T22:09:15.603599+0000","last_became_active":"2024-05-20T22:09:15.596022+0000","last_became_peered":"2024-05-20T22:09:15.596022+0000","last_unstale":"2024-05-20T22:09:15.603599+0000","last_undegraded":"2024-05-20T22:09:15.603599+0000","last_fullsized":"2024-05-20T22:09:15.603599+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:53:29.486115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.597856+0000","last_change":"2024-05-20T22:09:15.595109+0000","last_active":"2024-05-20T22:09:15.597856+0000","last_peered":"2024-05-20T22:09:15.597856+0000","last_clean":"2024-05-20T22:09:15.597856+0000","last_became_active":"2024-05-20T22:09:15.594776+0000","last_became_peered":"2024-05-20T22:09:15.594776+0000","last_unstale":"2024-05-20T22:09:15.597856+0000","last_undegraded":"2024-05-20T22:09:15.597856+0000","last_fullsized":"2024-05-20T22:09:15.597856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:43.194449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.590754+0000","last_change":"2024-05-20T22:09:15.578905+0000","last_active":"2024-05-20T22:09:15.590754+0000","last_peered":"2024-05-20T22:09:15.590754+0000","last_clean":"2024-05-20T22:09:15.590754+0000","last_became_active":"2024-05-20T22:09:15.578431+0000","last_became_peered":"2024-05-20T22:09:15.578431+0000","last_unstale":"2024-05-20T22:09:15.590754+0000","last_undegraded":"2024-05-20T22:09:15.590754+0000","last_fullsized":"2024-05-20T22:09:15.590754+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:56:38.658039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.607018+0000","last_change":"2024-05-20T22:09:15.603208+0000","last_active":"2024-05-20T22:09:15.607018+0000","last_peered":"2024-05-20T22:09:15.607018+0000","last_clean":"2024-05-20T22:09:15.607018+0000","last_became_active":"2024-05-20T22:09:15.602954+0000","last_became_peered":"2024-05-20T22:09:15.602954+0000","last_unstale":"2024-05-20T22:09:15.607018+0000","last_undegraded":"2024-05-20T22:09:15.607018+0000","last_fullsized":"2024-05-20T22:09:15.607018+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:31:49.942146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.595008+0000","last_change":"2024-05-20T22:09:15.593432+0000","last_active":"2024-05-20T22:09:15.595008+0000","last_peered":"2024-05-20T22:09:15.595008+0000","last_clean":"2024-05-20T22:09:15.595008+0000","last_became_active":"2024-05-20T22:09:15.593089+0000","last_became_peered":"2024-05-20T22:09:15.593089+0000","last_unstale":"2024-05-20T22:09:15.595008+0000","last_undegraded":"2024-05-20T22:09:15.595008+0000","last_fullsized":"2024-05-20T22:09:15.595008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:11.987128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.592410+0000","last_change":"2024-05-20T22:09:15.579819+0000","last_active":"2024-05-20T22:09:15.592410+0000","last_peered":"2024-05-20T22:09:15.592410+0000","last_clean":"2024-05-20T22:09:15.592410+0000","last_became_active":"2024-05-20T22:09:15.579506+0000","last_became_peered":"2024-05-20T22:09:15.579506+0000","last_unstale":"2024-05-20T22:09:15.592410+0000","last_undegraded":"2024-05-20T22:09:15.592410+0000","last_fullsized":"2024-05-20T22:09:15.592410+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:42:32.261900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.595630+0000","last_change":"2024-05-20T22:09:15.591900+0000","last_active":"2024-05-20T22:09:15.595630+0000","last_peered":"2024-05-20T22:09:15.595630+0000","last_clean":"2024-05-20T22:09:15.595630+0000","last_became_active":"2024-05-20T22:09:15.591550+0000","last_became_peered":"2024-05-20T22:09:15.591550+0000","last_unstale":"2024-05-20T22:09:15.595630+0000","last_undegraded":"2024-05-20T22:09:15.595630+0000","last_fullsized":"2024-05-20T22:09:15.595630+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:49.594726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.590698+0000","last_change":"2024-05-20T22:09:15.579193+0000","last_active":"2024-05-20T22:09:15.590698+0000","last_peered":"2024-05-20T22:09:15.590698+0000","last_clean":"2024-05-20T22:09:15.590698+0000","last_became_active":"2024-05-20T22:09:15.578868+0000","last_became_peered":"2024-05-20T22:09:15.578868+0000","last_unstale":"2024-05-20T22:09:15.590698+0000","last_undegraded":"2024-05-20T22:09:15.590698+0000","last_fullsized":"2024-05-20T22:09:15.590698+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:07.259805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.578848+0000","last_change":"2024-05-20T22:09:15.576338+0000","last_active":"2024-05-20T22:09:15.578848+0000","last_peered":"2024-05-20T22:09:15.578848+0000","last_clean":"2024-05-20T22:09:15.578848+0000","last_became_active":"2024-05-20T22:09:15.576016+0000","last_became_peered":"2024-05-20T22:09:15.576016+0000","last_unstale":"2024-05-20T22:09:15.578848+0000","last_undegraded":"2024-05-20T22:09:15.578848+0000","last_fullsized":"2024-05-20T22:09:15.578848+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:11.987128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.587575+0000","last_change":"2024-05-20T22:09:15.580857+0000","last_active":"2024-05-20T22:09:15.587575+0000","last_peered":"2024-05-20T22:09:15.587575+0000","last_clean":"2024-05-20T22:09:15.587575+0000","last_became_active":"2024-05-20T22:09:15.578124+0000","last_became_peered":"2024-05-20T22:09:15.578124+0000","last_unstale":"2024-05-20T22:09:15.587575+0000","last_undegraded":"2024-05-20T22:09:15.587575+0000","last_fullsized":"2024-05-20T22:09:15.587575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:14:17.302572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.597697+0000","last_change":"2024-05-20T22:09:15.591794+0000","last_active":"2024-05-20T22:09:15.597697+0000","last_peered":"2024-05-20T22:09:15.597697+0000","last_clean":"2024-05-20T22:09:15.597697+0000","last_became_active":"2024-05-20T22:09:15.591410+0000","last_became_peered":"2024-05-20T22:09:15.591410+0000","last_unstale":"2024-05-20T22:09:15.597697+0000","last_undegraded":"2024-05-20T22:09:15.597697+0000","last_fullsized":"2024-05-20T22:09:15.597697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:11.777797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598820+0000","last_change":"2024-05-20T22:09:15.588662+0000","last_active":"2024-05-20T22:09:15.598820+0000","last_peered":"2024-05-20T22:09:15.598820+0000","last_clean":"2024-05-20T22:09:15.598820+0000","last_became_active":"2024-05-20T22:09:15.588333+0000","last_became_peered":"2024-05-20T22:09:15.588333+0000","last_unstale":"2024-05-20T22:09:15.598820+0000","last_undegraded":"2024-05-20T22:09:15.598820+0000","last_fullsized":"2024-05-20T22:09:15.598820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:44:50.765253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599704+0000","last_change":"2024-05-20T22:09:15.595539+0000","last_active":"2024-05-20T22:09:15.599704+0000","last_peered":"2024-05-20T22:09:15.599704+0000","last_clean":"2024-05-20T22:09:15.599704+0000","last_became_active":"2024-05-20T22:09:15.595231+0000","last_became_peered":"2024-05-20T22:09:15.595231+0000","last_unstale":"2024-05-20T22:09:15.599704+0000","last_undegraded":"2024-05-20T22:09:15.599704+0000","last_fullsized":"2024-05-20T22:09:15.599704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:43:25.338982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.592435+0000","last_change":"2024-05-20T22:09:15.583836+0000","last_active":"2024-05-20T22:09:15.592435+0000","last_peered":"2024-05-20T22:09:15.592435+0000","last_clean":"2024-05-20T22:09:15.592435+0000","last_became_active":"2024-05-20T22:09:15.583494+0000","last_became_peered":"2024-05-20T22:09:15.583494+0000","last_unstale":"2024-05-20T22:09:15.592435+0000","last_undegraded":"2024-05-20T22:09:15.592435+0000","last_fullsized":"2024-05-20T22:09:15.592435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:22:03.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598294+0000","last_change":"2024-05-20T22:09:15.595159+0000","last_active":"2024-05-20T22:09:15.598294+0000","last_peered":"2024-05-20T22:09:15.598294+0000","last_clean":"2024-05-20T22:09:15.598294+0000","last_became_active":"2024-05-20T22:09:15.594866+0000","last_became_peered":"2024-05-20T22:09:15.594866+0000","last_unstale":"2024-05-20T22:09:15.598294+0000","last_undegraded":"2024-05-20T22:09:15.598294+0000","last_fullsized":"2024-05-20T22:09:15.598294+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:43.194449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.589315+0000","last_change":"2024-05-20T22:09:15.586016+0000","last_active":"2024-05-20T22:09:15.589315+0000","last_peered":"2024-05-20T22:09:15.589315+0000","last_clean":"2024-05-20T22:09:15.589315+0000","last_became_active":"2024-05-20T22:09:15.585476+0000","last_became_peered":"2024-05-20T22:09:15.585476+0000","last_unstale":"2024-05-20T22:09:15.589315+0000","last_undegraded":"2024-05-20T22:09:15.589315+0000","last_fullsized":"2024-05-20T22:09:15.589315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:50.583252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.589012+0000","last_change":"2024-05-20T22:09:15.581855+0000","last_active":"2024-05-20T22:09:15.589012+0000","last_peered":"2024-05-20T22:09:15.589012+0000","last_clean":"2024-05-20T22:09:15.589012+0000","last_became_active":"2024-05-20T22:09:15.581500+0000","last_became_peered":"2024-05-20T22:09:15.581500+0000","last_unstale":"2024-05-20T22:09:15.589012+0000","last_undegraded":"2024-05-20T22:09:15.589012+0000","last_fullsized":"2024-05-20T22:09:15.589012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:56:52.634701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.604231+0000","last_change":"2024-05-20T22:09:15.598538+0000","last_active":"2024-05-20T22:09:15.604231+0000","last_peered":"2024-05-20T22:09:15.604231+0000","last_clean":"2024-05-20T22:09:15.604231+0000","last_became_active":"2024-05-20T22:09:15.598202+0000","last_became_peered":"2024-05-20T22:09:15.598202+0000","last_unstale":"2024-05-20T22:09:15.604231+0000","last_undegraded":"2024-05-20T22:09:15.604231+0000","last_fullsized":"2024-05-20T22:09:15.604231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:49.594726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.600377+0000","last_change":"2024-05-20T22:09:15.594532+0000","last_active":"2024-05-20T22:09:15.600377+0000","last_peered":"2024-05-20T22:09:15.600377+0000","last_clean":"2024-05-20T22:09:15.600377+0000","last_became_active":"2024-05-20T22:09:15.594229+0000","last_became_peered":"2024-05-20T22:09:15.594229+0000","last_unstale":"2024-05-20T22:09:15.600377+0000","last_undegraded":"2024-05-20T22:09:15.600377+0000","last_fullsized":"2024-05-20T22:09:15.600377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:42.312844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.604337+0000","last_change":"2024-05-20T22:09:15.600641+0000","last_active":"2024-05-20T22:09:15.604337+0000","last_peered":"2024-05-20T22:09:15.604337+0000","last_clean":"2024-05-20T22:09:15.604337+0000","last_became_active":"2024-05-20T22:09:15.600035+0000","last_became_peered":"2024-05-20T22:09:15.600035+0000","last_unstale":"2024-05-20T22:09:15.604337+0000","last_undegraded":"2024-05-20T22:09:15.604337+0000","last_fullsized":"2024-05-20T22:09:15.604337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:11.987128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599011+0000","last_change":"2024-05-20T22:09:15.595816+0000","last_active":"2024-05-20T22:09:15.599011+0000","last_peered":"2024-05-20T22:09:15.599011+0000","last_clean":"2024-05-20T22:09:15.599011+0000","last_became_active":"2024-05-20T22:09:15.592835+0000","last_became_peered":"2024-05-20T22:09:15.592835+0000","last_unstale":"2024-05-20T22:09:15.599011+0000","last_undegraded":"2024-05-20T22:09:15.599011+0000","last_fullsized":"2024-05-20T22:09:15.599011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:26:09.912138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.590483+0000","last_change":"2024-05-20T22:09:15.589115+0000","last_active":"2024-05-20T22:09:15.590483+0000","last_peered":"2024-05-20T22:09:15.590483+0000","last_clean":"2024-05-20T22:09:15.590483+0000","last_became_active":"2024-05-20T22:09:15.588770+0000","last_became_peered":"2024-05-20T22:09:15.588770+0000","last_unstale":"2024-05-20T22:09:15.590483+0000","last_undegraded":"2024-05-20T22:09:15.590483+0000","last_fullsized":"2024-05-20T22:09:15.590483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:52:33.846428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599714+0000","last_change":"2024-05-20T22:09:15.594084+0000","last_active":"2024-05-20T22:09:15.599714+0000","last_peered":"2024-05-20T22:09:15.599714+0000","last_clean":"2024-05-20T22:09:15.599714+0000","last_became_active":"2024-05-20T22:09:15.593768+0000","last_became_peered":"2024-05-20T22:09:15.593768+0000","last_unstale":"2024-05-20T22:09:15.599714+0000","last_undegraded":"2024-05-20T22:09:15.599714+0000","last_fullsized":"2024-05-20T22:09:15.599714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:57.430431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.589291+0000","last_change":"2024-05-20T22:09:15.587113+0000","last_active":"2024-05-20T22:09:15.589291+0000","last_peered":"2024-05-20T22:09:15.589291+0000","last_clean":"2024-05-20T22:09:15.589291+0000","last_became_active":"2024-05-20T22:09:15.586006+0000","last_became_peered":"2024-05-20T22:09:15.586006+0000","last_unstale":"2024-05-20T22:09:15.589291+0000","last_undegraded":"2024-05-20T22:09:15.589291+0000","last_fullsized":"2024-05-20T22:09:15.589291+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:07.259805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.591520+0000","last_change":"2024-05-20T22:09:15.590135+0000","last_active":"2024-05-20T22:09:15.591520+0000","last_peered":"2024-05-20T22:09:15.591520+0000","last_clean":"2024-05-20T22:09:15.591520+0000","last_became_active":"2024-05-20T22:09:15.589692+0000","last_became_peered":"2024-05-20T22:09:15.589692+0000","last_unstale":"2024-05-20T22:09:15.591520+0000","last_undegraded":"2024-05-20T22:09:15.591520+0000","last_fullsized":"2024-05-20T22:09:15.591520+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:11.987128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.583787+0000","last_change":"2024-05-20T22:09:15.576519+0000","last_active":"2024-05-20T22:09:15.583787+0000","last_peered":"2024-05-20T22:09:15.583787+0000","last_clean":"2024-05-20T22:09:15.583787+0000","last_became_active":"2024-05-20T22:09:15.576142+0000","last_became_peered":"2024-05-20T22:09:15.576142+0000","last_unstale":"2024-05-20T22:09:15.583787+0000","last_undegraded":"2024-05-20T22:09:15.583787+0000","last_fullsized":"2024-05-20T22:09:15.583787+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:43.194449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.607979+0000","last_change":"2024-05-20T22:09:14.587928+0000","last_active":"2024-05-20T22:09:15.607979+0000","last_peered":"2024-05-20T22:09:15.607979+0000","last_clean":"2024-05-20T22:09:15.607979+0000","last_became_active":"2024-05-20T22:09:14.587611+0000","last_became_peered":"2024-05-20T22:09:14.587611+0000","last_unstale":"2024-05-20T22:09:15.607979+0000","last_undegraded":"2024-05-20T22:09:15.607979+0000","last_fullsized":"2024-05-20T22:09:15.607979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:08:23.898642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.600585+0000","last_change":"2024-05-20T22:09:15.594513+0000","last_active":"2024-05-20T22:09:15.600585+0000","last_peered":"2024-05-20T22:09:15.600585+0000","last_clean":"2024-05-20T22:09:15.600585+0000","last_became_active":"2024-05-20T22:09:15.594122+0000","last_became_peered":"2024-05-20T22:09:15.594122+0000","last_unstale":"2024-05-20T22:09:15.600585+0000","last_undegraded":"2024-05-20T22:09:15.600585+0000","last_fullsized":"2024-05-20T22:09:15.600585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:14:17.302572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603964+0000","last_change":"2024-05-20T22:09:14.598928+0000","last_active":"2024-05-20T22:09:15.603964+0000","last_peered":"2024-05-20T22:09:15.603964+0000","last_clean":"2024-05-20T22:09:15.603964+0000","last_became_active":"2024-05-20T22:09:14.598568+0000","last_became_peered":"2024-05-20T22:09:14.598568+0000","last_unstale":"2024-05-20T22:09:15.603964+0000","last_undegraded":"2024-05-20T22:09:15.603964+0000","last_fullsized":"2024-05-20T22:09:15.603964+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:56.411862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.593831+0000","last_change":"2024-05-20T22:09:15.590685+0000","last_active":"2024-05-20T22:09:15.593831+0000","last_peered":"2024-05-20T22:09:15.593831+0000","last_clean":"2024-05-20T22:09:15.593831+0000","last_became_active":"2024-05-20T22:09:15.590256+0000","last_became_peered":"2024-05-20T22:09:15.590256+0000","last_unstale":"2024-05-20T22:09:15.593831+0000","last_undegraded":"2024-05-20T22:09:15.593831+0000","last_fullsized":"2024-05-20T22:09:15.593831+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:43:25.338982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.608333+0000","last_change":"2024-05-20T22:09:14.588250+0000","last_active":"2024-05-20T22:09:15.608333+0000","last_peered":"2024-05-20T22:09:15.608333+0000","last_clean":"2024-05-20T22:09:15.608333+0000","last_became_active":"2024-05-20T22:09:14.587967+0000","last_became_peered":"2024-05-20T22:09:14.587967+0000","last_unstale":"2024-05-20T22:09:15.608333+0000","last_undegraded":"2024-05-20T22:09:15.608333+0000","last_fullsized":"2024-05-20T22:09:15.608333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:43:24.320414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.601597+0000","last_change":"2024-05-20T22:09:15.590842+0000","last_active":"2024-05-20T22:09:15.601597+0000","last_peered":"2024-05-20T22:09:15.601597+0000","last_clean":"2024-05-20T22:09:15.601597+0000","last_became_active":"2024-05-20T22:09:15.590373+0000","last_became_peered":"2024-05-20T22:09:15.590373+0000","last_unstale":"2024-05-20T22:09:15.601597+0000","last_undegraded":"2024-05-20T22:09:15.601597+0000","last_fullsized":"2024-05-20T22:09:15.601597+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:54:21.137627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.607320+0000","last_change":"2024-05-20T22:09:14.593210+0000","last_active":"2024-05-20T22:09:15.607320+0000","last_peered":"2024-05-20T22:09:15.607320+0000","last_clean":"2024-05-20T22:09:15.607320+0000","last_became_active":"2024-05-20T22:09:14.592746+0000","last_became_peered":"2024-05-20T22:09:14.592746+0000","last_unstale":"2024-05-20T22:09:15.607320+0000","last_undegraded":"2024-05-20T22:09:15.607320+0000","last_fullsized":"2024-05-20T22:09:15.607320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:10.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599504+0000","last_change":"2024-05-20T22:09:15.596549+0000","last_active":"2024-05-20T22:09:15.599504+0000","last_peered":"2024-05-20T22:09:15.599504+0000","last_clean":"2024-05-20T22:09:15.599504+0000","last_became_active":"2024-05-20T22:09:15.595818+0000","last_became_peered":"2024-05-20T22:09:15.595818+0000","last_unstale":"2024-05-20T22:09:15.599504+0000","last_undegraded":"2024-05-20T22:09:15.599504+0000","last_fullsized":"2024-05-20T22:09:15.599504+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:42.312844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.600112+0000","last_change":"2024-05-20T22:09:14.597763+0000","last_active":"2024-05-20T22:09:15.600112+0000","last_peered":"2024-05-20T22:09:15.600112+0000","last_clean":"2024-05-20T22:09:15.600112+0000","last_became_active":"2024-05-20T22:09:14.597423+0000","last_became_peered":"2024-05-20T22:09:14.597423+0000","last_unstale":"2024-05-20T22:09:15.600112+0000","last_undegraded":"2024-05-20T22:09:15.600112+0000","last_fullsized":"2024-05-20T22:09:15.600112+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:49.564683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603381+0000","last_change":"2024-05-20T22:09:15.599505+0000","last_active":"2024-05-20T22:09:15.603381+0000","last_peered":"2024-05-20T22:09:15.603381+0000","last_clean":"2024-05-20T22:09:15.603381+0000","last_became_active":"2024-05-20T22:09:15.599208+0000","last_became_peered":"2024-05-20T22:09:15.599208+0000","last_unstale":"2024-05-20T22:09:15.603381+0000","last_undegraded":"2024-05-20T22:09:15.603381+0000","last_fullsized":"2024-05-20T22:09:15.603381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:54:21.137627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.605051+0000","last_change":"2024-05-20T22:09:14.583621+0000","last_active":"2024-05-20T22:09:15.605051+0000","last_peered":"2024-05-20T22:09:15.605051+0000","last_clean":"2024-05-20T22:09:15.605051+0000","last_became_active":"2024-05-20T22:09:14.583321+0000","last_became_peered":"2024-05-20T22:09:14.583321+0000","last_unstale":"2024-05-20T22:09:15.605051+0000","last_undegraded":"2024-05-20T22:09:15.605051+0000","last_fullsized":"2024-05-20T22:09:15.605051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:43:24.320414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598993+0000","last_change":"2024-05-20T22:09:15.596210+0000","last_active":"2024-05-20T22:09:15.598993+0000","last_peered":"2024-05-20T22:09:15.598993+0000","last_clean":"2024-05-20T22:09:15.598993+0000","last_became_active":"2024-05-20T22:09:15.595923+0000","last_became_peered":"2024-05-20T22:09:15.595923+0000","last_unstale":"2024-05-20T22:09:15.598993+0000","last_undegraded":"2024-05-20T22:09:15.598993+0000","last_fullsized":"2024-05-20T22:09:15.598993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:42.312844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.580425+0000","last_change":"2024-05-20T22:09:14.587629+0000","last_active":"2024-05-20T22:09:15.580425+0000","last_peered":"2024-05-20T22:09:15.580425+0000","last_clean":"2024-05-20T22:09:15.580425+0000","last_became_active":"2024-05-20T22:09:14.587338+0000","last_became_peered":"2024-05-20T22:09:14.587338+0000","last_unstale":"2024-05-20T22:09:15.580425+0000","last_undegraded":"2024-05-20T22:09:15.580425+0000","last_fullsized":"2024-05-20T22:09:15.580425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:52:32.827860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.604093+0000","last_change":"2024-05-20T22:09:15.598577+0000","last_active":"2024-05-20T22:09:15.604093+0000","last_peered":"2024-05-20T22:09:15.604093+0000","last_clean":"2024-05-20T22:09:15.604093+0000","last_became_active":"2024-05-20T22:09:15.598255+0000","last_became_peered":"2024-05-20T22:09:15.598255+0000","last_unstale":"2024-05-20T22:09:15.604093+0000","last_undegraded":"2024-05-20T22:09:15.604093+0000","last_fullsized":"2024-05-20T22:09:15.604093+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.543717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.586771+0000","last_change":"2024-05-20T22:09:14.584597+0000","last_active":"2024-05-20T22:09:15.586771+0000","last_peered":"2024-05-20T22:09:15.586771+0000","last_clean":"2024-05-20T22:09:15.586771+0000","last_became_active":"2024-05-20T22:09:14.584278+0000","last_became_peered":"2024-05-20T22:09:14.584278+0000","last_unstale":"2024-05-20T22:09:15.586771+0000","last_undegraded":"2024-05-20T22:09:15.586771+0000","last_fullsized":"2024-05-20T22:09:15.586771+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:08:23.898642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598988+0000","last_change":"2024-05-20T22:09:15.595503+0000","last_active":"2024-05-20T22:09:15.598988+0000","last_peered":"2024-05-20T22:09:15.598988+0000","last_clean":"2024-05-20T22:09:15.598988+0000","last_became_active":"2024-05-20T22:09:15.594546+0000","last_became_peered":"2024-05-20T22:09:15.594546+0000","last_unstale":"2024-05-20T22:09:15.598988+0000","last_undegraded":"2024-05-20T22:09:15.598988+0000","last_fullsized":"2024-05-20T22:09:15.598988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:10:37.676805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.581106+0000","last_change":"2024-05-20T22:09:14.594644+0000","last_active":"2024-05-20T22:09:15.581106+0000","last_peered":"2024-05-20T22:09:15.581106+0000","last_clean":"2024-05-20T22:09:15.581106+0000","last_became_active":"2024-05-20T22:09:14.593182+0000","last_became_peered":"2024-05-20T22:09:14.593182+0000","last_unstale":"2024-05-20T22:09:15.581106+0000","last_undegraded":"2024-05-20T22:09:15.581106+0000","last_fullsized":"2024-05-20T22:09:15.581106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:22:02.937399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.597889+0000","last_change":"2024-05-20T22:09:15.589640+0000","last_active":"2024-05-20T22:09:15.597889+0000","last_peered":"2024-05-20T22:09:15.597889+0000","last_clean":"2024-05-20T22:09:15.597889+0000","last_became_active":"2024-05-20T22:09:15.589335+0000","last_became_peered":"2024-05-20T22:09:15.589335+0000","last_unstale":"2024-05-20T22:09:15.597889+0000","last_undegraded":"2024-05-20T22:09:15.597889+0000","last_fullsized":"2024-05-20T22:09:15.597889+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:42:32.261900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.573710+0000","last_change":"2024-05-20T22:09:14.585593+0000","last_active":"2024-05-20T22:09:15.573710+0000","last_peered":"2024-05-20T22:09:15.573710+0000","last_clean":"2024-05-20T22:09:15.573710+0000","last_became_active":"2024-05-20T22:09:14.585273+0000","last_became_peered":"2024-05-20T22:09:14.585273+0000","last_unstale":"2024-05-20T22:09:15.573710+0000","last_undegraded":"2024-05-20T22:09:15.573710+0000","last_fullsized":"2024-05-20T22:09:15.573710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:42.175880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.595077+0000","last_change":"2024-05-20T22:09:15.589747+0000","last_active":"2024-05-20T22:09:15.595077+0000","last_peered":"2024-05-20T22:09:15.595077+0000","last_clean":"2024-05-20T22:09:15.595077+0000","last_became_active":"2024-05-20T22:09:15.589440+0000","last_became_peered":"2024-05-20T22:09:15.589440+0000","last_unstale":"2024-05-20T22:09:15.595077+0000","last_undegraded":"2024-05-20T22:09:15.595077+0000","last_fullsized":"2024-05-20T22:09:15.595077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:49.594726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.606947+0000","last_change":"2024-05-20T22:09:14.587654+0000","last_active":"2024-05-20T22:09:15.606947+0000","last_peered":"2024-05-20T22:09:15.606947+0000","last_clean":"2024-05-20T22:09:15.606947+0000","last_became_active":"2024-05-20T22:09:14.587349+0000","last_became_peered":"2024-05-20T22:09:14.587349+0000","last_unstale":"2024-05-20T22:09:15.606947+0000","last_undegraded":"2024-05-20T22:09:15.606947+0000","last_fullsized":"2024-05-20T22:09:15.606947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:41.294276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.607432+0000","last_change":"2024-05-20T22:09:15.603058+0000","last_active":"2024-05-20T22:09:15.607432+0000","last_peered":"2024-05-20T22:09:15.607432+0000","last_clean":"2024-05-20T22:09:15.607432+0000","last_became_active":"2024-05-20T22:09:15.602626+0000","last_became_peered":"2024-05-20T22:09:15.602626+0000","last_unstale":"2024-05-20T22:09:15.607432+0000","last_undegraded":"2024-05-20T22:09:15.607432+0000","last_fullsized":"2024-05-20T22:09:15.607432+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:50.583252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.608762+0000","last_change":"2024-05-20T22:09:14.593124+0000","last_active":"2024-05-20T22:09:15.608762+0000","last_peered":"2024-05-20T22:09:15.608762+0000","last_clean":"2024-05-20T22:09:15.608762+0000","last_became_active":"2024-05-20T22:09:14.592606+0000","last_became_peered":"2024-05-20T22:09:14.592606+0000","last_unstale":"2024-05-20T22:09:15.608762+0000","last_undegraded":"2024-05-20T22:09:15.608762+0000","last_fullsized":"2024-05-20T22:09:15.608762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:26:08.893570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599535+0000","last_change":"2024-05-20T22:09:15.596887+0000","last_active":"2024-05-20T22:09:15.599535+0000","last_peered":"2024-05-20T22:09:15.599535+0000","last_clean":"2024-05-20T22:09:15.599535+0000","last_became_active":"2024-05-20T22:09:15.596213+0000","last_became_peered":"2024-05-20T22:09:15.596213+0000","last_unstale":"2024-05-20T22:09:15.599535+0000","last_undegraded":"2024-05-20T22:09:15.599535+0000","last_fullsized":"2024-05-20T22:09:15.599535+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:50.583252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.605038+0000","last_change":"2024-05-20T22:09:14.599497+0000","last_active":"2024-05-20T22:09:15.605038+0000","last_peered":"2024-05-20T22:09:15.605038+0000","last_clean":"2024-05-20T22:09:15.605038+0000","last_became_active":"2024-05-20T22:09:14.599200+0000","last_became_peered":"2024-05-20T22:09:14.599200+0000","last_unstale":"2024-05-20T22:09:15.605038+0000","last_undegraded":"2024-05-20T22:09:15.605038+0000","last_fullsized":"2024-05-20T22:09:15.605038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:02:40.870830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.601542+0000","last_change":"2024-05-20T22:09:15.591648+0000","last_active":"2024-05-20T22:09:15.601542+0000","last_peered":"2024-05-20T22:09:15.601542+0000","last_clean":"2024-05-20T22:09:15.601542+0000","last_became_active":"2024-05-20T22:09:15.591317+0000","last_became_peered":"2024-05-20T22:09:15.591317+0000","last_unstale":"2024-05-20T22:09:15.601542+0000","last_undegraded":"2024-05-20T22:09:15.601542+0000","last_fullsized":"2024-05-20T22:09:15.601542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:47.991818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.580855+0000","last_change":"2024-05-20T22:09:14.589828+0000","last_active":"2024-05-20T22:09:15.580855+0000","last_peered":"2024-05-20T22:09:15.580855+0000","last_clean":"2024-05-20T22:09:15.580855+0000","last_became_active":"2024-05-20T22:09:14.589522+0000","last_became_peered":"2024-05-20T22:09:14.589522+0000","last_unstale":"2024-05-20T22:09:15.580855+0000","last_undegraded":"2024-05-20T22:09:15.580855+0000","last_fullsized":"2024-05-20T22:09:15.580855+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:10.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.593455+0000","last_change":"2024-05-20T22:09:15.586829+0000","last_active":"2024-05-20T22:09:15.593455+0000","last_peered":"2024-05-20T22:09:15.593455+0000","last_clean":"2024-05-20T22:09:15.593455+0000","last_became_active":"2024-05-20T22:09:15.586519+0000","last_became_peered":"2024-05-20T22:09:15.586519+0000","last_unstale":"2024-05-20T22:09:15.593455+0000","last_undegraded":"2024-05-20T22:09:15.593455+0000","last_fullsized":"2024-05-20T22:09:15.593455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:56:38.658039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.579071+0000","last_change":"2024-05-20T22:09:14.587890+0000","last_active":"2024-05-20T22:09:15.579071+0000","last_peered":"2024-05-20T22:09:15.579071+0000","last_clean":"2024-05-20T22:09:15.579071+0000","last_became_active":"2024-05-20T22:09:14.587202+0000","last_became_peered":"2024-05-20T22:09:14.587202+0000","last_unstale":"2024-05-20T22:09:15.579071+0000","last_undegraded":"2024-05-20T22:09:15.579071+0000","last_fullsized":"2024-05-20T22:09:15.579071+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:35.638264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603892+0000","last_change":"2024-05-20T22:09:15.598240+0000","last_active":"2024-05-20T22:09:15.603892+0000","last_peered":"2024-05-20T22:09:15.603892+0000","last_clean":"2024-05-20T22:09:15.603892+0000","last_became_active":"2024-05-20T22:09:15.597805+0000","last_became_peered":"2024-05-20T22:09:15.597805+0000","last_unstale":"2024-05-20T22:09:15.603892+0000","last_undegraded":"2024-05-20T22:09:15.603892+0000","last_fullsized":"2024-05-20T22:09:15.603892+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:16:34.904538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.608585+0000","last_change":"2024-05-20T22:09:14.580303+0000","last_active":"2024-05-20T22:09:15.608585+0000","last_peered":"2024-05-20T22:09:15.608585+0000","last_clean":"2024-05-20T22:09:15.608585+0000","last_became_active":"2024-05-20T22:09:14.579966+0000","last_became_peered":"2024-05-20T22:09:14.579966+0000","last_unstale":"2024-05-20T22:09:15.608585+0000","last_undegraded":"2024-05-20T22:09:15.608585+0000","last_fullsized":"2024-05-20T22:09:15.608585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:10.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.604529+0000","last_change":"2024-05-20T22:09:15.590769+0000","last_active":"2024-05-20T22:09:15.604529+0000","last_peered":"2024-05-20T22:09:15.604529+0000","last_clean":"2024-05-20T22:09:15.604529+0000","last_became_active":"2024-05-20T22:09:15.590390+0000","last_became_peered":"2024-05-20T22:09:15.590390+0000","last_unstale":"2024-05-20T22:09:15.604529+0000","last_undegraded":"2024-05-20T22:09:15.604529+0000","last_fullsized":"2024-05-20T22:09:15.604529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:39:35.572524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.607688+0000","last_change":"2024-05-20T22:09:14.584480+0000","last_active":"2024-05-20T22:09:15.607688+0000","last_peered":"2024-05-20T22:09:15.607688+0000","last_clean":"2024-05-20T22:09:15.607688+0000","last_became_active":"2024-05-20T22:09:14.584010+0000","last_became_peered":"2024-05-20T22:09:14.584010+0000","last_unstale":"2024-05-20T22:09:15.607688+0000","last_undegraded":"2024-05-20T22:09:15.607688+0000","last_fullsized":"2024-05-20T22:09:15.607688+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:10.968559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603925+0000","last_change":"2024-05-20T22:09:15.600566+0000","last_active":"2024-05-20T22:09:15.603925+0000","last_peered":"2024-05-20T22:09:15.603925+0000","last_clean":"2024-05-20T22:09:15.603925+0000","last_became_active":"2024-05-20T22:09:15.599828+0000","last_became_peered":"2024-05-20T22:09:15.599828+0000","last_unstale":"2024-05-20T22:09:15.603925+0000","last_undegraded":"2024-05-20T22:09:15.603925+0000","last_fullsized":"2024-05-20T22:09:15.603925+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:47.991818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.604866+0000","last_change":"2024-05-20T22:09:14.586009+0000","last_active":"2024-05-20T22:09:15.604866+0000","last_peered":"2024-05-20T22:09:15.604866+0000","last_clean":"2024-05-20T22:09:15.604866+0000","last_became_active":"2024-05-20T22:09:14.585682+0000","last_became_peered":"2024-05-20T22:09:14.585682+0000","last_unstale":"2024-05-20T22:09:15.604866+0000","last_undegraded":"2024-05-20T22:09:15.604866+0000","last_fullsized":"2024-05-20T22:09:15.604866+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:56.411862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.595336+0000","last_change":"2024-05-20T22:09:15.593957+0000","last_active":"2024-05-20T22:09:15.595336+0000","last_peered":"2024-05-20T22:09:15.595336+0000","last_clean":"2024-05-20T22:09:15.595336+0000","last_became_active":"2024-05-20T22:09:15.593643+0000","last_became_peered":"2024-05-20T22:09:15.593643+0000","last_unstale":"2024-05-20T22:09:15.595336+0000","last_undegraded":"2024-05-20T22:09:15.595336+0000","last_fullsized":"2024-05-20T22:09:15.595336+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:11.987128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.606010+0000","last_change":"2024-05-20T22:09:14.585460+0000","last_active":"2024-05-20T22:09:15.606010+0000","last_peered":"2024-05-20T22:09:15.606010+0000","last_clean":"2024-05-20T22:09:15.606010+0000","last_became_active":"2024-05-20T22:09:14.585162+0000","last_became_peered":"2024-05-20T22:09:14.585162+0000","last_unstale":"2024-05-20T22:09:15.606010+0000","last_undegraded":"2024-05-20T22:09:15.606010+0000","last_fullsized":"2024-05-20T22:09:15.606010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:26:45.153031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599057+0000","last_change":"2024-05-20T22:09:15.592593+0000","last_active":"2024-05-20T22:09:15.599057+0000","last_peered":"2024-05-20T22:09:15.599057+0000","last_clean":"2024-05-20T22:09:15.599057+0000","last_became_active":"2024-05-20T22:09:15.592265+0000","last_became_peered":"2024-05-20T22:09:15.592265+0000","last_unstale":"2024-05-20T22:09:15.599057+0000","last_undegraded":"2024-05-20T22:09:15.599057+0000","last_fullsized":"2024-05-20T22:09:15.599057+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:48:56.049319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.581504+0000","last_change":"2024-05-20T22:09:14.590106+0000","last_active":"2024-05-20T22:09:15.581504+0000","last_peered":"2024-05-20T22:09:15.581504+0000","last_clean":"2024-05-20T22:09:15.581504+0000","last_became_active":"2024-05-20T22:09:14.588392+0000","last_became_peered":"2024-05-20T22:09:14.588392+0000","last_unstale":"2024-05-20T22:09:15.581504+0000","last_undegraded":"2024-05-20T22:09:15.581504+0000","last_fullsized":"2024-05-20T22:09:15.581504+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:56.411862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.588255+0000","last_change":"2024-05-20T22:09:15.585090+0000","last_active":"2024-05-20T22:09:15.588255+0000","last_peered":"2024-05-20T22:09:15.588255+0000","last_clean":"2024-05-20T22:09:15.588255+0000","last_became_active":"2024-05-20T22:09:15.584500+0000","last_became_peered":"2024-05-20T22:09:15.584500+0000","last_unstale":"2024-05-20T22:09:15.588255+0000","last_undegraded":"2024-05-20T22:09:15.588255+0000","last_fullsized":"2024-05-20T22:09:15.588255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:42.312844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.605936+0000","last_change":"2024-05-20T22:09:14.599390+0000","last_active":"2024-05-20T22:09:15.605936+0000","last_peered":"2024-05-20T22:09:15.605936+0000","last_clean":"2024-05-20T22:09:15.605936+0000","last_became_active":"2024-05-20T22:09:14.599051+0000","last_became_peered":"2024-05-20T22:09:14.599051+0000","last_unstale":"2024-05-20T22:09:15.605936+0000","last_undegraded":"2024-05-20T22:09:15.605936+0000","last_fullsized":"2024-05-20T22:09:15.605936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:34:48.166047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.595041+0000","last_change":"2024-05-20T22:09:15.588908+0000","last_active":"2024-05-20T22:09:15.595041+0000","last_peered":"2024-05-20T22:09:15.595041+0000","last_clean":"2024-05-20T22:09:15.595041+0000","last_became_active":"2024-05-20T22:09:15.588590+0000","last_became_peered":"2024-05-20T22:09:15.588590+0000","last_unstale":"2024-05-20T22:09:15.595041+0000","last_undegraded":"2024-05-20T22:09:15.595041+0000","last_fullsized":"2024-05-20T22:09:15.595041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:43:25.338982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.573348+0000","last_change":"2024-05-20T22:09:14.588289+0000","last_active":"2024-05-20T22:09:15.573348+0000","last_peered":"2024-05-20T22:09:15.573348+0000","last_clean":"2024-05-20T22:09:15.573348+0000","last_became_active":"2024-05-20T22:09:14.587984+0000","last_became_peered":"2024-05-20T22:09:14.587984+0000","last_unstale":"2024-05-20T22:09:15.573348+0000","last_undegraded":"2024-05-20T22:09:15.573348+0000","last_fullsized":"2024-05-20T22:09:15.573348+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:38.834402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599057+0000","last_change":"2024-05-20T22:09:15.589045+0000","last_active":"2024-05-20T22:09:15.599057+0000","last_peered":"2024-05-20T22:09:15.599057+0000","last_clean":"2024-05-20T22:09:15.599057+0000","last_became_active":"2024-05-20T22:09:15.588605+0000","last_became_peered":"2024-05-20T22:09:15.588605+0000","last_unstale":"2024-05-20T22:09:15.599057+0000","last_undegraded":"2024-05-20T22:09:15.599057+0000","last_fullsized":"2024-05-20T22:09:15.599057+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:02:41.889399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.573366+0000","last_change":"2024-05-20T22:09:14.584389+0000","last_active":"2024-05-20T22:09:15.573366+0000","last_peered":"2024-05-20T22:09:15.573366+0000","last_clean":"2024-05-20T22:09:15.573366+0000","last_became_active":"2024-05-20T22:09:14.583886+0000","last_became_peered":"2024-05-20T22:09:14.583886+0000","last_unstale":"2024-05-20T22:09:15.573366+0000","last_undegraded":"2024-05-20T22:09:15.573366+0000","last_fullsized":"2024-05-20T22:09:15.573366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:56.411862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603918+0000","last_change":"2024-05-20T22:09:15.598530+0000","last_active":"2024-05-20T22:09:15.603918+0000","last_peered":"2024-05-20T22:09:15.603918+0000","last_clean":"2024-05-20T22:09:15.603918+0000","last_became_active":"2024-05-20T22:09:15.597912+0000","last_became_peered":"2024-05-20T22:09:15.597912+0000","last_unstale":"2024-05-20T22:09:15.603918+0000","last_undegraded":"2024-05-20T22:09:15.603918+0000","last_fullsized":"2024-05-20T22:09:15.603918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:39.014811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.573755+0000","last_change":"2024-05-20T22:09:14.595048+0000","last_active":"2024-05-20T22:09:15.573755+0000","last_peered":"2024-05-20T22:09:15.573755+0000","last_clean":"2024-05-20T22:09:15.573755+0000","last_became_active":"2024-05-20T22:09:14.594748+0000","last_became_peered":"2024-05-20T22:09:14.594748+0000","last_unstale":"2024-05-20T22:09:15.573755+0000","last_undegraded":"2024-05-20T22:09:15.573755+0000","last_fullsized":"2024-05-20T22:09:15.573755+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:52:51.186712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.600522+0000","last_change":"2024-05-20T22:09:15.595806+0000","last_active":"2024-05-20T22:09:15.600522+0000","last_peered":"2024-05-20T22:09:15.600522+0000","last_clean":"2024-05-20T22:09:15.600522+0000","last_became_active":"2024-05-20T22:09:15.594138+0000","last_became_peered":"2024-05-20T22:09:15.594138+0000","last_unstale":"2024-05-20T22:09:15.600522+0000","last_undegraded":"2024-05-20T22:09:15.600522+0000","last_fullsized":"2024-05-20T22:09:15.600522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:50.583252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.606177+0000","last_change":"2024-05-20T22:09:14.585095+0000","last_active":"2024-05-20T22:09:15.606177+0000","last_peered":"2024-05-20T22:09:15.606177+0000","last_clean":"2024-05-20T22:09:15.606177+0000","last_became_active":"2024-05-20T22:09:14.583426+0000","last_became_peered":"2024-05-20T22:09:14.583426+0000","last_unstale":"2024-05-20T22:09:15.606177+0000","last_undegraded":"2024-05-20T22:09:15.606177+0000","last_fullsized":"2024-05-20T22:09:15.606177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:14:16.284004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.602975+0000","last_change":"2024-05-20T22:09:15.596852+0000","last_active":"2024-05-20T22:09:15.602975+0000","last_peered":"2024-05-20T22:09:15.602975+0000","last_clean":"2024-05-20T22:09:15.602975+0000","last_became_active":"2024-05-20T22:09:15.596479+0000","last_became_peered":"2024-05-20T22:09:15.596479+0000","last_unstale":"2024-05-20T22:09:15.602975+0000","last_undegraded":"2024-05-20T22:09:15.602975+0000","last_fullsized":"2024-05-20T22:09:15.602975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:02:41.889399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.574080+0000","last_change":"2024-05-20T22:09:14.587347+0000","last_active":"2024-05-20T22:09:15.574080+0000","last_peered":"2024-05-20T22:09:15.574080+0000","last_clean":"2024-05-20T22:09:15.574080+0000","last_became_active":"2024-05-20T22:09:14.587017+0000","last_became_peered":"2024-05-20T22:09:14.587017+0000","last_unstale":"2024-05-20T22:09:15.574080+0000","last_undegraded":"2024-05-20T22:09:15.574080+0000","last_fullsized":"2024-05-20T22:09:15.574080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:00:52.784682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.601435+0000","last_change":"2024-05-20T22:09:15.593597+0000","last_active":"2024-05-20T22:09:15.601435+0000","last_peered":"2024-05-20T22:09:15.601435+0000","last_clean":"2024-05-20T22:09:15.601435+0000","last_became_active":"2024-05-20T22:09:15.593293+0000","last_became_peered":"2024-05-20T22:09:15.593293+0000","last_unstale":"2024-05-20T22:09:15.601435+0000","last_undegraded":"2024-05-20T22:09:15.601435+0000","last_fullsized":"2024-05-20T22:09:15.601435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:15:11.782755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.583474+0000","last_change":"2024-05-20T22:09:14.600115+0000","last_active":"2024-05-20T22:09:15.583474+0000","last_peered":"2024-05-20T22:09:15.583474+0000","last_clean":"2024-05-20T22:09:15.583474+0000","last_became_active":"2024-05-20T22:09:14.599862+0000","last_became_peered":"2024-05-20T22:09:14.599862+0000","last_unstale":"2024-05-20T22:09:15.583474+0000","last_undegraded":"2024-05-20T22:09:15.583474+0000","last_fullsized":"2024-05-20T22:09:15.583474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:52:32.827860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599559+0000","last_change":"2024-05-20T22:09:15.597670+0000","last_active":"2024-05-20T22:09:15.599559+0000","last_peered":"2024-05-20T22:09:15.599559+0000","last_clean":"2024-05-20T22:09:15.599559+0000","last_became_active":"2024-05-20T22:09:15.597319+0000","last_became_peered":"2024-05-20T22:09:15.597319+0000","last_unstale":"2024-05-20T22:09:15.599559+0000","last_undegraded":"2024-05-20T22:09:15.599559+0000","last_fullsized":"2024-05-20T22:09:15.599559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:56:38.658039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.583914+0000","last_change":"2024-05-20T22:09:14.601476+0000","last_active":"2024-05-20T22:09:15.583914+0000","last_peered":"2024-05-20T22:09:15.583914+0000","last_clean":"2024-05-20T22:09:15.583914+0000","last_became_active":"2024-05-20T22:09:14.601122+0000","last_became_peered":"2024-05-20T22:09:14.601122+0000","last_unstale":"2024-05-20T22:09:15.583914+0000","last_undegraded":"2024-05-20T22:09:15.583914+0000","last_fullsized":"2024-05-20T22:09:15.583914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:10.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.589392+0000","last_change":"2024-05-20T22:09:15.587468+0000","last_active":"2024-05-20T22:09:15.589392+0000","last_peered":"2024-05-20T22:09:15.589392+0000","last_clean":"2024-05-20T22:09:15.589392+0000","last_became_active":"2024-05-20T22:09:15.587127+0000","last_became_peered":"2024-05-20T22:09:15.587127+0000","last_unstale":"2024-05-20T22:09:15.589392+0000","last_undegraded":"2024-05-20T22:09:15.589392+0000","last_fullsized":"2024-05-20T22:09:15.589392+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:49.594726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.579839+0000","last_change":"2024-05-20T22:09:14.595997+0000","last_active":"2024-05-20T22:09:15.579839+0000","last_peered":"2024-05-20T22:09:15.579839+0000","last_clean":"2024-05-20T22:09:15.579839+0000","last_became_active":"2024-05-20T22:09:14.595731+0000","last_became_peered":"2024-05-20T22:09:14.595731+0000","last_unstale":"2024-05-20T22:09:15.579839+0000","last_undegraded":"2024-05-20T22:09:15.579839+0000","last_fullsized":"2024-05-20T22:09:15.579839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:07:42.143440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598317+0000","last_change":"2024-05-20T22:09:15.595473+0000","last_active":"2024-05-20T22:09:15.598317+0000","last_peered":"2024-05-20T22:09:15.598317+0000","last_clean":"2024-05-20T22:09:15.598317+0000","last_became_active":"2024-05-20T22:09:15.592918+0000","last_became_peered":"2024-05-20T22:09:15.592918+0000","last_unstale":"2024-05-20T22:09:15.598317+0000","last_undegraded":"2024-05-20T22:09:15.598317+0000","last_fullsized":"2024-05-20T22:09:15.598317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:14:17.302572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.587255+0000","last_change":"2024-05-20T22:09:14.593206+0000","last_active":"2024-05-20T22:09:15.587255+0000","last_peered":"2024-05-20T22:09:15.587255+0000","last_clean":"2024-05-20T22:09:15.587255+0000","last_became_active":"2024-05-20T22:09:14.592751+0000","last_became_peered":"2024-05-20T22:09:14.592751+0000","last_unstale":"2024-05-20T22:09:15.587255+0000","last_undegraded":"2024-05-20T22:09:15.587255+0000","last_fullsized":"2024-05-20T22:09:15.587255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:52:32.827860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.602749+0000","last_change":"2024-05-20T22:09:15.594116+0000","last_active":"2024-05-20T22:09:15.602749+0000","last_peered":"2024-05-20T22:09:15.602749+0000","last_clean":"2024-05-20T22:09:15.602749+0000","last_became_active":"2024-05-20T22:09:15.593808+0000","last_became_peered":"2024-05-20T22:09:15.593808+0000","last_unstale":"2024-05-20T22:09:15.602749+0000","last_undegraded":"2024-05-20T22:09:15.602749+0000","last_fullsized":"2024-05-20T22:09:15.602749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:22:03.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.580711+0000","last_change":"2024-05-20T22:09:14.594884+0000","last_active":"2024-05-20T22:09:15.580711+0000","last_peered":"2024-05-20T22:09:15.580711+0000","last_clean":"2024-05-20T22:09:15.580711+0000","last_became_active":"2024-05-20T22:09:14.594591+0000","last_became_peered":"2024-05-20T22:09:14.594591+0000","last_unstale":"2024-05-20T22:09:15.580711+0000","last_undegraded":"2024-05-20T22:09:15.580711+0000","last_fullsized":"2024-05-20T22:09:15.580711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:42:31.243332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599046+0000","last_change":"2024-05-20T22:09:15.596333+0000","last_active":"2024-05-20T22:09:15.599046+0000","last_peered":"2024-05-20T22:09:15.599046+0000","last_clean":"2024-05-20T22:09:15.599046+0000","last_became_active":"2024-05-20T22:09:15.592862+0000","last_became_peered":"2024-05-20T22:09:15.592862+0000","last_unstale":"2024-05-20T22:09:15.599046+0000","last_undegraded":"2024-05-20T22:09:15.599046+0000","last_fullsized":"2024-05-20T22:09:15.599046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:39:35.572524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.573987+0000","last_change":"2024-05-20T22:09:14.582811+0000","last_active":"2024-05-20T22:09:15.573987+0000","last_peered":"2024-05-20T22:09:15.573987+0000","last_clean":"2024-05-20T22:09:15.573987+0000","last_became_active":"2024-05-20T22:09:14.582488+0000","last_became_peered":"2024-05-20T22:09:14.582488+0000","last_unstale":"2024-05-20T22:09:15.573987+0000","last_undegraded":"2024-05-20T22:09:15.573987+0000","last_fullsized":"2024-05-20T22:09:15.573987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:48.576157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.571920+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:09:15.571920+0000","last_peered":"2024-05-20T22:09:15.571920+0000","last_clean":"2024-05-20T22:09:15.571920+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:09:15.571920+0000","last_undegraded":"2024-05-20T22:09:15.571920+0000","last_fullsized":"2024-05-20T22:09:15.571920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":775386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":430080,"data_stored":183370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":773738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:09:19.661 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:09:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.810+0000 7fb79de2b640 1 Processor -- start 2024-05-20T22:09:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.810+0000 7fb79de2b640 1 -- start start 2024-05-20T22:09:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.811+0000 7fb79de2b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb7980b7fe0 0x7fb798169f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.811+0000 7fb79de2b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7980b83f0 con 0x7fb79815f100 2024-05-20T22:09:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.812+0000 7fb79de2b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7980d3dd0 con 0x7fb7980b3b30 2024-05-20T22:09:19.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.812+0000 7fb79de2b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fb79806cea0 con 0x7fb7980b7fe0 2024-05-20T22:09:19.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.812+0000 7fb797fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb7980b7fe0 0x7fb798169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.812+0000 7fb796ffd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb79815f100 0x7fb79806c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49620/0 (socket says 172.21.15.55:49620) 2024-05-20T22:09:19.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.812+0000 7fb796ffd640 1 -- 172.21.15.55:0/600738301 learned_addr learned my addr 172.21.15.55:0/600738301 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.812+0000 7fb797fff640 1 -- 172.21.15.55:0/600738301 >> v1:172.21.15.138:6789/0 conn(0x7fb7980b3b30 legacy=0x7fb79806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.812+0000 7fb797fff640 1 -- 172.21.15.55:0/600738301 >> v1:172.21.15.55:6789/0 conn(0x7fb79815f100 legacy=0x7fb79806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.812+0000 7fb797fff640 1 -- 172.21.15.55:0/600738301 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7980d40b0 con 0x7fb7980b7fe0 2024-05-20T22:09:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.812+0000 7fb7967fc640 1 -- 172.21.15.55:0/600738301 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3068604543 0 0) 0x7fb7980b83f0 con 0x7fb79815f100 2024-05-20T22:09:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.813+0000 7fb797fff640 1 --2- 172.21.15.55:0/600738301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7980b7fe0 0x7fb798169f00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb790009d90 tx=0x7fb79002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4f9b358b6feeb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.813+0000 7fb7967fc640 1 -- 172.21.15.55:0/600738301 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790007740 con 0x7fb7980b7fe0 2024-05-20T22:09:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.813+0000 7fb7967fc640 1 -- 172.21.15.55:0/600738301 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7900078e0 con 0x7fb7980b7fe0 2024-05-20T22:09:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.813+0000 7fb7967fc640 1 -- 172.21.15.55:0/600738301 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790007cb0 con 0x7fb7980b7fe0 2024-05-20T22:09:19.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.814+0000 7fb79de2b640 1 -- 172.21.15.55:0/600738301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7980b7fe0 msgr2=0x7fb798169f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.814+0000 7fb79de2b640 1 --2- 172.21.15.55:0/600738301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7980b7fe0 0x7fb798169f00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb790009d90 tx=0x7fb79002a6f0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.814+0000 7fb79de2b640 1 -- 172.21.15.55:0/600738301 shutdown_connections 2024-05-20T22:09:19.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.814+0000 7fb79de2b640 1 --2- 172.21.15.55:0/600738301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7980b7fe0 0x7fb798169f00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.814+0000 7fb79de2b640 1 -- 172.21.15.55:0/600738301 >> 172.21.15.55:0/600738301 conn(0x7fb7980d51d0 msgr2=0x7fb7980b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:19.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.814+0000 7fb79de2b640 1 -- 172.21.15.55:0/600738301 shutdown_connections 2024-05-20T22:09:19.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.814+0000 7fb79de2b640 1 -- 172.21.15.55:0/600738301 wait complete. 2024-05-20T22:09:19.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.816+0000 7fb79de2b640 1 Processor -- start 2024-05-20T22:09:19.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.816+0000 7fb79de2b640 1 -- start start 2024-05-20T22:09:19.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.817+0000 7fb79de2b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7980b3b30 0x7fb7980b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.817+0000 7fb79de2b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7980b7fe0 0x7fb7981f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.817+0000 7fb79de2b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb79815f100 0x7fb798158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.817+0000 7fb79de2b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb79806b880 con 0x7fb79815f100 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.817+0000 7fb79de2b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb79806b5a0 con 0x7fb7980b7fe0 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.817+0000 7fb79de2b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb79806bb60 con 0x7fb7980b3b30 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.817+0000 7fb797fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb79815f100 0x7fb798158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.817+0000 7fb797fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb79815f100 0x7fb798158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43226/0 (socket says 172.21.15.55:43226) 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.817+0000 7fb797fff640 1 -- 172.21.15.55:0/3707408318 learned_addr learned my addr 172.21.15.55:0/3707408318 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb796ffd640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7980b7fe0 0x7fb7981f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb7977fe640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7980b3b30 0x7fb7980b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb797fff640 1 -- 172.21.15.55:0/3707408318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7980b3b30 msgr2=0x7fb7980b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb797fff640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7980b3b30 0x7fb7980b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb797fff640 1 -- 172.21.15.55:0/3707408318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7980b7fe0 msgr2=0x7fb7981f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:19.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb797fff640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7980b7fe0 0x7fb7981f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:19.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb797fff640 1 -- 172.21.15.55:0/3707408318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79815a640 con 0x7fb79815f100 2024-05-20T22:09:19.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb796ffd640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7980b7fe0 0x7fb7981f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:19.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb797fff640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb79815f100 0x7fb798158470 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb79002ace0 tx=0x7fb790004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:19.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb77ffff640 1 -- 172.21.15.55:0/3707408318 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790040070 con 0x7fb79815f100 2024-05-20T22:09:19.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb77ffff640 1 -- 172.21.15.55:0/3707408318 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb790031c90 con 0x7fb79815f100 2024-05-20T22:09:19.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb77ffff640 1 -- 172.21.15.55:0/3707408318 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790045680 con 0x7fb79815f100 2024-05-20T22:09:19.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb79de2b640 1 -- 172.21.15.55:0/3707408318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb798153200 con 0x7fb79815f100 2024-05-20T22:09:19.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.818+0000 7fb79de2b640 1 -- 172.21.15.55:0/3707408318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7981537c0 con 0x7fb79815f100 2024-05-20T22:09:19.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.819+0000 7fb79de2b640 1 -- 172.21.15.55:0/3707408318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb798069090 con 0x7fb79815f100 2024-05-20T22:09:19.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.820+0000 7fb77ffff640 1 -- 172.21.15.55:0/3707408318 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb790039040 con 0x7fb79815f100 2024-05-20T22:09:19.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.820+0000 7fb77ffff640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb78406be60 0x7fb78406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:19.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.820+0000 7fb77ffff640 1 -- 172.21.15.55:0/3707408318 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fb7900aca60 con 0x7fb79815f100 2024-05-20T22:09:19.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.820+0000 7fb7977fe640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb78406be60 0x7fb78406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:19.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.821+0000 7fb7977fe640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb78406be60 0x7fb78406e370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb788000f60 tx=0x7fb788009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:19.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:19.825+0000 7fb77ffff640 1 -- 172.21.15.55:0/3707408318 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb79007b2b0 con 0x7fb79815f100 2024-05-20T22:09:20.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.008+0000 7fb79de2b640 1 -- 172.21.15.55:0/3707408318 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb79815a3d0 con 0x7fb78406be60 2024-05-20T22:09:20.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.015+0000 7fb77ffff640 1 -- 172.21.15.55:0/3707408318 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251888 (secure 0 0 0) 0x7fb79815a3d0 con 0x7fb78406be60 2024-05-20T22:09:20.020 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:20.022 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:09:20.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 -- 172.21.15.55:0/3707408318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb78406be60 msgr2=0x7fb78406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:20.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb78406be60 0x7fb78406e370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb788000f60 tx=0x7fb788009040 comp rx=0 tx=0).stop 2024-05-20T22:09:20.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 -- 172.21.15.55:0/3707408318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb79815f100 msgr2=0x7fb798158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:20.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb79815f100 0x7fb798158470 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb79002ace0 tx=0x7fb790004a40 comp rx=0 tx=0).stop 2024-05-20T22:09:20.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 -- 172.21.15.55:0/3707408318 shutdown_connections 2024-05-20T22:09:20.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb78406be60 0x7fb78406e370 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb79815f100 0x7fb798158470 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7980b7fe0 0x7fb7981f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 --2- 172.21.15.55:0/3707408318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7980b3b30 0x7fb7980b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 -- 172.21.15.55:0/3707408318 >> 172.21.15.55:0/3707408318 conn(0x7fb7980d51d0 msgr2=0x7fb79806c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:20.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 -- 172.21.15.55:0/3707408318 shutdown_connections 2024-05-20T22:09:20.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.018+0000 7fb79de2b640 1 -- 172.21.15.55:0/3707408318 wait complete. 2024-05-20T22:09:20.030 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-05-20T22:09:18.346557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237264,"kb_used_data":4452,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737456,"statfs":{"total":773094113280,"available":772851154944,"internally_reserved":0,"allocated":4558848,"data_stored":2613339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.791004"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603789+0000","last_change":"2024-05-20T22:09:15.599140+0000","last_active":"2024-05-20T22:09:15.603789+0000","last_peered":"2024-05-20T22:09:15.603789+0000","last_clean":"2024-05-20T22:09:15.603789+0000","last_became_active":"2024-05-20T22:09:15.597445+0000","last_became_peered":"2024-05-20T22:09:15.597445+0000","last_unstale":"2024-05-20T22:09:15.603789+0000","last_undegraded":"2024-05-20T22:09:15.603789+0000","last_fullsized":"2024-05-20T22:09:15.603789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:47:45.945869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.601471+0000","last_change":"2024-05-20T22:09:15.595806+0000","last_active":"2024-05-20T22:09:15.601471+0000","last_peered":"2024-05-20T22:09:15.601471+0000","last_clean":"2024-05-20T22:09:15.601471+0000","last_became_active":"2024-05-20T22:09:15.595458+0000","last_became_peered":"2024-05-20T22:09:15.595458+0000","last_unstale":"2024-05-20T22:09:15.601471+0000","last_undegraded":"2024-05-20T22:09:15.601471+0000","last_fullsized":"2024-05-20T22:09:15.601471+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:31:49.942146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598895+0000","last_change":"2024-05-20T22:09:15.595458+0000","last_active":"2024-05-20T22:09:15.598895+0000","last_peered":"2024-05-20T22:09:15.598895+0000","last_clean":"2024-05-20T22:09:15.598895+0000","last_became_active":"2024-05-20T22:09:15.595102+0000","last_became_peered":"2024-05-20T22:09:15.595102+0000","last_unstale":"2024-05-20T22:09:15.598895+0000","last_undegraded":"2024-05-20T22:09:15.598895+0000","last_fullsized":"2024-05-20T22:09:15.598895+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:14:17.302572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599009+0000","last_change":"2024-05-20T22:09:15.595790+0000","last_active":"2024-05-20T22:09:15.599009+0000","last_peered":"2024-05-20T22:09:15.599009+0000","last_clean":"2024-05-20T22:09:15.599009+0000","last_became_active":"2024-05-20T22:09:15.595023+0000","last_became_peered":"2024-05-20T22:09:15.595023+0000","last_unstale":"2024-05-20T22:09:15.599009+0000","last_undegraded":"2024-05-20T22:09:15.599009+0000","last_fullsized":"2024-05-20T22:09:15.599009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:06:03.627688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598918+0000","last_change":"2024-05-20T22:09:15.591609+0000","last_active":"2024-05-20T22:09:15.598918+0000","last_peered":"2024-05-20T22:09:15.598918+0000","last_clean":"2024-05-20T22:09:15.598918+0000","last_became_active":"2024-05-20T22:09:15.591266+0000","last_became_peered":"2024-05-20T22:09:15.591266+0000","last_unstale":"2024-05-20T22:09:15.598918+0000","last_undegraded":"2024-05-20T22:09:15.598918+0000","last_fullsized":"2024-05-20T22:09:15.598918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:46:43.359541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.592037+0000","last_change":"2024-05-20T22:09:15.590992+0000","last_active":"2024-05-20T22:09:15.592037+0000","last_peered":"2024-05-20T22:09:15.592037+0000","last_clean":"2024-05-20T22:09:15.592037+0000","last_became_active":"2024-05-20T22:09:15.590642+0000","last_became_peered":"2024-05-20T22:09:15.590642+0000","last_unstale":"2024-05-20T22:09:15.592037+0000","last_undegraded":"2024-05-20T22:09:15.592037+0000","last_fullsized":"2024-05-20T22:09:15.592037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:43.194449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603996+0000","last_change":"2024-05-20T22:09:15.602750+0000","last_active":"2024-05-20T22:09:15.603996+0000","last_peered":"2024-05-20T22:09:15.603996+0000","last_clean":"2024-05-20T22:09:15.603996+0000","last_became_active":"2024-05-20T22:09:15.602445+0000","last_became_peered":"2024-05-20T22:09:15.602445+0000","last_unstale":"2024-05-20T22:09:15.603996+0000","last_undegraded":"2024-05-20T22:09:15.603996+0000","last_fullsized":"2024-05-20T22:09:15.603996+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:15:11.782755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603599+0000","last_change":"2024-05-20T22:09:15.596617+0000","last_active":"2024-05-20T22:09:15.603599+0000","last_peered":"2024-05-20T22:09:15.603599+0000","last_clean":"2024-05-20T22:09:15.603599+0000","last_became_active":"2024-05-20T22:09:15.596022+0000","last_became_peered":"2024-05-20T22:09:15.596022+0000","last_unstale":"2024-05-20T22:09:15.603599+0000","last_undegraded":"2024-05-20T22:09:15.603599+0000","last_fullsized":"2024-05-20T22:09:15.603599+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:53:29.486115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.597856+0000","last_change":"2024-05-20T22:09:15.595109+0000","last_active":"2024-05-20T22:09:15.597856+0000","last_peered":"2024-05-20T22:09:15.597856+0000","last_clean":"2024-05-20T22:09:15.597856+0000","last_became_active":"2024-05-20T22:09:15.594776+0000","last_became_peered":"2024-05-20T22:09:15.594776+0000","last_unstale":"2024-05-20T22:09:15.597856+0000","last_undegraded":"2024-05-20T22:09:15.597856+0000","last_fullsized":"2024-05-20T22:09:15.597856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:43.194449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.590754+0000","last_change":"2024-05-20T22:09:15.578905+0000","last_active":"2024-05-20T22:09:15.590754+0000","last_peered":"2024-05-20T22:09:15.590754+0000","last_clean":"2024-05-20T22:09:15.590754+0000","last_became_active":"2024-05-20T22:09:15.578431+0000","last_became_peered":"2024-05-20T22:09:15.578431+0000","last_unstale":"2024-05-20T22:09:15.590754+0000","last_undegraded":"2024-05-20T22:09:15.590754+0000","last_fullsized":"2024-05-20T22:09:15.590754+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:56:38.658039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.607018+0000","last_change":"2024-05-20T22:09:15.603208+0000","last_active":"2024-05-20T22:09:15.607018+0000","last_peered":"2024-05-20T22:09:15.607018+0000","last_clean":"2024-05-20T22:09:15.607018+0000","last_became_active":"2024-05-20T22:09:15.602954+0000","last_became_peered":"2024-05-20T22:09:15.602954+0000","last_unstale":"2024-05-20T22:09:15.607018+0000","last_undegraded":"2024-05-20T22:09:15.607018+0000","last_fullsized":"2024-05-20T22:09:15.607018+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:31:49.942146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.595008+0000","last_change":"2024-05-20T22:09:15.593432+0000","last_active":"2024-05-20T22:09:15.595008+0000","last_peered":"2024-05-20T22:09:15.595008+0000","last_clean":"2024-05-20T22:09:15.595008+0000","last_became_active":"2024-05-20T22:09:15.593089+0000","last_became_peered":"2024-05-20T22:09:15.593089+0000","last_unstale":"2024-05-20T22:09:15.595008+0000","last_undegraded":"2024-05-20T22:09:15.595008+0000","last_fullsized":"2024-05-20T22:09:15.595008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:11.987128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.592410+0000","last_change":"2024-05-20T22:09:15.579819+0000","last_active":"2024-05-20T22:09:15.592410+0000","last_peered":"2024-05-20T22:09:15.592410+0000","last_clean":"2024-05-20T22:09:15.592410+0000","last_became_active":"2024-05-20T22:09:15.579506+0000","last_became_peered":"2024-05-20T22:09:15.579506+0000","last_unstale":"2024-05-20T22:09:15.592410+0000","last_undegraded":"2024-05-20T22:09:15.592410+0000","last_fullsized":"2024-05-20T22:09:15.592410+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:42:32.261900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.595630+0000","last_change":"2024-05-20T22:09:15.591900+0000","last_active":"2024-05-20T22:09:15.595630+0000","last_peered":"2024-05-20T22:09:15.595630+0000","last_clean":"2024-05-20T22:09:15.595630+0000","last_became_active":"2024-05-20T22:09:15.591550+0000","last_became_peered":"2024-05-20T22:09:15.591550+0000","last_unstale":"2024-05-20T22:09:15.595630+0000","last_undegraded":"2024-05-20T22:09:15.595630+0000","last_fullsized":"2024-05-20T22:09:15.595630+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:49.594726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.590698+0000","last_change":"2024-05-20T22:09:15.579193+0000","last_active":"2024-05-20T22:09:15.590698+0000","last_peered":"2024-05-20T22:09:15.590698+0000","last_clean":"2024-05-20T22:09:15.590698+0000","last_became_active":"2024-05-20T22:09:15.578868+0000","last_became_peered":"2024-05-20T22:09:15.578868+0000","last_unstale":"2024-05-20T22:09:15.590698+0000","last_undegraded":"2024-05-20T22:09:15.590698+0000","last_fullsized":"2024-05-20T22:09:15.590698+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:07.259805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.578848+0000","last_change":"2024-05-20T22:09:15.576338+0000","last_active":"2024-05-20T22:09:15.578848+0000","last_peered":"2024-05-20T22:09:15.578848+0000","last_clean":"2024-05-20T22:09:15.578848+0000","last_became_active":"2024-05-20T22:09:15.576016+0000","last_became_peered":"2024-05-20T22:09:15.576016+0000","last_unstale":"2024-05-20T22:09:15.578848+0000","last_undegraded":"2024-05-20T22:09:15.578848+0000","last_fullsized":"2024-05-20T22:09:15.578848+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:11.987128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.587575+0000","last_change":"2024-05-20T22:09:15.580857+0000","last_active":"2024-05-20T22:09:15.587575+0000","last_peered":"2024-05-20T22:09:15.587575+0000","last_clean":"2024-05-20T22:09:15.587575+0000","last_became_active":"2024-05-20T22:09:15.578124+0000","last_became_peered":"2024-05-20T22:09:15.578124+0000","last_unstale":"2024-05-20T22:09:15.587575+0000","last_undegraded":"2024-05-20T22:09:15.587575+0000","last_fullsized":"2024-05-20T22:09:15.587575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:14:17.302572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.597697+0000","last_change":"2024-05-20T22:09:15.591794+0000","last_active":"2024-05-20T22:09:15.597697+0000","last_peered":"2024-05-20T22:09:15.597697+0000","last_clean":"2024-05-20T22:09:15.597697+0000","last_became_active":"2024-05-20T22:09:15.591410+0000","last_became_peered":"2024-05-20T22:09:15.591410+0000","last_unstale":"2024-05-20T22:09:15.597697+0000","last_undegraded":"2024-05-20T22:09:15.597697+0000","last_fullsized":"2024-05-20T22:09:15.597697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:11.777797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598820+0000","last_change":"2024-05-20T22:09:15.588662+0000","last_active":"2024-05-20T22:09:15.598820+0000","last_peered":"2024-05-20T22:09:15.598820+0000","last_clean":"2024-05-20T22:09:15.598820+0000","last_became_active":"2024-05-20T22:09:15.588333+0000","last_became_peered":"2024-05-20T22:09:15.588333+0000","last_unstale":"2024-05-20T22:09:15.598820+0000","last_undegraded":"2024-05-20T22:09:15.598820+0000","last_fullsized":"2024-05-20T22:09:15.598820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:44:50.765253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599704+0000","last_change":"2024-05-20T22:09:15.595539+0000","last_active":"2024-05-20T22:09:15.599704+0000","last_peered":"2024-05-20T22:09:15.599704+0000","last_clean":"2024-05-20T22:09:15.599704+0000","last_became_active":"2024-05-20T22:09:15.595231+0000","last_became_peered":"2024-05-20T22:09:15.595231+0000","last_unstale":"2024-05-20T22:09:15.599704+0000","last_undegraded":"2024-05-20T22:09:15.599704+0000","last_fullsized":"2024-05-20T22:09:15.599704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:43:25.338982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.592435+0000","last_change":"2024-05-20T22:09:15.583836+0000","last_active":"2024-05-20T22:09:15.592435+0000","last_peered":"2024-05-20T22:09:15.592435+0000","last_clean":"2024-05-20T22:09:15.592435+0000","last_became_active":"2024-05-20T22:09:15.583494+0000","last_became_peered":"2024-05-20T22:09:15.583494+0000","last_unstale":"2024-05-20T22:09:15.592435+0000","last_undegraded":"2024-05-20T22:09:15.592435+0000","last_fullsized":"2024-05-20T22:09:15.592435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:22:03.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598294+0000","last_change":"2024-05-20T22:09:15.595159+0000","last_active":"2024-05-20T22:09:15.598294+0000","last_peered":"2024-05-20T22:09:15.598294+0000","last_clean":"2024-05-20T22:09:15.598294+0000","last_became_active":"2024-05-20T22:09:15.594866+0000","last_became_peered":"2024-05-20T22:09:15.594866+0000","last_unstale":"2024-05-20T22:09:15.598294+0000","last_undegraded":"2024-05-20T22:09:15.598294+0000","last_fullsized":"2024-05-20T22:09:15.598294+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:43.194449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.589315+0000","last_change":"2024-05-20T22:09:15.586016+0000","last_active":"2024-05-20T22:09:15.589315+0000","last_peered":"2024-05-20T22:09:15.589315+0000","last_clean":"2024-05-20T22:09:15.589315+0000","last_became_active":"2024-05-20T22:09:15.585476+0000","last_became_peered":"2024-05-20T22:09:15.585476+0000","last_unstale":"2024-05-20T22:09:15.589315+0000","last_undegraded":"2024-05-20T22:09:15.589315+0000","last_fullsized":"2024-05-20T22:09:15.589315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:50.583252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.589012+0000","last_change":"2024-05-20T22:09:15.581855+0000","last_active":"2024-05-20T22:09:15.589012+0000","last_peered":"2024-05-20T22:09:15.589012+0000","last_clean":"2024-05-20T22:09:15.589012+0000","last_became_active":"2024-05-20T22:09:15.581500+0000","last_became_peered":"2024-05-20T22:09:15.581500+0000","last_unstale":"2024-05-20T22:09:15.589012+0000","last_undegraded":"2024-05-20T22:09:15.589012+0000","last_fullsized":"2024-05-20T22:09:15.589012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:56:52.634701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.604231+0000","last_change":"2024-05-20T22:09:15.598538+0000","last_active":"2024-05-20T22:09:15.604231+0000","last_peered":"2024-05-20T22:09:15.604231+0000","last_clean":"2024-05-20T22:09:15.604231+0000","last_became_active":"2024-05-20T22:09:15.598202+0000","last_became_peered":"2024-05-20T22:09:15.598202+0000","last_unstale":"2024-05-20T22:09:15.604231+0000","last_undegraded":"2024-05-20T22:09:15.604231+0000","last_fullsized":"2024-05-20T22:09:15.604231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:49.594726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.600377+0000","last_change":"2024-05-20T22:09:15.594532+0000","last_active":"2024-05-20T22:09:15.600377+0000","last_peered":"2024-05-20T22:09:15.600377+0000","last_clean":"2024-05-20T22:09:15.600377+0000","last_became_active":"2024-05-20T22:09:15.594229+0000","last_became_peered":"2024-05-20T22:09:15.594229+0000","last_unstale":"2024-05-20T22:09:15.600377+0000","last_undegraded":"2024-05-20T22:09:15.600377+0000","last_fullsized":"2024-05-20T22:09:15.600377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:42.312844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.604337+0000","last_change":"2024-05-20T22:09:15.600641+0000","last_active":"2024-05-20T22:09:15.604337+0000","last_peered":"2024-05-20T22:09:15.604337+0000","last_clean":"2024-05-20T22:09:15.604337+0000","last_became_active":"2024-05-20T22:09:15.600035+0000","last_became_peered":"2024-05-20T22:09:15.600035+0000","last_unstale":"2024-05-20T22:09:15.604337+0000","last_undegraded":"2024-05-20T22:09:15.604337+0000","last_fullsized":"2024-05-20T22:09:15.604337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:11.987128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599011+0000","last_change":"2024-05-20T22:09:15.595816+0000","last_active":"2024-05-20T22:09:15.599011+0000","last_peered":"2024-05-20T22:09:15.599011+0000","last_clean":"2024-05-20T22:09:15.599011+0000","last_became_active":"2024-05-20T22:09:15.592835+0000","last_became_peered":"2024-05-20T22:09:15.592835+0000","last_unstale":"2024-05-20T22:09:15.599011+0000","last_undegraded":"2024-05-20T22:09:15.599011+0000","last_fullsized":"2024-05-20T22:09:15.599011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:26:09.912138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.590483+0000","last_change":"2024-05-20T22:09:15.589115+0000","last_active":"2024-05-20T22:09:15.590483+0000","last_peered":"2024-05-20T22:09:15.590483+0000","last_clean":"2024-05-20T22:09:15.590483+0000","last_became_active":"2024-05-20T22:09:15.588770+0000","last_became_peered":"2024-05-20T22:09:15.588770+0000","last_unstale":"2024-05-20T22:09:15.590483+0000","last_undegraded":"2024-05-20T22:09:15.590483+0000","last_fullsized":"2024-05-20T22:09:15.590483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:52:33.846428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599714+0000","last_change":"2024-05-20T22:09:15.594084+0000","last_active":"2024-05-20T22:09:15.599714+0000","last_peered":"2024-05-20T22:09:15.599714+0000","last_clean":"2024-05-20T22:09:15.599714+0000","last_became_active":"2024-05-20T22:09:15.593768+0000","last_became_peered":"2024-05-20T22:09:15.593768+0000","last_unstale":"2024-05-20T22:09:15.599714+0000","last_undegraded":"2024-05-20T22:09:15.599714+0000","last_fullsized":"2024-05-20T22:09:15.599714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:57.430431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.589291+0000","last_change":"2024-05-20T22:09:15.587113+0000","last_active":"2024-05-20T22:09:15.589291+0000","last_peered":"2024-05-20T22:09:15.589291+0000","last_clean":"2024-05-20T22:09:15.589291+0000","last_became_active":"2024-05-20T22:09:15.586006+0000","last_became_peered":"2024-05-20T22:09:15.586006+0000","last_unstale":"2024-05-20T22:09:15.589291+0000","last_undegraded":"2024-05-20T22:09:15.589291+0000","last_fullsized":"2024-05-20T22:09:15.589291+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:37:07.259805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.591520+0000","last_change":"2024-05-20T22:09:15.590135+0000","last_active":"2024-05-20T22:09:15.591520+0000","last_peered":"2024-05-20T22:09:15.591520+0000","last_clean":"2024-05-20T22:09:15.591520+0000","last_became_active":"2024-05-20T22:09:15.589692+0000","last_became_peered":"2024-05-20T22:09:15.589692+0000","last_unstale":"2024-05-20T22:09:15.591520+0000","last_undegraded":"2024-05-20T22:09:15.591520+0000","last_fullsized":"2024-05-20T22:09:15.591520+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:11.987128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.583787+0000","last_change":"2024-05-20T22:09:15.576519+0000","last_active":"2024-05-20T22:09:15.583787+0000","last_peered":"2024-05-20T22:09:15.583787+0000","last_clean":"2024-05-20T22:09:15.583787+0000","last_became_active":"2024-05-20T22:09:15.576142+0000","last_became_peered":"2024-05-20T22:09:15.576142+0000","last_unstale":"2024-05-20T22:09:15.583787+0000","last_undegraded":"2024-05-20T22:09:15.583787+0000","last_fullsized":"2024-05-20T22:09:15.583787+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:43.194449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.607979+0000","last_change":"2024-05-20T22:09:14.587928+0000","last_active":"2024-05-20T22:09:15.607979+0000","last_peered":"2024-05-20T22:09:15.607979+0000","last_clean":"2024-05-20T22:09:15.607979+0000","last_became_active":"2024-05-20T22:09:14.587611+0000","last_became_peered":"2024-05-20T22:09:14.587611+0000","last_unstale":"2024-05-20T22:09:15.607979+0000","last_undegraded":"2024-05-20T22:09:15.607979+0000","last_fullsized":"2024-05-20T22:09:15.607979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:08:23.898642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.600585+0000","last_change":"2024-05-20T22:09:15.594513+0000","last_active":"2024-05-20T22:09:15.600585+0000","last_peered":"2024-05-20T22:09:15.600585+0000","last_clean":"2024-05-20T22:09:15.600585+0000","last_became_active":"2024-05-20T22:09:15.594122+0000","last_became_peered":"2024-05-20T22:09:15.594122+0000","last_unstale":"2024-05-20T22:09:15.600585+0000","last_undegraded":"2024-05-20T22:09:15.600585+0000","last_fullsized":"2024-05-20T22:09:15.600585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:14:17.302572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603964+0000","last_change":"2024-05-20T22:09:14.598928+0000","last_active":"2024-05-20T22:09:15.603964+0000","last_peered":"2024-05-20T22:09:15.603964+0000","last_clean":"2024-05-20T22:09:15.603964+0000","last_became_active":"2024-05-20T22:09:14.598568+0000","last_became_peered":"2024-05-20T22:09:14.598568+0000","last_unstale":"2024-05-20T22:09:15.603964+0000","last_undegraded":"2024-05-20T22:09:15.603964+0000","last_fullsized":"2024-05-20T22:09:15.603964+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:56.411862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.593831+0000","last_change":"2024-05-20T22:09:15.590685+0000","last_active":"2024-05-20T22:09:15.593831+0000","last_peered":"2024-05-20T22:09:15.593831+0000","last_clean":"2024-05-20T22:09:15.593831+0000","last_became_active":"2024-05-20T22:09:15.590256+0000","last_became_peered":"2024-05-20T22:09:15.590256+0000","last_unstale":"2024-05-20T22:09:15.593831+0000","last_undegraded":"2024-05-20T22:09:15.593831+0000","last_fullsized":"2024-05-20T22:09:15.593831+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:43:25.338982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.608333+0000","last_change":"2024-05-20T22:09:14.588250+0000","last_active":"2024-05-20T22:09:15.608333+0000","last_peered":"2024-05-20T22:09:15.608333+0000","last_clean":"2024-05-20T22:09:15.608333+0000","last_became_active":"2024-05-20T22:09:14.587967+0000","last_became_peered":"2024-05-20T22:09:14.587967+0000","last_unstale":"2024-05-20T22:09:15.608333+0000","last_undegraded":"2024-05-20T22:09:15.608333+0000","last_fullsized":"2024-05-20T22:09:15.608333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:43:24.320414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.601597+0000","last_change":"2024-05-20T22:09:15.590842+0000","last_active":"2024-05-20T22:09:15.601597+0000","last_peered":"2024-05-20T22:09:15.601597+0000","last_clean":"2024-05-20T22:09:15.601597+0000","last_became_active":"2024-05-20T22:09:15.590373+0000","last_became_peered":"2024-05-20T22:09:15.590373+0000","last_unstale":"2024-05-20T22:09:15.601597+0000","last_undegraded":"2024-05-20T22:09:15.601597+0000","last_fullsized":"2024-05-20T22:09:15.601597+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:54:21.137627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.607320+0000","last_change":"2024-05-20T22:09:14.593210+0000","last_active":"2024-05-20T22:09:15.607320+0000","last_peered":"2024-05-20T22:09:15.607320+0000","last_clean":"2024-05-20T22:09:15.607320+0000","last_became_active":"2024-05-20T22:09:14.592746+0000","last_became_peered":"2024-05-20T22:09:14.592746+0000","last_unstale":"2024-05-20T22:09:15.607320+0000","last_undegraded":"2024-05-20T22:09:15.607320+0000","last_fullsized":"2024-05-20T22:09:15.607320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:10.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599504+0000","last_change":"2024-05-20T22:09:15.596549+0000","last_active":"2024-05-20T22:09:15.599504+0000","last_peered":"2024-05-20T22:09:15.599504+0000","last_clean":"2024-05-20T22:09:15.599504+0000","last_became_active":"2024-05-20T22:09:15.595818+0000","last_became_peered":"2024-05-20T22:09:15.595818+0000","last_unstale":"2024-05-20T22:09:15.599504+0000","last_undegraded":"2024-05-20T22:09:15.599504+0000","last_fullsized":"2024-05-20T22:09:15.599504+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:42.312844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.600112+0000","last_change":"2024-05-20T22:09:14.597763+0000","last_active":"2024-05-20T22:09:15.600112+0000","last_peered":"2024-05-20T22:09:15.600112+0000","last_clean":"2024-05-20T22:09:15.600112+0000","last_became_active":"2024-05-20T22:09:14.597423+0000","last_became_peered":"2024-05-20T22:09:14.597423+0000","last_unstale":"2024-05-20T22:09:15.600112+0000","last_undegraded":"2024-05-20T22:09:15.600112+0000","last_fullsized":"2024-05-20T22:09:15.600112+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:49.564683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603381+0000","last_change":"2024-05-20T22:09:15.599505+0000","last_active":"2024-05-20T22:09:15.603381+0000","last_peered":"2024-05-20T22:09:15.603381+0000","last_clean":"2024-05-20T22:09:15.603381+0000","last_became_active":"2024-05-20T22:09:15.599208+0000","last_became_peered":"2024-05-20T22:09:15.599208+0000","last_unstale":"2024-05-20T22:09:15.603381+0000","last_undegraded":"2024-05-20T22:09:15.603381+0000","last_fullsized":"2024-05-20T22:09:15.603381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:54:21.137627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.605051+0000","last_change":"2024-05-20T22:09:14.583621+0000","last_active":"2024-05-20T22:09:15.605051+0000","last_peered":"2024-05-20T22:09:15.605051+0000","last_clean":"2024-05-20T22:09:15.605051+0000","last_became_active":"2024-05-20T22:09:14.583321+0000","last_became_peered":"2024-05-20T22:09:14.583321+0000","last_unstale":"2024-05-20T22:09:15.605051+0000","last_undegraded":"2024-05-20T22:09:15.605051+0000","last_fullsized":"2024-05-20T22:09:15.605051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:43:24.320414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598993+0000","last_change":"2024-05-20T22:09:15.596210+0000","last_active":"2024-05-20T22:09:15.598993+0000","last_peered":"2024-05-20T22:09:15.598993+0000","last_clean":"2024-05-20T22:09:15.598993+0000","last_became_active":"2024-05-20T22:09:15.595923+0000","last_became_peered":"2024-05-20T22:09:15.595923+0000","last_unstale":"2024-05-20T22:09:15.598993+0000","last_undegraded":"2024-05-20T22:09:15.598993+0000","last_fullsized":"2024-05-20T22:09:15.598993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:42.312844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.580425+0000","last_change":"2024-05-20T22:09:14.587629+0000","last_active":"2024-05-20T22:09:15.580425+0000","last_peered":"2024-05-20T22:09:15.580425+0000","last_clean":"2024-05-20T22:09:15.580425+0000","last_became_active":"2024-05-20T22:09:14.587338+0000","last_became_peered":"2024-05-20T22:09:14.587338+0000","last_unstale":"2024-05-20T22:09:15.580425+0000","last_undegraded":"2024-05-20T22:09:15.580425+0000","last_fullsized":"2024-05-20T22:09:15.580425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:52:32.827860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.604093+0000","last_change":"2024-05-20T22:09:15.598577+0000","last_active":"2024-05-20T22:09:15.604093+0000","last_peered":"2024-05-20T22:09:15.604093+0000","last_clean":"2024-05-20T22:09:15.604093+0000","last_became_active":"2024-05-20T22:09:15.598255+0000","last_became_peered":"2024-05-20T22:09:15.598255+0000","last_unstale":"2024-05-20T22:09:15.604093+0000","last_undegraded":"2024-05-20T22:09:15.604093+0000","last_fullsized":"2024-05-20T22:09:15.604093+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.543717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.586771+0000","last_change":"2024-05-20T22:09:14.584597+0000","last_active":"2024-05-20T22:09:15.586771+0000","last_peered":"2024-05-20T22:09:15.586771+0000","last_clean":"2024-05-20T22:09:15.586771+0000","last_became_active":"2024-05-20T22:09:14.584278+0000","last_became_peered":"2024-05-20T22:09:14.584278+0000","last_unstale":"2024-05-20T22:09:15.586771+0000","last_undegraded":"2024-05-20T22:09:15.586771+0000","last_fullsized":"2024-05-20T22:09:15.586771+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:08:23.898642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598988+0000","last_change":"2024-05-20T22:09:15.595503+0000","last_active":"2024-05-20T22:09:15.598988+0000","last_peered":"2024-05-20T22:09:15.598988+0000","last_clean":"2024-05-20T22:09:15.598988+0000","last_became_active":"2024-05-20T22:09:15.594546+0000","last_became_peered":"2024-05-20T22:09:15.594546+0000","last_unstale":"2024-05-20T22:09:15.598988+0000","last_undegraded":"2024-05-20T22:09:15.598988+0000","last_fullsized":"2024-05-20T22:09:15.598988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:10:37.676805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.581106+0000","last_change":"2024-05-20T22:09:14.594644+0000","last_active":"2024-05-20T22:09:15.581106+0000","last_peered":"2024-05-20T22:09:15.581106+0000","last_clean":"2024-05-20T22:09:15.581106+0000","last_became_active":"2024-05-20T22:09:14.593182+0000","last_became_peered":"2024-05-20T22:09:14.593182+0000","last_unstale":"2024-05-20T22:09:15.581106+0000","last_undegraded":"2024-05-20T22:09:15.581106+0000","last_fullsized":"2024-05-20T22:09:15.581106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:22:02.937399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.597889+0000","last_change":"2024-05-20T22:09:15.589640+0000","last_active":"2024-05-20T22:09:15.597889+0000","last_peered":"2024-05-20T22:09:15.597889+0000","last_clean":"2024-05-20T22:09:15.597889+0000","last_became_active":"2024-05-20T22:09:15.589335+0000","last_became_peered":"2024-05-20T22:09:15.589335+0000","last_unstale":"2024-05-20T22:09:15.597889+0000","last_undegraded":"2024-05-20T22:09:15.597889+0000","last_fullsized":"2024-05-20T22:09:15.597889+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:42:32.261900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.573710+0000","last_change":"2024-05-20T22:09:14.585593+0000","last_active":"2024-05-20T22:09:15.573710+0000","last_peered":"2024-05-20T22:09:15.573710+0000","last_clean":"2024-05-20T22:09:15.573710+0000","last_became_active":"2024-05-20T22:09:14.585273+0000","last_became_peered":"2024-05-20T22:09:14.585273+0000","last_unstale":"2024-05-20T22:09:15.573710+0000","last_undegraded":"2024-05-20T22:09:15.573710+0000","last_fullsized":"2024-05-20T22:09:15.573710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:47:42.175880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.595077+0000","last_change":"2024-05-20T22:09:15.589747+0000","last_active":"2024-05-20T22:09:15.595077+0000","last_peered":"2024-05-20T22:09:15.595077+0000","last_clean":"2024-05-20T22:09:15.595077+0000","last_became_active":"2024-05-20T22:09:15.589440+0000","last_became_peered":"2024-05-20T22:09:15.589440+0000","last_unstale":"2024-05-20T22:09:15.595077+0000","last_undegraded":"2024-05-20T22:09:15.595077+0000","last_fullsized":"2024-05-20T22:09:15.595077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:49.594726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.606947+0000","last_change":"2024-05-20T22:09:14.587654+0000","last_active":"2024-05-20T22:09:15.606947+0000","last_peered":"2024-05-20T22:09:15.606947+0000","last_clean":"2024-05-20T22:09:15.606947+0000","last_became_active":"2024-05-20T22:09:14.587349+0000","last_became_peered":"2024-05-20T22:09:14.587349+0000","last_unstale":"2024-05-20T22:09:15.606947+0000","last_undegraded":"2024-05-20T22:09:15.606947+0000","last_fullsized":"2024-05-20T22:09:15.606947+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:41.294276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.607432+0000","last_change":"2024-05-20T22:09:15.603058+0000","last_active":"2024-05-20T22:09:15.607432+0000","last_peered":"2024-05-20T22:09:15.607432+0000","last_clean":"2024-05-20T22:09:15.607432+0000","last_became_active":"2024-05-20T22:09:15.602626+0000","last_became_peered":"2024-05-20T22:09:15.602626+0000","last_unstale":"2024-05-20T22:09:15.607432+0000","last_undegraded":"2024-05-20T22:09:15.607432+0000","last_fullsized":"2024-05-20T22:09:15.607432+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:50.583252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.608762+0000","last_change":"2024-05-20T22:09:14.593124+0000","last_active":"2024-05-20T22:09:15.608762+0000","last_peered":"2024-05-20T22:09:15.608762+0000","last_clean":"2024-05-20T22:09:15.608762+0000","last_became_active":"2024-05-20T22:09:14.592606+0000","last_became_peered":"2024-05-20T22:09:14.592606+0000","last_unstale":"2024-05-20T22:09:15.608762+0000","last_undegraded":"2024-05-20T22:09:15.608762+0000","last_fullsized":"2024-05-20T22:09:15.608762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:26:08.893570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599535+0000","last_change":"2024-05-20T22:09:15.596887+0000","last_active":"2024-05-20T22:09:15.599535+0000","last_peered":"2024-05-20T22:09:15.599535+0000","last_clean":"2024-05-20T22:09:15.599535+0000","last_became_active":"2024-05-20T22:09:15.596213+0000","last_became_peered":"2024-05-20T22:09:15.596213+0000","last_unstale":"2024-05-20T22:09:15.599535+0000","last_undegraded":"2024-05-20T22:09:15.599535+0000","last_fullsized":"2024-05-20T22:09:15.599535+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:50.583252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.605038+0000","last_change":"2024-05-20T22:09:14.599497+0000","last_active":"2024-05-20T22:09:15.605038+0000","last_peered":"2024-05-20T22:09:15.605038+0000","last_clean":"2024-05-20T22:09:15.605038+0000","last_became_active":"2024-05-20T22:09:14.599200+0000","last_became_peered":"2024-05-20T22:09:14.599200+0000","last_unstale":"2024-05-20T22:09:15.605038+0000","last_undegraded":"2024-05-20T22:09:15.605038+0000","last_fullsized":"2024-05-20T22:09:15.605038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:02:40.870830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.601542+0000","last_change":"2024-05-20T22:09:15.591648+0000","last_active":"2024-05-20T22:09:15.601542+0000","last_peered":"2024-05-20T22:09:15.601542+0000","last_clean":"2024-05-20T22:09:15.601542+0000","last_became_active":"2024-05-20T22:09:15.591317+0000","last_became_peered":"2024-05-20T22:09:15.591317+0000","last_unstale":"2024-05-20T22:09:15.601542+0000","last_undegraded":"2024-05-20T22:09:15.601542+0000","last_fullsized":"2024-05-20T22:09:15.601542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:47.991818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.580855+0000","last_change":"2024-05-20T22:09:14.589828+0000","last_active":"2024-05-20T22:09:15.580855+0000","last_peered":"2024-05-20T22:09:15.580855+0000","last_clean":"2024-05-20T22:09:15.580855+0000","last_became_active":"2024-05-20T22:09:14.589522+0000","last_became_peered":"2024-05-20T22:09:14.589522+0000","last_unstale":"2024-05-20T22:09:15.580855+0000","last_undegraded":"2024-05-20T22:09:15.580855+0000","last_fullsized":"2024-05-20T22:09:15.580855+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:10.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.593455+0000","last_change":"2024-05-20T22:09:15.586829+0000","last_active":"2024-05-20T22:09:15.593455+0000","last_peered":"2024-05-20T22:09:15.593455+0000","last_clean":"2024-05-20T22:09:15.593455+0000","last_became_active":"2024-05-20T22:09:15.586519+0000","last_became_peered":"2024-05-20T22:09:15.586519+0000","last_unstale":"2024-05-20T22:09:15.593455+0000","last_undegraded":"2024-05-20T22:09:15.593455+0000","last_fullsized":"2024-05-20T22:09:15.593455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:56:38.658039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.579071+0000","last_change":"2024-05-20T22:09:14.587890+0000","last_active":"2024-05-20T22:09:15.579071+0000","last_peered":"2024-05-20T22:09:15.579071+0000","last_clean":"2024-05-20T22:09:15.579071+0000","last_became_active":"2024-05-20T22:09:14.587202+0000","last_became_peered":"2024-05-20T22:09:14.587202+0000","last_unstale":"2024-05-20T22:09:15.579071+0000","last_undegraded":"2024-05-20T22:09:15.579071+0000","last_fullsized":"2024-05-20T22:09:15.579071+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:03:35.638264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603892+0000","last_change":"2024-05-20T22:09:15.598240+0000","last_active":"2024-05-20T22:09:15.603892+0000","last_peered":"2024-05-20T22:09:15.603892+0000","last_clean":"2024-05-20T22:09:15.603892+0000","last_became_active":"2024-05-20T22:09:15.597805+0000","last_became_peered":"2024-05-20T22:09:15.597805+0000","last_unstale":"2024-05-20T22:09:15.603892+0000","last_undegraded":"2024-05-20T22:09:15.603892+0000","last_fullsized":"2024-05-20T22:09:15.603892+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:16:34.904538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.608585+0000","last_change":"2024-05-20T22:09:14.580303+0000","last_active":"2024-05-20T22:09:15.608585+0000","last_peered":"2024-05-20T22:09:15.608585+0000","last_clean":"2024-05-20T22:09:15.608585+0000","last_became_active":"2024-05-20T22:09:14.579966+0000","last_became_peered":"2024-05-20T22:09:14.579966+0000","last_unstale":"2024-05-20T22:09:15.608585+0000","last_undegraded":"2024-05-20T22:09:15.608585+0000","last_fullsized":"2024-05-20T22:09:15.608585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:10.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.604529+0000","last_change":"2024-05-20T22:09:15.590769+0000","last_active":"2024-05-20T22:09:15.604529+0000","last_peered":"2024-05-20T22:09:15.604529+0000","last_clean":"2024-05-20T22:09:15.604529+0000","last_became_active":"2024-05-20T22:09:15.590390+0000","last_became_peered":"2024-05-20T22:09:15.590390+0000","last_unstale":"2024-05-20T22:09:15.604529+0000","last_undegraded":"2024-05-20T22:09:15.604529+0000","last_fullsized":"2024-05-20T22:09:15.604529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:39:35.572524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.607688+0000","last_change":"2024-05-20T22:09:14.584480+0000","last_active":"2024-05-20T22:09:15.607688+0000","last_peered":"2024-05-20T22:09:15.607688+0000","last_clean":"2024-05-20T22:09:15.607688+0000","last_became_active":"2024-05-20T22:09:14.584010+0000","last_became_peered":"2024-05-20T22:09:14.584010+0000","last_unstale":"2024-05-20T22:09:15.607688+0000","last_undegraded":"2024-05-20T22:09:15.607688+0000","last_fullsized":"2024-05-20T22:09:15.607688+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:10.968559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603925+0000","last_change":"2024-05-20T22:09:15.600566+0000","last_active":"2024-05-20T22:09:15.603925+0000","last_peered":"2024-05-20T22:09:15.603925+0000","last_clean":"2024-05-20T22:09:15.603925+0000","last_became_active":"2024-05-20T22:09:15.599828+0000","last_became_peered":"2024-05-20T22:09:15.599828+0000","last_unstale":"2024-05-20T22:09:15.603925+0000","last_undegraded":"2024-05-20T22:09:15.603925+0000","last_fullsized":"2024-05-20T22:09:15.603925+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:47.991818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.604866+0000","last_change":"2024-05-20T22:09:14.586009+0000","last_active":"2024-05-20T22:09:15.604866+0000","last_peered":"2024-05-20T22:09:15.604866+0000","last_clean":"2024-05-20T22:09:15.604866+0000","last_became_active":"2024-05-20T22:09:14.585682+0000","last_became_peered":"2024-05-20T22:09:14.585682+0000","last_unstale":"2024-05-20T22:09:15.604866+0000","last_undegraded":"2024-05-20T22:09:15.604866+0000","last_fullsized":"2024-05-20T22:09:15.604866+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:56.411862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.595336+0000","last_change":"2024-05-20T22:09:15.593957+0000","last_active":"2024-05-20T22:09:15.595336+0000","last_peered":"2024-05-20T22:09:15.595336+0000","last_clean":"2024-05-20T22:09:15.595336+0000","last_became_active":"2024-05-20T22:09:15.593643+0000","last_became_peered":"2024-05-20T22:09:15.593643+0000","last_unstale":"2024-05-20T22:09:15.595336+0000","last_undegraded":"2024-05-20T22:09:15.595336+0000","last_fullsized":"2024-05-20T22:09:15.595336+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:23:11.987128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.606010+0000","last_change":"2024-05-20T22:09:14.585460+0000","last_active":"2024-05-20T22:09:15.606010+0000","last_peered":"2024-05-20T22:09:15.606010+0000","last_clean":"2024-05-20T22:09:15.606010+0000","last_became_active":"2024-05-20T22:09:14.585162+0000","last_became_peered":"2024-05-20T22:09:14.585162+0000","last_unstale":"2024-05-20T22:09:15.606010+0000","last_undegraded":"2024-05-20T22:09:15.606010+0000","last_fullsized":"2024-05-20T22:09:15.606010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:26:45.153031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599057+0000","last_change":"2024-05-20T22:09:15.592593+0000","last_active":"2024-05-20T22:09:15.599057+0000","last_peered":"2024-05-20T22:09:15.599057+0000","last_clean":"2024-05-20T22:09:15.599057+0000","last_became_active":"2024-05-20T22:09:15.592265+0000","last_became_peered":"2024-05-20T22:09:15.592265+0000","last_unstale":"2024-05-20T22:09:15.599057+0000","last_undegraded":"2024-05-20T22:09:15.599057+0000","last_fullsized":"2024-05-20T22:09:15.599057+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:48:56.049319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.581504+0000","last_change":"2024-05-20T22:09:14.590106+0000","last_active":"2024-05-20T22:09:15.581504+0000","last_peered":"2024-05-20T22:09:15.581504+0000","last_clean":"2024-05-20T22:09:15.581504+0000","last_became_active":"2024-05-20T22:09:14.588392+0000","last_became_peered":"2024-05-20T22:09:14.588392+0000","last_unstale":"2024-05-20T22:09:15.581504+0000","last_undegraded":"2024-05-20T22:09:15.581504+0000","last_fullsized":"2024-05-20T22:09:15.581504+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:56.411862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.588255+0000","last_change":"2024-05-20T22:09:15.585090+0000","last_active":"2024-05-20T22:09:15.588255+0000","last_peered":"2024-05-20T22:09:15.588255+0000","last_clean":"2024-05-20T22:09:15.588255+0000","last_became_active":"2024-05-20T22:09:15.584500+0000","last_became_peered":"2024-05-20T22:09:15.584500+0000","last_unstale":"2024-05-20T22:09:15.588255+0000","last_undegraded":"2024-05-20T22:09:15.588255+0000","last_fullsized":"2024-05-20T22:09:15.588255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:42.312844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.605936+0000","last_change":"2024-05-20T22:09:14.599390+0000","last_active":"2024-05-20T22:09:15.605936+0000","last_peered":"2024-05-20T22:09:15.605936+0000","last_clean":"2024-05-20T22:09:15.605936+0000","last_became_active":"2024-05-20T22:09:14.599051+0000","last_became_peered":"2024-05-20T22:09:14.599051+0000","last_unstale":"2024-05-20T22:09:15.605936+0000","last_undegraded":"2024-05-20T22:09:15.605936+0000","last_fullsized":"2024-05-20T22:09:15.605936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:34:48.166047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.595041+0000","last_change":"2024-05-20T22:09:15.588908+0000","last_active":"2024-05-20T22:09:15.595041+0000","last_peered":"2024-05-20T22:09:15.595041+0000","last_clean":"2024-05-20T22:09:15.595041+0000","last_became_active":"2024-05-20T22:09:15.588590+0000","last_became_peered":"2024-05-20T22:09:15.588590+0000","last_unstale":"2024-05-20T22:09:15.595041+0000","last_undegraded":"2024-05-20T22:09:15.595041+0000","last_fullsized":"2024-05-20T22:09:15.595041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:43:25.338982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.573348+0000","last_change":"2024-05-20T22:09:14.588289+0000","last_active":"2024-05-20T22:09:15.573348+0000","last_peered":"2024-05-20T22:09:15.573348+0000","last_clean":"2024-05-20T22:09:15.573348+0000","last_became_active":"2024-05-20T22:09:14.587984+0000","last_became_peered":"2024-05-20T22:09:14.587984+0000","last_unstale":"2024-05-20T22:09:15.573348+0000","last_undegraded":"2024-05-20T22:09:15.573348+0000","last_fullsized":"2024-05-20T22:09:15.573348+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:38.834402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599057+0000","last_change":"2024-05-20T22:09:15.589045+0000","last_active":"2024-05-20T22:09:15.599057+0000","last_peered":"2024-05-20T22:09:15.599057+0000","last_clean":"2024-05-20T22:09:15.599057+0000","last_became_active":"2024-05-20T22:09:15.588605+0000","last_became_peered":"2024-05-20T22:09:15.588605+0000","last_unstale":"2024-05-20T22:09:15.599057+0000","last_undegraded":"2024-05-20T22:09:15.599057+0000","last_fullsized":"2024-05-20T22:09:15.599057+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:02:41.889399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.573366+0000","last_change":"2024-05-20T22:09:14.584389+0000","last_active":"2024-05-20T22:09:15.573366+0000","last_peered":"2024-05-20T22:09:15.573366+0000","last_clean":"2024-05-20T22:09:15.573366+0000","last_became_active":"2024-05-20T22:09:14.583886+0000","last_became_peered":"2024-05-20T22:09:14.583886+0000","last_unstale":"2024-05-20T22:09:15.573366+0000","last_undegraded":"2024-05-20T22:09:15.573366+0000","last_fullsized":"2024-05-20T22:09:15.573366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:13:56.411862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.603918+0000","last_change":"2024-05-20T22:09:15.598530+0000","last_active":"2024-05-20T22:09:15.603918+0000","last_peered":"2024-05-20T22:09:15.603918+0000","last_clean":"2024-05-20T22:09:15.603918+0000","last_became_active":"2024-05-20T22:09:15.597912+0000","last_became_peered":"2024-05-20T22:09:15.597912+0000","last_unstale":"2024-05-20T22:09:15.603918+0000","last_undegraded":"2024-05-20T22:09:15.603918+0000","last_fullsized":"2024-05-20T22:09:15.603918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:39.014811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.573755+0000","last_change":"2024-05-20T22:09:14.595048+0000","last_active":"2024-05-20T22:09:15.573755+0000","last_peered":"2024-05-20T22:09:15.573755+0000","last_clean":"2024-05-20T22:09:15.573755+0000","last_became_active":"2024-05-20T22:09:14.594748+0000","last_became_peered":"2024-05-20T22:09:14.594748+0000","last_unstale":"2024-05-20T22:09:15.573755+0000","last_undegraded":"2024-05-20T22:09:15.573755+0000","last_fullsized":"2024-05-20T22:09:15.573755+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:52:51.186712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.600522+0000","last_change":"2024-05-20T22:09:15.595806+0000","last_active":"2024-05-20T22:09:15.600522+0000","last_peered":"2024-05-20T22:09:15.600522+0000","last_clean":"2024-05-20T22:09:15.600522+0000","last_became_active":"2024-05-20T22:09:15.594138+0000","last_became_peered":"2024-05-20T22:09:15.594138+0000","last_unstale":"2024-05-20T22:09:15.600522+0000","last_undegraded":"2024-05-20T22:09:15.600522+0000","last_fullsized":"2024-05-20T22:09:15.600522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:50.583252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.606177+0000","last_change":"2024-05-20T22:09:14.585095+0000","last_active":"2024-05-20T22:09:15.606177+0000","last_peered":"2024-05-20T22:09:15.606177+0000","last_clean":"2024-05-20T22:09:15.606177+0000","last_became_active":"2024-05-20T22:09:14.583426+0000","last_became_peered":"2024-05-20T22:09:14.583426+0000","last_unstale":"2024-05-20T22:09:15.606177+0000","last_undegraded":"2024-05-20T22:09:15.606177+0000","last_fullsized":"2024-05-20T22:09:15.606177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:14:16.284004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.602975+0000","last_change":"2024-05-20T22:09:15.596852+0000","last_active":"2024-05-20T22:09:15.602975+0000","last_peered":"2024-05-20T22:09:15.602975+0000","last_clean":"2024-05-20T22:09:15.602975+0000","last_became_active":"2024-05-20T22:09:15.596479+0000","last_became_peered":"2024-05-20T22:09:15.596479+0000","last_unstale":"2024-05-20T22:09:15.602975+0000","last_undegraded":"2024-05-20T22:09:15.602975+0000","last_fullsized":"2024-05-20T22:09:15.602975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:02:41.889399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.574080+0000","last_change":"2024-05-20T22:09:14.587347+0000","last_active":"2024-05-20T22:09:15.574080+0000","last_peered":"2024-05-20T22:09:15.574080+0000","last_clean":"2024-05-20T22:09:15.574080+0000","last_became_active":"2024-05-20T22:09:14.587017+0000","last_became_peered":"2024-05-20T22:09:14.587017+0000","last_unstale":"2024-05-20T22:09:15.574080+0000","last_undegraded":"2024-05-20T22:09:15.574080+0000","last_fullsized":"2024-05-20T22:09:15.574080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:00:52.784682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.601435+0000","last_change":"2024-05-20T22:09:15.593597+0000","last_active":"2024-05-20T22:09:15.601435+0000","last_peered":"2024-05-20T22:09:15.601435+0000","last_clean":"2024-05-20T22:09:15.601435+0000","last_became_active":"2024-05-20T22:09:15.593293+0000","last_became_peered":"2024-05-20T22:09:15.593293+0000","last_unstale":"2024-05-20T22:09:15.601435+0000","last_undegraded":"2024-05-20T22:09:15.601435+0000","last_fullsized":"2024-05-20T22:09:15.601435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:15:11.782755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.583474+0000","last_change":"2024-05-20T22:09:14.600115+0000","last_active":"2024-05-20T22:09:15.583474+0000","last_peered":"2024-05-20T22:09:15.583474+0000","last_clean":"2024-05-20T22:09:15.583474+0000","last_became_active":"2024-05-20T22:09:14.599862+0000","last_became_peered":"2024-05-20T22:09:14.599862+0000","last_unstale":"2024-05-20T22:09:15.583474+0000","last_undegraded":"2024-05-20T22:09:15.583474+0000","last_fullsized":"2024-05-20T22:09:15.583474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:52:32.827860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599559+0000","last_change":"2024-05-20T22:09:15.597670+0000","last_active":"2024-05-20T22:09:15.599559+0000","last_peered":"2024-05-20T22:09:15.599559+0000","last_clean":"2024-05-20T22:09:15.599559+0000","last_became_active":"2024-05-20T22:09:15.597319+0000","last_became_peered":"2024-05-20T22:09:15.597319+0000","last_unstale":"2024-05-20T22:09:15.599559+0000","last_undegraded":"2024-05-20T22:09:15.599559+0000","last_fullsized":"2024-05-20T22:09:15.599559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:56:38.658039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.583914+0000","last_change":"2024-05-20T22:09:14.601476+0000","last_active":"2024-05-20T22:09:15.583914+0000","last_peered":"2024-05-20T22:09:15.583914+0000","last_clean":"2024-05-20T22:09:15.583914+0000","last_became_active":"2024-05-20T22:09:14.601122+0000","last_became_peered":"2024-05-20T22:09:14.601122+0000","last_unstale":"2024-05-20T22:09:15.583914+0000","last_undegraded":"2024-05-20T22:09:15.583914+0000","last_fullsized":"2024-05-20T22:09:15.583914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:10.759229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.589392+0000","last_change":"2024-05-20T22:09:15.587468+0000","last_active":"2024-05-20T22:09:15.589392+0000","last_peered":"2024-05-20T22:09:15.589392+0000","last_clean":"2024-05-20T22:09:15.589392+0000","last_became_active":"2024-05-20T22:09:15.587127+0000","last_became_peered":"2024-05-20T22:09:15.587127+0000","last_unstale":"2024-05-20T22:09:15.589392+0000","last_undegraded":"2024-05-20T22:09:15.589392+0000","last_fullsized":"2024-05-20T22:09:15.589392+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:49.594726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.579839+0000","last_change":"2024-05-20T22:09:14.595997+0000","last_active":"2024-05-20T22:09:15.579839+0000","last_peered":"2024-05-20T22:09:15.579839+0000","last_clean":"2024-05-20T22:09:15.579839+0000","last_became_active":"2024-05-20T22:09:14.595731+0000","last_became_peered":"2024-05-20T22:09:14.595731+0000","last_unstale":"2024-05-20T22:09:15.579839+0000","last_undegraded":"2024-05-20T22:09:15.579839+0000","last_fullsized":"2024-05-20T22:09:15.579839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:07:42.143440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.598317+0000","last_change":"2024-05-20T22:09:15.595473+0000","last_active":"2024-05-20T22:09:15.598317+0000","last_peered":"2024-05-20T22:09:15.598317+0000","last_clean":"2024-05-20T22:09:15.598317+0000","last_became_active":"2024-05-20T22:09:15.592918+0000","last_became_peered":"2024-05-20T22:09:15.592918+0000","last_unstale":"2024-05-20T22:09:15.598317+0000","last_undegraded":"2024-05-20T22:09:15.598317+0000","last_fullsized":"2024-05-20T22:09:15.598317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:14:17.302572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.587255+0000","last_change":"2024-05-20T22:09:14.593206+0000","last_active":"2024-05-20T22:09:15.587255+0000","last_peered":"2024-05-20T22:09:15.587255+0000","last_clean":"2024-05-20T22:09:15.587255+0000","last_became_active":"2024-05-20T22:09:14.592751+0000","last_became_peered":"2024-05-20T22:09:14.592751+0000","last_unstale":"2024-05-20T22:09:15.587255+0000","last_undegraded":"2024-05-20T22:09:15.587255+0000","last_fullsized":"2024-05-20T22:09:15.587255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:52:32.827860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.602749+0000","last_change":"2024-05-20T22:09:15.594116+0000","last_active":"2024-05-20T22:09:15.602749+0000","last_peered":"2024-05-20T22:09:15.602749+0000","last_clean":"2024-05-20T22:09:15.602749+0000","last_became_active":"2024-05-20T22:09:15.593808+0000","last_became_peered":"2024-05-20T22:09:15.593808+0000","last_unstale":"2024-05-20T22:09:15.602749+0000","last_undegraded":"2024-05-20T22:09:15.602749+0000","last_fullsized":"2024-05-20T22:09:15.602749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:22:03.955967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.580711+0000","last_change":"2024-05-20T22:09:14.594884+0000","last_active":"2024-05-20T22:09:15.580711+0000","last_peered":"2024-05-20T22:09:15.580711+0000","last_clean":"2024-05-20T22:09:15.580711+0000","last_became_active":"2024-05-20T22:09:14.594591+0000","last_became_peered":"2024-05-20T22:09:14.594591+0000","last_unstale":"2024-05-20T22:09:15.580711+0000","last_undegraded":"2024-05-20T22:09:15.580711+0000","last_fullsized":"2024-05-20T22:09:15.580711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:42:31.243332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.599046+0000","last_change":"2024-05-20T22:09:15.596333+0000","last_active":"2024-05-20T22:09:15.599046+0000","last_peered":"2024-05-20T22:09:15.599046+0000","last_clean":"2024-05-20T22:09:15.599046+0000","last_became_active":"2024-05-20T22:09:15.592862+0000","last_became_peered":"2024-05-20T22:09:15.592862+0000","last_unstale":"2024-05-20T22:09:15.599046+0000","last_undegraded":"2024-05-20T22:09:15.599046+0000","last_fullsized":"2024-05-20T22:09:15.599046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:14.555022+0000","last_clean_scrub_stamp":"2024-05-20T22:09:14.555022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:39:35.572524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.573987+0000","last_change":"2024-05-20T22:09:14.582811+0000","last_active":"2024-05-20T22:09:15.573987+0000","last_peered":"2024-05-20T22:09:15.573987+0000","last_clean":"2024-05-20T22:09:15.573987+0000","last_became_active":"2024-05-20T22:09:14.582488+0000","last_became_peered":"2024-05-20T22:09:14.582488+0000","last_unstale":"2024-05-20T22:09:15.573987+0000","last_undegraded":"2024-05-20T22:09:15.573987+0000","last_fullsized":"2024-05-20T22:09:15.573987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:09:13.536454+0000","last_clean_scrub_stamp":"2024-05-20T22:09:13.536454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:48.576157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-20T22:09:15.571920+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:09:15.571920+0000","last_peered":"2024-05-20T22:09:15.571920+0000","last_clean":"2024-05-20T22:09:15.571920+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:09:15.571920+0000","last_undegraded":"2024-05-20T22:09:15.571920+0000","last_fullsized":"2024-05-20T22:09:15.571920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":775386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":430080,"data_stored":183370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":773738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:09:20.037 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T22:09:20.037 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-20T22:09:20.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.192+0000 7fc20ed1a640 1 Processor -- start 2024-05-20T22:09:20.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.192+0000 7fc20ed1a640 1 -- start start 2024-05-20T22:09:20.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20ed1a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2080d32a0 0x7fc20815f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20ed1a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc2080d01b0 0x7fc20806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20ed1a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2080d3780 con 0x7fc2080cf690 2024-05-20T22:09:20.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20ed1a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fc2080d3a60 con 0x7fc2080d32a0 2024-05-20T22:09:20.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20ed1a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc2080d3d40 con 0x7fc2080d01b0 2024-05-20T22:09:20.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20cab6640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2080d32a0 0x7fc20815f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:20.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20cab6640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2080d32a0 0x7fc20815f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43236/0 (socket says 172.21.15.55:43236) 2024-05-20T22:09:20.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20cab6640 1 -- 172.21.15.55:0/43403021 learned_addr learned my addr 172.21.15.55:0/43403021 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:20.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20d2b7640 1 --2- 172.21.15.55:0/43403021 >> v2:172.21.15.138:3300/0 conn(0x7fc2080d01b0 0x7fc20806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:20.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20cab6640 1 -- 172.21.15.55:0/43403021 >> v2:172.21.15.138:3300/0 conn(0x7fc2080d01b0 msgr2=0x7fc20806b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:20.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20cab6640 1 --2- 172.21.15.55:0/43403021 >> v2:172.21.15.138:3300/0 conn(0x7fc2080d01b0 0x7fc20806b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20cab6640 1 -- 172.21.15.55:0/43403021 >> v1:172.21.15.138:6789/0 conn(0x7fc2080cf690 legacy=0x7fc2080cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:20.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.194+0000 7fc20cab6640 1 -- 172.21.15.55:0/43403021 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2080d4020 con 0x7fc2080d32a0 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.195+0000 7fc20cab6640 1 --2- 172.21.15.55:0/43403021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 0x7fc20815f150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc208163d70 tx=0x7fc1f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae2c95bf298f7e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.195+0000 7fc2077fe640 1 -- 172.21.15.55:0/43403021 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8037070 con 0x7fc2080d32a0 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.195+0000 7fc2077fe640 1 -- 172.21.15.55:0/43403021 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f8032440 con 0x7fc2080d32a0 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.195+0000 7fc2077fe640 1 -- 172.21.15.55:0/43403021 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8032810 con 0x7fc2080d32a0 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.196+0000 7fc20ed1a640 1 -- 172.21.15.55:0/43403021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 msgr2=0x7fc20815f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.196+0000 7fc20ed1a640 1 --2- 172.21.15.55:0/43403021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 0x7fc20815f150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc208163d70 tx=0x7fc1f802a230 comp rx=0 tx=0).stop 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.196+0000 7fc20ed1a640 1 -- 172.21.15.55:0/43403021 shutdown_connections 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.196+0000 7fc20ed1a640 1 --2- 172.21.15.55:0/43403021 >> v2:172.21.15.138:3300/0 conn(0x7fc2080d01b0 0x7fc20806b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.196+0000 7fc20ed1a640 1 --2- 172.21.15.55:0/43403021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 0x7fc20815f150 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.196+0000 7fc20ed1a640 1 -- 172.21.15.55:0/43403021 >> 172.21.15.55:0/43403021 conn(0x7fc2080b6de0 msgr2=0x7fc2080d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:20.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.196+0000 7fc20ed1a640 1 -- 172.21.15.55:0/43403021 shutdown_connections 2024-05-20T22:09:20.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.196+0000 7fc20ed1a640 1 -- 172.21.15.55:0/43403021 wait complete. 2024-05-20T22:09:20.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.198+0000 7fc20ed1a640 1 Processor -- start 2024-05-20T22:09:20.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.198+0000 7fc20ed1a640 1 -- start start 2024-05-20T22:09:20.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.199+0000 7fc20ed1a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2080cf690 0x7fc2080cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.199+0000 7fc20ed1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d01b0 0x7fc208153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.199+0000 7fc20ed1a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 0x7fc20815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.199+0000 7fc20ed1a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2080d4820 con 0x7fc2080d32a0 2024-05-20T22:09:20.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.199+0000 7fc20ed1a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2080b4350 con 0x7fc2080d01b0 2024-05-20T22:09:20.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.199+0000 7fc20ed1a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc208155750 con 0x7fc2080cf690 2024-05-20T22:09:20.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.199+0000 7fc20d2b7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 0x7fc20815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.199+0000 7fc20d2b7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 0x7fc20815a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43252/0 (socket says 172.21.15.55:43252) 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.199+0000 7fc20d2b7640 1 -- 172.21.15.55:0/4056416619 learned_addr learned my addr 172.21.15.55:0/4056416619 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.199+0000 7fc207fff640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d01b0 0x7fc208153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc20d2b7640 1 -- 172.21.15.55:0/4056416619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2080cf690 msgr2=0x7fc2080cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc20d2b7640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2080cf690 0x7fc2080cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc20d2b7640 1 -- 172.21.15.55:0/4056416619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d01b0 msgr2=0x7fc208153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc20d2b7640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d01b0 0x7fc208153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc20d2b7640 1 -- 172.21.15.55:0/4056416619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2080c9a70 con 0x7fc2080d32a0 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc207fff640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d01b0 0x7fc208153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc20d2b7640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 0x7fc20815a650 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc200009fd0 tx=0x7fc20000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc2057fa640 1 -- 172.21.15.55:0/4056416619 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200014420 con 0x7fc2080d32a0 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc2057fa640 1 -- 172.21.15.55:0/4056416619 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc200019070 con 0x7fc2080d32a0 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc20ed1a640 1 -- 172.21.15.55:0/4056416619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2081542b0 con 0x7fc2080d32a0 2024-05-20T22:09:20.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc2057fa640 1 -- 172.21.15.55:0/4056416619 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200014800 con 0x7fc2080d32a0 2024-05-20T22:09:20.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.200+0000 7fc20ed1a640 1 -- 172.21.15.55:0/4056416619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc208159b70 con 0x7fc2080d32a0 2024-05-20T22:09:20.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.201+0000 7fc20ed1a640 1 -- 172.21.15.55:0/4056416619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc208069090 con 0x7fc2080d32a0 2024-05-20T22:09:20.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.202+0000 7fc2057fa640 1 -- 172.21.15.55:0/4056416619 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2000149a0 con 0x7fc2080d32a0 2024-05-20T22:09:20.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.202+0000 7fc2057fa640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc1f406be30 0x7fc1f406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.202+0000 7fc2057fa640 1 -- 172.21.15.55:0/4056416619 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc20008b7e0 con 0x7fc2080d32a0 2024-05-20T22:09:20.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.202+0000 7fc20cab6640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc1f406be30 0x7fc1f406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:20.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.203+0000 7fc20cab6640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc1f406be30 0x7fc1f406e340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8034000 tx=0x7fc1f802c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:20.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.207+0000 7fc2057fa640 1 -- 172.21.15.55:0/4056416619 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc20005a000 con 0x7fc2080d32a0 2024-05-20T22:09:20.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.464+0000 7fc20ed1a640 1 -- 172.21.15.55:0/4056416619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc2080c97c0 con 0x7fc2080d32a0 2024-05-20T22:09:20.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.471+0000 7fc2057fa640 1 -- 172.21.15.55:0/4056416619 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fc200059900 con 0x7fc2080d32a0 2024-05-20T22:09:20.472 INFO:teuthology.orchestra.run.smithi055.stdout:[client.0] 2024-05-20T22:09:20.472 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB2yUtm592RAxAAiaDYBHgkpVfsQHm+jLOy3g== 2024-05-20T22:09:20.472 INFO:teuthology.orchestra.run.smithi055.stdout: caps mds = "allow" 2024-05-20T22:09:20.472 INFO:teuthology.orchestra.run.smithi055.stdout: caps mon = "allow r" 2024-05-20T22:09:20.472 INFO:teuthology.orchestra.run.smithi055.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-20T22:09:20.472 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2024-05-20T22:09:20.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 -- 172.21.15.55:0/4056416619 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc1f406be30 msgr2=0x7fc1f406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc1f406be30 0x7fc1f406e340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8034000 tx=0x7fc1f802c000 comp rx=0 tx=0).stop 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 -- 172.21.15.55:0/4056416619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 msgr2=0x7fc20815a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 0x7fc20815a650 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc200009fd0 tx=0x7fc20000e4b0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 -- 172.21.15.55:0/4056416619 shutdown_connections 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc1f406be30 0x7fc1f406e340 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2080d32a0 0x7fc20815a650 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d01b0 0x7fc208153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 --2- 172.21.15.55:0/4056416619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2080cf690 0x7fc2080cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 -- 172.21.15.55:0/4056416619 >> 172.21.15.55:0/4056416619 conn(0x7fc2080b6de0 msgr2=0x7fc20815d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 -- 172.21.15.55:0/4056416619 shutdown_connections 2024-05-20T22:09:20.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.473+0000 7fc20ed1a640 1 -- 172.21.15.55:0/4056416619 wait complete. 2024-05-20T22:09:20.487 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:09:20.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.601+0000 7f0b8592c640 1 Processor -- start 2024-05-20T22:09:20.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.601+0000 7f0b8592c640 1 -- start start 2024-05-20T22:09:20.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.602+0000 7f0b8592c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b8006e520 0x7f0b800d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.602+0000 7f0b8592c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0b800d1d70 0x7f0b800b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.602+0000 7f0b8592c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b800d2180 con 0x7f0b8006dfd0 2024-05-20T22:09:20.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.602+0000 7f0b8592c640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f0b8006bd90 con 0x7f0b800d1d70 2024-05-20T22:09:20.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.602+0000 7f0b8592c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0b800cd330 con 0x7f0b8006e520 2024-05-20T22:09:20.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.602+0000 7f0b7f7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0b800d1d70 0x7f0b800b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:20.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.602+0000 7f0b7f7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0b800d1d70 0x7f0b800b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43264/0 (socket says 172.21.15.55:43264) 2024-05-20T22:09:20.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.602+0000 7f0b7f7fe640 1 -- 172.21.15.55:0/1855765001 learned_addr learned my addr 172.21.15.55:0/1855765001 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:20.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.602+0000 7f0b7effd640 1 --2- 172.21.15.55:0/1855765001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b8006e520 0x7f0b800d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:20.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.603+0000 7f0b7f7fe640 1 -- 172.21.15.55:0/1855765001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b8006e520 msgr2=0x7f0b800d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:20.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.603+0000 7f0b7f7fe640 1 --2- 172.21.15.55:0/1855765001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b8006e520 0x7f0b800d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.603+0000 7f0b7f7fe640 1 -- 172.21.15.55:0/1855765001 >> v1:172.21.15.138:6789/0 conn(0x7f0b8006dfd0 legacy=0x7f0b800b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:20.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.603+0000 7f0b7f7fe640 1 -- 172.21.15.55:0/1855765001 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b800b2720 con 0x7f0b800d1d70 2024-05-20T22:09:20.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.603+0000 7f0b7effd640 1 --2- 172.21.15.55:0/1855765001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b8006e520 0x7f0b800d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:20.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.603+0000 7f0b7f7fe640 1 --2- 172.21.15.55:0/1855765001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 0x7f0b800b1860 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009e50 tx=0x7f0b7802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898a6c68177ee6b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:20.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b7dffb640 1 -- 172.21.15.55:0/1855765001 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78036070 con 0x7f0b800d1d70 2024-05-20T22:09:20.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b7dffb640 1 -- 172.21.15.55:0/1855765001 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b78031600 con 0x7f0b800d1d70 2024-05-20T22:09:20.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b7dffb640 1 -- 172.21.15.55:0/1855765001 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780319d0 con 0x7f0b800d1d70 2024-05-20T22:09:20.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b8592c640 1 -- 172.21.15.55:0/1855765001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 msgr2=0x7f0b800b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:20.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b8592c640 1 --2- 172.21.15.55:0/1855765001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 0x7f0b800b1860 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009e50 tx=0x7f0b7802a6c0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b8592c640 1 -- 172.21.15.55:0/1855765001 shutdown_connections 2024-05-20T22:09:20.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b8592c640 1 --2- 172.21.15.55:0/1855765001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 0x7f0b800b1860 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b8592c640 1 --2- 172.21.15.55:0/1855765001 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b8006e520 0x7f0b800d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b8592c640 1 -- 172.21.15.55:0/1855765001 >> 172.21.15.55:0/1855765001 conn(0x7f0b800b6580 msgr2=0x7f0b800b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:20.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b8592c640 1 -- 172.21.15.55:0/1855765001 shutdown_connections 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.604+0000 7f0b8592c640 1 -- 172.21.15.55:0/1855765001 wait complete. 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.606+0000 7f0b8592c640 1 Processor -- start 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.606+0000 7f0b8592c640 1 -- start start 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b8592c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b8006dfd0 0x7f0b8014c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b8592c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b8006e520 0x7f0b8014c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b8592c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 0x7f0b8015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b8592c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0b800cd800 con 0x7f0b800d1d70 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b8592c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b800cd560 con 0x7f0b8006e520 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b8592c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8015d4f0 con 0x7f0b8006dfd0 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b7f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 0x7f0b8015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:20.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b7f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 0x7f0b8015a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43268/0 (socket says 172.21.15.55:43268) 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b7f7fe640 1 -- 172.21.15.55:0/2024686112 learned_addr learned my addr 172.21.15.55:0/2024686112 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b7f7fe640 1 -- 172.21.15.55:0/2024686112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b8006dfd0 msgr2=0x7f0b8014c420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b7f7fe640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b8006dfd0 0x7f0b8014c420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b7f7fe640 1 -- 172.21.15.55:0/2024686112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b8006e520 msgr2=0x7f0b8014c960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b7f7fe640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b8006e520 0x7f0b8014c960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b7f7fe640 1 -- 172.21.15.55:0/2024686112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b800cb6c0 con 0x7f0b800d1d70 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b7f7fe640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 0x7f0b8015a3e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0b78034040 tx=0x7f0b78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.608+0000 7f0b77fff640 1 -- 172.21.15.55:0/2024686112 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78036050 con 0x7f0b800d1d70 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.609+0000 7f0b8592c640 1 -- 172.21.15.55:0/2024686112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b800c8f30 con 0x7f0b800d1d70 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.609+0000 7f0b8592c640 1 -- 172.21.15.55:0/2024686112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b801605e0 con 0x7f0b800d1d70 2024-05-20T22:09:20.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.610+0000 7f0b8592c640 1 -- 172.21.15.55:0/2024686112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b80069090 con 0x7f0b800d1d70 2024-05-20T22:09:20.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.614+0000 7f0b77fff640 1 -- 172.21.15.55:0/2024686112 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b78042070 con 0x7f0b800d1d70 2024-05-20T22:09:20.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.614+0000 7f0b77fff640 1 -- 172.21.15.55:0/2024686112 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78031600 con 0x7f0b800d1d70 2024-05-20T22:09:20.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.616+0000 7f0b77fff640 1 -- 172.21.15.55:0/2024686112 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b780079b0 con 0x7f0b800d1d70 2024-05-20T22:09:20.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.616+0000 7f0b77fff640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b6006c000 0x7f0b6006e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:20.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.616+0000 7f0b77fff640 1 -- 172.21.15.55:0/2024686112 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0b780aca00 con 0x7f0b800d1d70 2024-05-20T22:09:20.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.616+0000 7f0b7effd640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b6006c000 0x7f0b6006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:20.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.617+0000 7f0b7effd640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b6006c000 0x7f0b6006e510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b8009f9e0 tx=0x7f0b70008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:20.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.620+0000 7f0b77fff640 1 -- 172.21.15.55:0/2024686112 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0b800d2180 con 0x7f0b800d1d70 2024-05-20T22:09:20.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.842+0000 7f0b8592c640 1 -- 172.21.15.55:0/2024686112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b800cb430 con 0x7f0b800d1d70 2024-05-20T22:09:20.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.842+0000 7f0b77fff640 1 -- 172.21.15.55:0/2024686112 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f0b7807b190 con 0x7f0b800d1d70 2024-05-20T22:09:20.843 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:20.843 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/1557521986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1557521986},{"type":"v1","addr":"172.21.15.138:6834","nonce":1557521986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3642520472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3642520472},{"type":"v1","addr":"172.21.15.55:6836","nonce":3642520472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":34976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/2179562051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":2179562051},{"type":"v1","addr":"172.21.15.55:6837","nonce":2179562051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:09:14.957632+0000","modified":"2024-05-20T22:09:17.975726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34979},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34979":{"gid":34979,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/4137968836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":4137968836},{"type":"v1","addr":"172.21.15.138:6835","nonce":4137968836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34979,"qdb_cluster":[34979]},"id":2}]} 2024-05-20T22:09:20.844 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 18 2024-05-20T22:09:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 -- 172.21.15.55:0/2024686112 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b6006c000 msgr2=0x7f0b6006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b6006c000 0x7f0b6006e510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b8009f9e0 tx=0x7f0b70008040 comp rx=0 tx=0).stop 2024-05-20T22:09:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 -- 172.21.15.55:0/2024686112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 msgr2=0x7f0b8015a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 0x7f0b8015a3e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0b78034040 tx=0x7f0b78004a40 comp rx=0 tx=0).stop 2024-05-20T22:09:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 -- 172.21.15.55:0/2024686112 shutdown_connections 2024-05-20T22:09:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0b6006c000 0x7f0b6006e510 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b800d1d70 0x7f0b8015a3e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b8006e520 0x7f0b8014c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 --2- 172.21.15.55:0/2024686112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b8006dfd0 0x7f0b8014c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 -- 172.21.15.55:0/2024686112 >> 172.21.15.55:0/2024686112 conn(0x7f0b800b6580 msgr2=0x7f0b800b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:20.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 -- 172.21.15.55:0/2024686112 shutdown_connections 2024-05-20T22:09:20.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:20.844+0000 7f0b8592c640 1 -- 172.21.15.55:0/2024686112 wait complete. 2024-05-20T22:09:21.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:09:14.957632+0000', 'modified': '2024-05-20T22:09:17.975726+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34979}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34979': {'gid': 34979, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6835/4137968836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6833', 'nonce': 4137968836}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 4137968836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34979, 'qdb_cluster': [34979]} 2024-05-20T22:09:21.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T22:09:21.260 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-20T22:09:21.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.378+0000 7f6d0ef21640 1 Processor -- start 2024-05-20T22:09:21.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.378+0000 7f6d0ef21640 1 -- start start 2024-05-20T22:09:21.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.379+0000 7f6d0ef21640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6d08167ff0 0x7f6d0816a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.379+0000 7f6d0ef21640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d080d04b0 con 0x7f6d0815f150 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.379+0000 7f6d0ef21640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d080d3780 con 0x7f6d080d32a0 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.379+0000 7f6d0ef21640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6d0806c510 con 0x7f6d08167ff0 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.380+0000 7f6d07fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6d0815f150 0x7f6d080cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49630/0 (socket says 172.21.15.55:49630) 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.380+0000 7f6d07fff640 1 -- 172.21.15.55:0/2907136239 learned_addr learned my addr 172.21.15.55:0/2907136239 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.380+0000 7f6d0d4be640 1 --2- 172.21.15.55:0/2907136239 >> v2:172.21.15.138:3300/0 conn(0x7f6d08167ff0 0x7f6d0816a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.380+0000 7f6d077fe640 1 -- 172.21.15.55:0/2907136239 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2676256927 0 0) 0x7f6d080d04b0 con 0x7f6d0815f150 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.380+0000 7f6d077fe640 1 -- 172.21.15.55:0/2907136239 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cf4003520 con 0x7f6d0815f150 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.380+0000 7f6d0d4be640 1 -- 172.21.15.55:0/2907136239 >> v1:172.21.15.138:6789/0 conn(0x7f6d080d32a0 legacy=0x7f6d080cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.380+0000 7f6d0d4be640 1 -- 172.21.15.55:0/2907136239 >> v1:172.21.15.55:6789/0 conn(0x7f6d0815f150 legacy=0x7f6d080cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.380+0000 7f6d0d4be640 1 -- 172.21.15.55:0/2907136239 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d080d3a60 con 0x7f6d08167ff0 2024-05-20T22:09:21.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.381+0000 7f6d0d4be640 1 --2- 172.21.15.55:0/2907136239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08167ff0 0x7f6d0816a410 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009d90 tx=0x7f6d0002a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=443da09442fd7360 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:21.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.381+0000 7f6d077fe640 1 -- 172.21.15.55:0/2907136239 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00007740 con 0x7f6d08167ff0 2024-05-20T22:09:21.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.381+0000 7f6d077fe640 1 -- 172.21.15.55:0/2907136239 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d000078e0 con 0x7f6d08167ff0 2024-05-20T22:09:21.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.381+0000 7f6d077fe640 1 -- 172.21.15.55:0/2907136239 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00007c90 con 0x7f6d08167ff0 2024-05-20T22:09:21.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.382+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2907136239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08167ff0 msgr2=0x7f6d0816a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.382+0000 7f6d0ef21640 1 --2- 172.21.15.55:0/2907136239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08167ff0 0x7f6d0816a410 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009d90 tx=0x7f6d0002a6a0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.382+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2907136239 shutdown_connections 2024-05-20T22:09:21.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.382+0000 7f6d0ef21640 1 --2- 172.21.15.55:0/2907136239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08167ff0 0x7f6d0816a410 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.382+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2907136239 >> 172.21.15.55:0/2907136239 conn(0x7f6d080b6de0 msgr2=0x7f6d080d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:21.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.382+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2907136239 shutdown_connections 2024-05-20T22:09:21.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.382+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2907136239 wait complete. 2024-05-20T22:09:21.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.384+0000 7f6d0ef21640 1 Processor -- start 2024-05-20T22:09:21.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.384+0000 7f6d0ef21640 1 -- start start 2024-05-20T22:09:21.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d0ef21640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d080d32a0 0x7f6d0815f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d0ef21640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d0815f150 0x7f6d081518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d0ef21640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08167ff0 0x7f6d080b9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d0ef21640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6d080d4300 con 0x7f6d0815f150 2024-05-20T22:09:21.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d0ef21640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d080d4020 con 0x7f6d08167ff0 2024-05-20T22:09:21.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d0ef21640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0815f560 con 0x7f6d080d32a0 2024-05-20T22:09:21.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d0815f150 0x7f6d081518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d0815f150 0x7f6d081518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43276/0 (socket says 172.21.15.55:43276) 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d07fff640 1 -- 172.21.15.55:0/2625929216 learned_addr learned my addr 172.21.15.55:0/2625929216 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d080d32a0 0x7f6d0815f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.385+0000 7f6d0d4be640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08167ff0 0x7f6d080b9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d07fff640 1 -- 172.21.15.55:0/2625929216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d080d32a0 msgr2=0x7f6d0815f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d07fff640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d080d32a0 0x7f6d0815f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d07fff640 1 -- 172.21.15.55:0/2625929216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08167ff0 msgr2=0x7f6d080b9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d07fff640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08167ff0 0x7f6d080b9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d07fff640 1 -- 172.21.15.55:0/2625929216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d080b8ea0 con 0x7f6d0815f150 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d0d4be640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08167ff0 0x7f6d080b9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d080d32a0 0x7f6d0815f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d07fff640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d0815f150 0x7f6d081518d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000c3c0 tx=0x7f6cf000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d057fa640 1 -- 172.21.15.55:0/2625929216 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0017070 con 0x7f6d0815f150 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d057fa640 1 -- 172.21.15.55:0/2625929216 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf0005ce0 con 0x7f6d0815f150 2024-05-20T22:09:21.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2625929216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d080b9180 con 0x7f6d0815f150 2024-05-20T22:09:21.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d057fa640 1 -- 172.21.15.55:0/2625929216 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf00072f0 con 0x7f6d0815f150 2024-05-20T22:09:21.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.386+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2625929216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d080b9740 con 0x7f6d0815f150 2024-05-20T22:09:21.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.388+0000 7f6d057fa640 1 -- 172.21.15.55:0/2625929216 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6cf001a070 con 0x7f6d0815f150 2024-05-20T22:09:21.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.388+0000 7f6d057fa640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ce806be60 0x7f6ce806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.389+0000 7f6d057fa640 1 -- 172.21.15.55:0/2625929216 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f6cf008a460 con 0x7f6d0815f150 2024-05-20T22:09:21.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.389+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ce806be60 0x7f6ce806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.389+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ce806be60 0x7f6ce806e370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8006fd0 tx=0x7f6cf8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:21.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.389+0000 7f6d15dc7740 1 Processor -- start 2024-05-20T22:09:21.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.390+0000 7f6d15dc7740 1 -- start start 2024-05-20T22:09:21.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb119c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb11990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb1172360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564fb10d54d0 con 0x564fb0db0590 2024-05-20T22:09:21.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564fb107d3e0 con 0x564fb0da83b0 2024-05-20T22:09:21.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564fb10d1490 con 0x564fb0dd1270 2024-05-20T22:09:21.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d0d4be640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb1172360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d0d4be640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb1172360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43290/0 (socket says 172.21.15.55:43290) 2024-05-20T22:09:21.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d0d4be640 1 -- 172.21.15.55:0/3823338553 learned_addr learned my addr 172.21.15.55:0/3823338553 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:21.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d07fff640 1 --2- 172.21.15.55:0/3823338553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb11990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/3823338553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb119c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d0d4be640 1 -- 172.21.15.55:0/3823338553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 msgr2=0x564fb119c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d0d4be640 1 --2- 172.21.15.55:0/3823338553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb119c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d0d4be640 1 -- 172.21.15.55:0/3823338553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 msgr2=0x564fb11990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d0d4be640 1 --2- 172.21.15.55:0/3823338553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb11990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d0d4be640 1 -- 172.21.15.55:0/3823338553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb0de92d0 con 0x564fb0db0590 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d07fff640 1 --2- 172.21.15.55:0/3823338553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb11990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.391+0000 7f6d0d4be640 1 --2- 172.21.15.55:0/3823338553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb1172360 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009d90 tx=0x7f6d0002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55a53d1c503c9fb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6cfe7fc640 1 -- 172.21.15.55:0/3823338553 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00007940 con 0x564fb0db0590 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6cfe7fc640 1 -- 172.21.15.55:0/3823338553 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d00007ae0 con 0x564fb0db0590 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6cfe7fc640 1 -- 172.21.15.55:0/3823338553 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00046430 con 0x564fb0db0590 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6d15dc7740 1 -- 172.21.15.55:0/3823338553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 msgr2=0x564fb1172360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/3823338553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb1172360 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009d90 tx=0x7f6d0002e000 comp rx=0 tx=0).stop 2024-05-20T22:09:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6d15dc7740 1 -- 172.21.15.55:0/3823338553 shutdown_connections 2024-05-20T22:09:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/3823338553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb1172360 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/3823338553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb11990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/3823338553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb119c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6d15dc7740 1 -- 172.21.15.55:0/3823338553 >> 172.21.15.55:0/3823338553 conn(0x564fb0dc53c0 msgr2=0x564fb1171af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6d15dc7740 1 -- 172.21.15.55:0/3823338553 shutdown_connections 2024-05-20T22:09:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.392+0000 7f6d15dc7740 1 -- 172.21.15.55:0/3823338553 wait complete. 2024-05-20T22:09:21.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.393+0000 7f6d15dc7740 1 -- start start 2024-05-20T22:09:21.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.394+0000 7f6d15dc7740 1 Processor -- start 2024-05-20T22:09:21.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb11b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb1198920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564fb10908e0 con 0x564fb0db0590 2024-05-20T22:09:21.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564fb11b7290 con 0x564fb0da83b0 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564fb11b5d00 con 0x564fb0dd1270 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d15dc7740 10 client.0 _ll_register_callbacks cb 0x564fb105d9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43302/0 (socket says 172.21.15.55:43302) 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d07fff640 1 -- 172.21.15.55:0/2433084502 learned_addr learned my addr 172.21.15.55:0/2433084502 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb11b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d0d4be640 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb1198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d07fff640 1 -- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 msgr2=0x564fb1198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d07fff640 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb1198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d07fff640 1 -- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 msgr2=0x564fb11b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d07fff640 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb11b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d07fff640 1 -- 172.21.15.55:0/2433084502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb0eb36b0 con 0x564fb0db0590 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb11b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.395+0000 7f6d07fff640 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11983e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0060b90 tx=0x7f6cf0058c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948ea62cf04773ef server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:21.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6ce77fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6ce77fe640 1 -- 172.21.15.55:0/2433084502 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf00610c0 con 0x564fb0db0590 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6ce77fe640 1 -- 172.21.15.55:0/2433084502 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f6cf008a0b0 con 0x564fb0db0590 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6d15dc7740 1 -- 172.21.15.55:0/2433084502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d00007ae0 con 0x564fb0db0590 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6ce77fe640 1 -- 172.21.15.55:0/2433084502 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6cf0007490 con 0x564fb0db0590 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6d15dc7740 10 client.5718 fetch_fsmap learned FSMap version 18 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6d15dc7740 1 -- 172.21.15.55:0/2433084502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x564fb11b6fd0 con 0x564fb0db0590 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6ce77fe640 1 -- 172.21.15.55:0/2433084502 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d00007ae0 con 0x564fb0db0590 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6ce77fe640 1 -- 172.21.15.55:0/2433084502 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f6cf00772f0 con 0x564fb0db0590 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6d15dc7740 10 client.5718 fetch_fsmap finished waiting for FSMap version 18 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6d15dc7740 10 client.5718 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6d15dc7740 20 client.5718 populate_metadata read hostname 'smithi055' 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x564fb0d8fe40 0x564fb11b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6d15dc7740 4 client.5718 mds_command: new command op to 34979 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T22:09:21.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.396+0000 7f6d15dc7740 1 -- 172.21.15.55:0/2433084502 --> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fb10e2c00 con 0x564fb0d8fe40 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.397+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x564fb0d8fe40 0x564fb11b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.397+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x564fb0d8fe40 0x564fb11b6400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dfde1c495ed7f8d server_cookie=12feea9defbf00cf in_seq=0 out_seq=0 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.397+0000 7f6ce77fe640 10 client.5718 ms_handle_connect on v2:172.21.15.138:6833/4137968836 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.398+0000 7f6ce77fe640 1 -- 172.21.15.55:0/2433084502 <== mds.0 v2:172.21.15.138:6833/4137968836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564fb10e2c00 con 0x564fb0d8fe40 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.398+0000 7f6ce77fe640 10 client.5718 handle_command_reply: tid=0 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.398+0000 7f6d15dc7740 1 client.5718 shutdown 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 1 -- 172.21.15.55:0/2433084502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 msgr2=0x564fb11983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11983e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0060b90 tx=0x7f6cf0058c90 comp rx=0 tx=0).stop 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 1 -- 172.21.15.55:0/2433084502 shutdown_connections 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb1198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11983e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x564fb0d8fe40 0x564fb11b6400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/2433084502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb11b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6ce77fe640 0 client.5718 ms_handle_reset on v2:172.21.15.138:6833/4137968836 2024-05-20T22:09:21.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 1 -- 172.21.15.55:0/2433084502 >> 172.21.15.55:0/2433084502 conn(0x564fb0dc53c0 msgr2=0x564fb116be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:21.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 1 -- 172.21.15.55:0/2433084502 shutdown_connections 2024-05-20T22:09:21.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 1 -- 172.21.15.55:0/2433084502 wait complete. 2024-05-20T22:09:21.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.399+0000 7f6d15dc7740 20 client.5718 trim_cache size 0 max 16384 2024-05-20T22:09:21.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.418+0000 7f6d15dc7740 1 Processor -- start 2024-05-20T22:09:21.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.418+0000 7f6d15dc7740 1 -- start start 2024-05-20T22:09:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.419+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0da83b0 0x564fb119bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.419+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0db0590 0x564fb11b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.419+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb11b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.419+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08201090 con 0x564fb0da83b0 2024-05-20T22:09:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.419+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d080c9cb0 con 0x564fb0db0590 2024-05-20T22:09:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.419+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6d08156fe0 con 0x564fb0dd1270 2024-05-20T22:09:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.419+0000 7f6d0ccbd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0da83b0 0x564fb119bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.419+0000 7f6d0ccbd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0da83b0 0x564fb119bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43304/0 (socket says 172.21.15.55:43304) 2024-05-20T22:09:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.419+0000 7f6d0ccbd640 1 -- 172.21.15.55:0/4282486342 learned_addr learned my addr 172.21.15.55:0/4282486342 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6d0ccbd640 1 -- 172.21.15.55:0/4282486342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 msgr2=0x564fb11b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6d07fff640 1 --2- 172.21.15.55:0/4282486342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0db0590 0x564fb11b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6d0d4be640 1 --2- 172.21.15.55:0/4282486342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb11b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/4282486342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb11b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6d0ccbd640 1 -- 172.21.15.55:0/4282486342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0db0590 msgr2=0x564fb11b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/4282486342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0db0590 0x564fb11b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6d0ccbd640 1 -- 172.21.15.55:0/4282486342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb116b220 con 0x564fb0da83b0 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6d07fff640 1 --2- 172.21.15.55:0/4282486342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0db0590 0x564fb11b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/4282486342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0da83b0 0x564fb119bbc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x564fb119e460 tx=0x7f6cf8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be820f819f97215 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6cfdffb640 1 -- 172.21.15.55:0/4282486342 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf801d040 con 0x564fb0da83b0 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6cfdffb640 1 -- 172.21.15.55:0/4282486342 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf80149c0 con 0x564fb0da83b0 2024-05-20T22:09:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.420+0000 7f6cfdffb640 1 -- 172.21.15.55:0/4282486342 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8014d90 con 0x564fb0da83b0 2024-05-20T22:09:21.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.421+0000 7f6d15dc7740 1 -- 172.21.15.55:0/4282486342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0da83b0 msgr2=0x564fb119bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.421+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/4282486342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0da83b0 0x564fb119bbc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x564fb119e460 tx=0x7f6cf8013000 comp rx=0 tx=0).stop 2024-05-20T22:09:21.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.421+0000 7f6d15dc7740 1 -- 172.21.15.55:0/4282486342 shutdown_connections 2024-05-20T22:09:21.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.421+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/4282486342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb11b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.421+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/4282486342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0db0590 0x564fb11b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.421+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/4282486342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0da83b0 0x564fb119bbc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.421+0000 7f6d15dc7740 1 -- 172.21.15.55:0/4282486342 >> 172.21.15.55:0/4282486342 conn(0x564fb0dc53c0 msgr2=0x564fb11987e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:21.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.421+0000 7f6d15dc7740 1 -- 172.21.15.55:0/4282486342 shutdown_connections 2024-05-20T22:09:21.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.421+0000 7f6d15dc7740 1 -- 172.21.15.55:0/4282486342 wait complete. 2024-05-20T22:09:21.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.422+0000 7f6d15dc7740 1 -- start start 2024-05-20T22:09:21.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.422+0000 7f6d15dc7740 1 Processor -- start 2024-05-20T22:09:21.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb1234b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d15dc7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb11b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564fb11c9b90 con 0x564fb0db0590 2024-05-20T22:09:21.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564fb119fe10 con 0x564fb0da83b0 2024-05-20T22:09:21.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d15dc7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564fb11ca1b0 con 0x564fb0dd1270 2024-05-20T22:09:21.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d15dc7740 10 client.0 _ll_register_callbacks cb 0x7f6cf0005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:09:21.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43316/0 (socket says 172.21.15.55:43316) 2024-05-20T22:09:21.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d07fff640 1 -- 172.21.15.55:0/424979886 learned_addr learned my addr 172.21.15.55:0/424979886 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:21.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb1234b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.423+0000 7f6d0d4be640 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb11b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d07fff640 1 -- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 msgr2=0x564fb11b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d07fff640 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb11b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d07fff640 1 -- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 msgr2=0x564fb1234b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d07fff640 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb1234b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d07fff640 1 -- 172.21.15.55:0/424979886 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb1197cc0 con 0x564fb0db0590 2024-05-20T22:09:21.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d07fff640 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11b5390 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0058d00 tx=0x7f6cf005b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829cc311f924eab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:21.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6ce77fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:09:21.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6ce77fe640 1 -- 172.21.15.55:0/424979886 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0057db0 con 0x564fb0db0590 2024-05-20T22:09:21.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6ce77fe640 1 -- 172.21.15.55:0/424979886 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f6cf0088390 con 0x564fb0db0590 2024-05-20T22:09:21.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d15dc7740 1 -- 172.21.15.55:0/424979886 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf80149c0 con 0x564fb0db0590 2024-05-20T22:09:21.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6ce77fe640 1 -- 172.21.15.55:0/424979886 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6cf00610c0 con 0x564fb0db0590 2024-05-20T22:09:21.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d15dc7740 10 client.5724 fetch_fsmap learned FSMap version 18 2024-05-20T22:09:21.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d15dc7740 1 -- 172.21.15.55:0/424979886 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x564fb1197cc0 con 0x564fb0db0590 2024-05-20T22:09:21.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6ce77fe640 1 -- 172.21.15.55:0/424979886 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf80149c0 con 0x564fb0db0590 2024-05-20T22:09:21.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6ce77fe640 1 -- 172.21.15.55:0/424979886 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f6cf0088b80 con 0x564fb0db0590 2024-05-20T22:09:21.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d15dc7740 10 client.5724 fetch_fsmap finished waiting for FSMap version 18 2024-05-20T22:09:21.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d15dc7740 10 client.5724 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T22:09:21.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d15dc7740 20 client.5724 populate_metadata read hostname 'smithi055' 2024-05-20T22:09:21.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x564fb11c4f60 0x564fb1237580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d15dc7740 4 client.5724 mds_command: new command op to 34979 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T22:09:21.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.424+0000 7f6d15dc7740 1 -- 172.21.15.55:0/424979886 --> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] -- command(tid 0: {"prefix": "status"}) -- 0x564fb10dcf50 con 0x564fb11c4f60 2024-05-20T22:09:21.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.425+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x564fb11c4f60 0x564fb1237580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.425+0000 7f6d0ccbd640 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x564fb11c4f60 0x564fb1237580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd8a24f71adc77b8 server_cookie=d55f5064f3b4578d in_seq=0 out_seq=0 2024-05-20T22:09:21.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.425+0000 7f6ce77fe640 10 client.5724 ms_handle_connect on v2:172.21.15.138:6833/4137968836 2024-05-20T22:09:21.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.425+0000 7f6ce77fe640 1 -- 172.21.15.55:0/424979886 <== mds.0 v2:172.21.15.138:6833/4137968836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564fb10dcf50 con 0x564fb11c4f60 2024-05-20T22:09:21.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.425+0000 7f6ce77fe640 10 client.5724 handle_command_reply: tid=0 2024-05-20T22:09:21.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.425+0000 7f6d15dc7740 1 client.5724 shutdown 2024-05-20T22:09:21.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 1 -- 172.21.15.55:0/424979886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 msgr2=0x564fb11b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11b5390 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0058d00 tx=0x7f6cf005b040 comp rx=0 tx=0).stop 2024-05-20T22:09:21.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 1 -- 172.21.15.55:0/424979886 shutdown_connections 2024-05-20T22:09:21.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fb0dd1270 0x564fb11b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fb0db0590 0x564fb11b5390 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x564fb11c4f60 0x564fb1237580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 1 --2- 172.21.15.55:0/424979886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fb0da83b0 0x564fb1234b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 1 -- 172.21.15.55:0/424979886 >> 172.21.15.55:0/424979886 conn(0x564fb0dc53c0 msgr2=0x564fb1170560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:21.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6ce77fe640 0 client.5724 ms_handle_reset on v2:172.21.15.138:6833/4137968836 2024-05-20T22:09:21.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 1 -- 172.21.15.55:0/424979886 shutdown_connections 2024-05-20T22:09:21.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 1 -- 172.21.15.55:0/424979886 wait complete. 2024-05-20T22:09:21.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d15dc7740 20 client.5724 trim_cache size 0 max 16384 2024-05-20T22:09:21.437 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:09:21.438 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "de8ca569-3fd1-4cb2-8bdc-99f367745927", 2024-05-20T22:09:21.438 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-20T22:09:21.438 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 34979, 2024-05-20T22:09:21.438 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-20T22:09:21.438 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-20T22:09:21.438 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-20T22:09:21.439 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 5.8465447939999997, 2024-05-20T22:09:21.439 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 18, 2024-05-20T22:09:21.439 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 38, 2024-05-20T22:09:21.439 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 38, 2024-05-20T22:09:21.439 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 13.769125538000001 2024-05-20T22:09:21.439 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:09:21.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2625929216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ce806be60 msgr2=0x7f6ce806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d0ef21640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ce806be60 0x7f6ce806e370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8006fd0 tx=0x7f6cf8008040 comp rx=0 tx=0).stop 2024-05-20T22:09:21.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2625929216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d0815f150 msgr2=0x7f6d081518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.426+0000 7f6d0ef21640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d0815f150 0x7f6d081518d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000c3c0 tx=0x7f6cf000c880 comp rx=0 tx=0).stop 2024-05-20T22:09:21.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.427+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2625929216 shutdown_connections 2024-05-20T22:09:21.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.427+0000 7f6d0ef21640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6ce806be60 0x7f6ce806e370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.427+0000 7f6d0ef21640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08167ff0 0x7f6d080b9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.427+0000 7f6d0ef21640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d0815f150 0x7f6d081518d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.427+0000 7f6d0ef21640 1 --2- 172.21.15.55:0/2625929216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d080d32a0 0x7f6d0815f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.427+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2625929216 >> 172.21.15.55:0/2625929216 conn(0x7f6d080b6de0 msgr2=0x7f6d080d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:21.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.427+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2625929216 shutdown_connections 2024-05-20T22:09:21.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.427+0000 7f6d0ef21640 1 -- 172.21.15.55:0/2625929216 wait complete. 2024-05-20T22:09:21.444 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T22:09:21.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.559+0000 7fe2995c2640 1 Processor -- start 2024-05-20T22:09:21.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.559+0000 7fe2995c2640 1 -- start start 2024-05-20T22:09:21.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.560+0000 7fe2995c2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2940b8190 0x7fe29406d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.560+0000 7fe2995c2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe294163ce0 0x7fe2940ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.560+0000 7fe2995c2640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe29406bf60 con 0x7fe29406d5c0 2024-05-20T22:09:21.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.560+0000 7fe2995c2640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe29409bca0 con 0x7fe294163ce0 2024-05-20T22:09:21.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.560+0000 7fe2995c2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe2940d1d20 con 0x7fe2940b8190 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.560+0000 7fe2927fc640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe29406d5c0 0x7fe2941635d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49644/0 (socket says 172.21.15.55:49644) 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.560+0000 7fe2927fc640 1 -- 172.21.15.55:0/2026541454 learned_addr learned my addr 172.21.15.55:0/2026541454 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.560+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1943239699 0 0) 0x7fe29406bf60 con 0x7fe29406d5c0 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe280003520 con 0x7fe29406d5c0 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 244518429 0 0) 0x7fe280003520 con 0x7fe29406d5c0 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2940b8190 msgr2=0x7fe29406d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2940b8190 0x7fe29406d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 >> v2:172.21.15.138:3300/0 conn(0x7fe294163ce0 msgr2=0x7fe2940ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 --2- 172.21.15.55:0/2026541454 >> v2:172.21.15.138:3300/0 conn(0x7fe294163ce0 0x7fe2940ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29409c130 con 0x7fe29406d5c0 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fe27c001670 con 0x7fe29406d5c0 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 >> v1:172.21.15.55:6789/0 conn(0x7fe29406d5c0 legacy=0x7fe2941635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2800039b0 0x7fe280003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe280004800 0x7fe280004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2940b8190 0x7fe28000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe29409bca0 con 0x7fe2800039b0 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2940d1d20 con 0x7fe280004800 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe280005610 con 0x7fe2940b8190 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe292ffd640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2800039b0 0x7fe280003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe2927fc640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe280004800 0x7fe280004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe292ffd640 1 -- 172.21.15.55:0/2026541454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2940b8190 msgr2=0x7fe28000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe292ffd640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2940b8190 0x7fe28000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe292ffd640 1 -- 172.21.15.55:0/2026541454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe280004800 msgr2=0x7fe280004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe292ffd640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe280004800 0x7fe280004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe292ffd640 1 -- 172.21.15.55:0/2026541454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe280003520 con 0x7fe2800039b0 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.561+0000 7fe2927fc640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe280004800 0x7fe280004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.562+0000 7fe292ffd640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2800039b0 0x7fe280003e20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe284009e10 tx=0x7fe280005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f868b5ed636380a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:21.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.562+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284033070 con 0x7fe2800039b0 2024-05-20T22:09:21.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.562+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe284006e50 con 0x7fe2800039b0 2024-05-20T22:09:21.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.562+0000 7fe291ffb640 1 -- 172.21.15.55:0/2026541454 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284009a10 con 0x7fe2800039b0 2024-05-20T22:09:21.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.563+0000 7fe2995c2640 1 -- 172.21.15.55:0/2026541454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2800039b0 msgr2=0x7fe280003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.563+0000 7fe2995c2640 1 --2- 172.21.15.55:0/2026541454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2800039b0 0x7fe280003e20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe284009e10 tx=0x7fe280005830 comp rx=0 tx=0).stop 2024-05-20T22:09:21.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.563+0000 7fe2937fe640 1 -- 172.21.15.55:0/2026541454 reap_dead start 2024-05-20T22:09:21.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.563+0000 7fe2995c2640 1 -- 172.21.15.55:0/2026541454 shutdown_connections 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.563+0000 7fe2995c2640 1 -- 172.21.15.55:0/2026541454 >> 172.21.15.55:0/2026541454 conn(0x7fe2940cf390 msgr2=0x7fe2940cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.563+0000 7fe2995c2640 1 -- 172.21.15.55:0/2026541454 shutdown_connections 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.563+0000 7fe2995c2640 1 -- 172.21.15.55:0/2026541454 wait complete. 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.565+0000 7fe2995c2640 1 Processor -- start 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.565+0000 7fe2995c2640 1 -- start start 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.566+0000 7fe2995c2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2800039b0 0x7fe2941f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.566+0000 7fe2995c2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe294155c20 0x7fe2940b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.566+0000 7fe2995c2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe294156030 0x7fe2940b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.566+0000 7fe2995c2640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe2940b1bb0 con 0x7fe294155c20 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.566+0000 7fe2995c2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2940d22a0 con 0x7fe294156030 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.566+0000 7fe2995c2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe2940b1d30 con 0x7fe2800039b0 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.566+0000 7fe2927fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe294155c20 0x7fe2940b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.566+0000 7fe2927fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe294155c20 0x7fe2940b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43342/0 (socket says 172.21.15.55:43342) 2024-05-20T22:09:21.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.566+0000 7fe2927fc640 1 -- 172.21.15.55:0/471265180 learned_addr learned my addr 172.21.15.55:0/471265180 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.566+0000 7fe292ffd640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2800039b0 0x7fe2941f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe2937fe640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe294156030 0x7fe2940b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe2927fc640 1 -- 172.21.15.55:0/471265180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2800039b0 msgr2=0x7fe2941f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe2927fc640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2800039b0 0x7fe2941f5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe2927fc640 1 -- 172.21.15.55:0/471265180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe294156030 msgr2=0x7fe2940b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe2927fc640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe294156030 0x7fe2940b6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe2927fc640 1 -- 172.21.15.55:0/471265180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe294162de0 con 0x7fe294155c20 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe2927fc640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe294155c20 0x7fe2940b5f20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00e8f0 tx=0x7fe27c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe28b7fe640 1 -- 172.21.15.55:0/471265180 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00c870 con 0x7fe294155c20 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe28b7fe640 1 -- 172.21.15.55:0/471265180 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe27c018070 con 0x7fe294155c20 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe28b7fe640 1 -- 172.21.15.55:0/471265180 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00cc50 con 0x7fe294155c20 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe2995c2640 1 -- 172.21.15.55:0/471265180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2941621d0 con 0x7fe294155c20 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.567+0000 7fe2995c2640 1 -- 172.21.15.55:0/471265180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe294162790 con 0x7fe294155c20 2024-05-20T22:09:21.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.568+0000 7fe2995c2640 1 -- 172.21.15.55:0/471265180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe294161f40 con 0x7fe294155c20 2024-05-20T22:09:21.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.569+0000 7fe28b7fe640 1 -- 172.21.15.55:0/471265180 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe27c01d460 con 0x7fe294155c20 2024-05-20T22:09:21.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.569+0000 7fe28b7fe640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe27406be30 0x7fe27406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:21.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.569+0000 7fe28b7fe640 1 -- 172.21.15.55:0/471265180 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe27c08af30 con 0x7fe294155c20 2024-05-20T22:09:21.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.570+0000 7fe292ffd640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe27406be30 0x7fe27406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:21.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.570+0000 7fe292ffd640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe27406be30 0x7fe27406e340 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe284035000 tx=0x7fe284030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:21.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.574+0000 7fe28b7fe640 1 -- 172.21.15.55:0/471265180 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe27c059060 con 0x7fe294155c20 2024-05-20T22:09:21.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.798+0000 7fe2995c2640 1 -- 172.21.15.55:0/471265180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe2940b1560 con 0x7fe294155c20 2024-05-20T22:09:21.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.798+0000 7fe28b7fe640 1 -- 172.21.15.55:0/471265180 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7fe2940b1560 con 0x7fe294155c20 2024-05-20T22:09:21.800 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T22:09:21.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 -- 172.21.15.55:0/471265180 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe27406be30 msgr2=0x7fe27406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe27406be30 0x7fe27406e340 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe284035000 tx=0x7fe284030000 comp rx=0 tx=0).stop 2024-05-20T22:09:21.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 -- 172.21.15.55:0/471265180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe294155c20 msgr2=0x7fe2940b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:21.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe294155c20 0x7fe2940b5f20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00e8f0 tx=0x7fe27c00edb0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 -- 172.21.15.55:0/471265180 shutdown_connections 2024-05-20T22:09:21.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe27406be30 0x7fe27406e340 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe294156030 0x7fe2940b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe294155c20 0x7fe2940b5f20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 --2- 172.21.15.55:0/471265180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2800039b0 0x7fe2941f5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:21.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 -- 172.21.15.55:0/471265180 >> 172.21.15.55:0/471265180 conn(0x7fe2940cf390 msgr2=0x7fe2940b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:21.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 -- 172.21.15.55:0/471265180 shutdown_connections 2024-05-20T22:09:21.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:21.800+0000 7fe2995c2640 1 -- 172.21.15.55:0/471265180 wait complete. 2024-05-20T22:09:21.812 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T22:09:21.812 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T22:09:21.813 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-20T22:09:21.813 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T22:09:21.813 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T22:09:21.813 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T22:09:21.813 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T22:09:21.813 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T22:09:21.813 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T22:09:21.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:21.813 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-20T22:09:21.830 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:09:21.831 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:09:21.831 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:09:21.831 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:09:21.831 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-20T22:09:21.831 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:09:21.831 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:09:21.831 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:09:21.831 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-20T22:09:21.831 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:09:21.833 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:09:21.833 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-20T22:09:21.833 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T22:09:21.833 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:09:21.833 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-20T22:09:21.834 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T22:09:21.834 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42630sec preferred_lft 42630sec 2024-05-20T22:09:21.834 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-20T22:09:21.834 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:09:21.834 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:09:21.834 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:09:21.835 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-20T22:09:21.835 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T22:09:21.835 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 52:e2:c2:8c:12:3b brd ff:ff:ff:ff:ff:ff 2024-05-20T22:09:21.835 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ba:c7ff:fe99:ba87/64 scope link 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 52:e2:c2:8c:12:3b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::50e2:c2ff:fe8c:123b/64 scope link 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether d2:62:32:ab:d0:b9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::d062:32ff:feab:d0b9/64 scope link 2024-05-20T22:09:21.836 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:09:21.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:21.837 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T22:09:21.896 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-20T22:09:21.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:21.897 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-20T22:09:21.955 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-20T22:09:21.955 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T22:09:21.955 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T22:09:21.955 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T22:09:21.955 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T22:09:21.956 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:09:21.956 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:09:21.956 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T22:09:21.956 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T22:09:21.956 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T22:09:21.956 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:09:22.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:09:22.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:09:22.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:22.085 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-20T22:09:22.101 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T22:09:22.101 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T22:09:22.102 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T22:09:22.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:22.102 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-20T22:09:22.168 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-20T22:09:22.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:09:22.169 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T22:09:22.225 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T22:09:22.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:09:22.225 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T22:09:22.281 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:09:22.348 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:09:22.405 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:09:22.406 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:09:22.407 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:09:22.407 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:09:22.407 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:09:22.407 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:09:22.407 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:09:22.407 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:09:22.407 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:09:22.407 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:09:22.407 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:09:22.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:22.407 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:09:22.460 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-20T22:09:22.460 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-20T22:09:22.503 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:09:22.529 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:09:22.588 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:09:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:09:22.590 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:09:22.590 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:09:22.590 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:09:22.590 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:09:22.590 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:09:22.590 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:09:22.590 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:09:22.590 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:09:22.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:22.591 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:09:22.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:09:22.598+0000 7f6db1f1b480 -1 init, newargv = 0x561538fff550 newargc=15 2024-05-20T22:09:22.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:ceph-fuse[54219]: starting ceph client 2024-05-20T22:09:22.606 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:ceph-fuse[54219]: starting fuse 2024-05-20T22:09:22.615 INFO:teuthology.orchestra.run.smithi055.stdout:40 2024-05-20T22:09:22.616 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-20T22:09:22.616 DEBUG:teuthology.orchestra.run.smithi055:> sudo stdin-killer -- python3 -c ' 2024-05-20T22:09:22.616 DEBUG:teuthology.orchestra.run.smithi055:> import glob 2024-05-20T22:09:22.616 DEBUG:teuthology.orchestra.run.smithi055:> import re 2024-05-20T22:09:22.616 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:09:22.616 DEBUG:teuthology.orchestra.run.smithi055:> import subprocess 2024-05-20T22:09:22.616 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:22.616 DEBUG:teuthology.orchestra.run.smithi055:> def _find_admin_socket(client_name): 2024-05-20T22:09:22.616 DEBUG:teuthology.orchestra.run.smithi055:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-20T22:09:22.616 DEBUG:teuthology.orchestra.run.smithi055:> files = glob.glob(asok_path) 2024-05-20T22:09:22.616 DEBUG:teuthology.orchestra.run.smithi055:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> # Given a non-glob path, it better be there 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> if "*" not in asok_path: 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> assert(len(files) == 1) 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> return files[0] 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> for f in files: 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> contents = proc_f.read() 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> if mountpoint in contents: 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> return f 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-20T22:09:22.617 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:22.618 DEBUG:teuthology.orchestra.run.smithi055:> print(_find_admin_socket("client.0")) 2024-05-20T22:09:22.618 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:09:22.738 INFO:teuthology.orchestra.run.smithi055.stdout:/var/run/ceph/ceph-client.0.54219.asok 2024-05-20T22:09:22.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:09:22.749 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54219.asok 2024-05-20T22:09:22.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:22.749 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54219.asok status 2024-05-20T22:09:22.897 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:09:22.897 INFO:teuthology.orchestra.run.smithi055.stdout: "metadata": { 2024-05-20T22:09:22.897 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_sha1": "d2b507c724ef6c8295ee0a35d185cdab167fd61d", 2024-05-20T22:09:22.897 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "entity_id": "0", 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": "smithi055", 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "pid": "54219", 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "root": "/" 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_count": 0, 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_pinned_count": 0, 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 5736, 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "inst": { 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "name": { 2024-05-20T22:09:22.898 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "client", 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: "num": 5736 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.1:0", 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 3080107143 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-20T22:09:22.899 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.1:0", 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 3080107143 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout: "inst_str": "client.5736 192.168.0.1:0/3080107143", 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout: "addr_str": "192.168.0.1:0/3080107143", 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout: "inode_count": 1, 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_epoch": 18, 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch": 38, 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch_barrier": 0, 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout: "blocklisted": false, 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs" 2024-05-20T22:09:22.900 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:09:22.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:22.906 DEBUG:teuthology.orchestra.run.smithi055:> stat --file-system '--printf=%T 2024-05-20T22:09:22.907 DEBUG:teuthology.orchestra.run.smithi055:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-20T22:09:22.923 INFO:teuthology.orchestra.run.smithi055.stdout:fuseblk 2024-05-20T22:09:22.923 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-20T22:09:22.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:22.924 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T22:09:22.992 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T22:09:23.033 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T22:09:23.075 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2024-05-20T22:09:23.075 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.076 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2024-05-20T22:09:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: mon_allow_pool_size_one = true 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2024-05-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd mclock profile = high_recovery_ops 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2024-05-20T22:09:23.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug trim objects = true 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.080 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2024-05-20T22:09:23.081 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-20T22:09:23.082 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T22:09:23.083 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: # 1m isn't quite enough 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: mon_down_mkfs_grace = 2m 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_filestore_osds = false 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: mon down mkfs grace = 300 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T22:09:23.084 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = "" 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2024-05-20T22:09:23.085 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2024-05-20T22:09:23.086 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds balancer = 20 2024-05-20T22:09:23.086 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:09:23.086 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2024-05-20T22:09:23.086 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2024-05-20T22:09:23.086 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2024-05-20T22:09:23.086 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2024-05-20T22:09:23.086 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-20T22:09:23.086 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-20T22:09:23.086 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-20T22:09:23.086 INFO:teuthology.orchestra.run.smithi055.stdout: osd_deep_scrub_large_omap_object_key_threshold = 5 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-05-20T22:09:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-05-20T22:09:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-05-20T22:09:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-05-20T22:09:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-05-20T22:09:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-20T22:09:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T22:09:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: mon down mkfs grace = 300 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:09:23.095 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-05-20T22:09:23.096 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-20T22:09:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-20T22:09:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-20T22:09:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: osd_deep_scrub_large_omap_object_key_threshold = 5 2024-05-20T22:09:23.099 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T22:09:23.099 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T22:09:23.100 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T22:09:23.100 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:09:23.100 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T22:09:23.100 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T22:09:23.100 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T22:09:23.100 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:09:23.100 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T22:09:23.100 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T22:09:23.101 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T22:09:23.101 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:09:23.101 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T22:09:23.101 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T22:09:23.101 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T22:09:23.101 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:09:23.114 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T22:09:23.114 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T22:09:23.151 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T22:09:23.151 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T22:09:23.152 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T22:09:23.152 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T22:09:23.157 INFO:tasks.ceph.mds.d:Started 2024-05-20T22:09:23.157 DEBUG:teuthology.parallel:result is None 2024-05-20T22:09:23.194 INFO:tasks.ceph.mds.a:Started 2024-05-20T22:09:23.194 DEBUG:teuthology.parallel:result is None 2024-05-20T22:09:23.195 INFO:tasks.ceph.mds.c:Started 2024-05-20T22:09:23.195 DEBUG:teuthology.parallel:result is None 2024-05-20T22:09:23.247 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-05-20T22:09:23.284 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-20T22:09:23.285 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-20T22:09:23.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:09:23.307+0000 7f6daa7a6640 -1 --2- 192.168.0.1:0/3080107143 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x5615390bbdf0 0x5615390be210 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] is using msgr V1 protocol 2024-05-20T22:09:23.709 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:09:23.708+0000 7f6daa7a6640 -1 --2- 192.168.0.1:0/3080107143 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x5615390bbdf0 0x5615390be210 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] is using msgr V1 protocol 2024-05-20T22:09:24.510 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:09:24.509+0000 7f6daa7a6640 -1 --2- 192.168.0.1:0/3080107143 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x5615390bbdf0 0x5615390be210 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] is using msgr V1 protocol 2024-05-20T22:09:26.113 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:09:26.112+0000 7f6daa7a6640 -1 --2- 192.168.0.1:0/3080107143 >> [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] conn(0x5615390bbdf0 0x5615390be210 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.138:6833/4137968836,v1:172.21.15.138:6835/4137968836] is using msgr V1 protocol 2024-05-20T22:09:28.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-05-20T22:09:29.114 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T22:09:29.114 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T22:09:29.117 INFO:tasks.ceph.mds.b:Started 2024-05-20T22:09:29.117 DEBUG:teuthology.parallel:result is None 2024-05-20T22:09:29.118 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:09:29.214 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-05-20T22:09:29.218 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-20T22:09:29.217+0000 7f0b3ef60640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T22:09:29.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.251+0000 7efe3df2c640 1 Processor -- start 2024-05-20T22:09:29.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.251+0000 7efe3df2c640 1 -- start start 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.252+0000 7efe3df2c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efe380b5b80 0x7efe38160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.252+0000 7efe3df2c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe380cd290 0x7efe3815cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.252+0000 7efe3df2c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe3806f0f0 0x7efe3806f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.252+0000 7efe3df2c640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7efe380d1090 con 0x7efe380b5b80 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.252+0000 7efe3df2c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efe38161100 con 0x7efe3806f0f0 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.252+0000 7efe3df2c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe3806fa40 con 0x7efe380cd290 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.252+0000 7efe377fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efe380b5b80 0x7efe38160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.252+0000 7efe377fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efe380b5b80 0x7efe38160bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52136/0 (socket says 172.21.15.55:52136) 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.252+0000 7efe377fe640 1 -- 172.21.15.55:0/1589099271 learned_addr learned my addr 172.21.15.55:0/1589099271 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.253+0000 7efe36ffd640 1 --2- 172.21.15.55:0/1589099271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe380cd290 0x7efe3815cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.253+0000 7efe37fff640 1 --2- 172.21.15.55:0/1589099271 >> v2:172.21.15.138:3300/0 conn(0x7efe3806f0f0 0x7efe3806f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:29.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.253+0000 7efe377fe640 1 -- 172.21.15.55:0/1589099271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe380cd290 msgr2=0x7efe3815cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:29.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.253+0000 7efe377fe640 1 --2- 172.21.15.55:0/1589099271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe380cd290 0x7efe3815cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.253+0000 7efe377fe640 1 -- 172.21.15.55:0/1589099271 >> v2:172.21.15.138:3300/0 conn(0x7efe3806f0f0 msgr2=0x7efe3806f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:29.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.253+0000 7efe377fe640 1 --2- 172.21.15.55:0/1589099271 >> v2:172.21.15.138:3300/0 conn(0x7efe3806f0f0 0x7efe3806f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.253+0000 7efe377fe640 1 -- 172.21.15.55:0/1589099271 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3815d4a0 con 0x7efe380b5b80 2024-05-20T22:09:29.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.253+0000 7efe377fe640 1 --2- 172.21.15.55:0/1589099271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe380b5b80 0x7efe38160bc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efe380b5f90 tx=0x7efe2802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dfdfd0183f7fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:29.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.254+0000 7efe367fc640 1 -- 172.21.15.55:0/1589099271 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28037070 con 0x7efe380b5b80 2024-05-20T22:09:29.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.254+0000 7efe367fc640 1 -- 172.21.15.55:0/1589099271 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe28032440 con 0x7efe380b5b80 2024-05-20T22:09:29.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.254+0000 7efe367fc640 1 -- 172.21.15.55:0/1589099271 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28032810 con 0x7efe380b5b80 2024-05-20T22:09:29.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.254+0000 7efe3df2c640 1 -- 172.21.15.55:0/1589099271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe380b5b80 msgr2=0x7efe38160bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:29.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.254+0000 7efe3df2c640 1 --2- 172.21.15.55:0/1589099271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe380b5b80 0x7efe38160bc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efe380b5f90 tx=0x7efe2802a140 comp rx=0 tx=0).stop 2024-05-20T22:09:29.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.254+0000 7efe3df2c640 1 -- 172.21.15.55:0/1589099271 shutdown_connections 2024-05-20T22:09:29.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.254+0000 7efe3df2c640 1 --2- 172.21.15.55:0/1589099271 >> v2:172.21.15.138:3300/0 conn(0x7efe3806f0f0 0x7efe3806f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.254+0000 7efe3df2c640 1 --2- 172.21.15.55:0/1589099271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe380cd290 0x7efe3815cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.254+0000 7efe3df2c640 1 --2- 172.21.15.55:0/1589099271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe380b5b80 0x7efe38160bc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.254+0000 7efe3df2c640 1 -- 172.21.15.55:0/1589099271 >> 172.21.15.55:0/1589099271 conn(0x7efe380d2910 msgr2=0x7efe380d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:29.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.255+0000 7efe3df2c640 1 -- 172.21.15.55:0/1589099271 shutdown_connections 2024-05-20T22:09:29.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.255+0000 7efe3df2c640 1 -- 172.21.15.55:0/1589099271 wait complete. 2024-05-20T22:09:29.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.256+0000 7efe3df2c640 1 Processor -- start 2024-05-20T22:09:29.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.257+0000 7efe3df2c640 1 -- start start 2024-05-20T22:09:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe3df2c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe3806f0f0 0x7efe380ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe3df2c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe380b5b80 0x7efe38159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe3df2c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe380cd290 0x7efe381e8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe3df2c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efe38161940 con 0x7efe3806f0f0 2024-05-20T22:09:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe3df2c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe38161660 con 0x7efe380b5b80 2024-05-20T22:09:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe3df2c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe380d12e0 con 0x7efe380cd290 2024-05-20T22:09:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe377fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe3806f0f0 0x7efe380ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe377fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe3806f0f0 0x7efe380ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52138/0 (socket says 172.21.15.55:52138) 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe377fe640 1 -- 172.21.15.55:0/1836080006 learned_addr learned my addr 172.21.15.55:0/1836080006 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe36ffd640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe380b5b80 0x7efe38159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe377fe640 1 -- 172.21.15.55:0/1836080006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe380cd290 msgr2=0x7efe381e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe377fe640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe380cd290 0x7efe381e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe377fe640 1 -- 172.21.15.55:0/1836080006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe380b5b80 msgr2=0x7efe38159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe377fe640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe380b5b80 0x7efe38159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe377fe640 1 -- 172.21.15.55:0/1836080006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe380b0fb0 con 0x7efe3806f0f0 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.258+0000 7efe36ffd640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe380b5b80 0x7efe38159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.259+0000 7efe377fe640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe3806f0f0 0x7efe380ca1d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efe2802a180 tx=0x7efe280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.259+0000 7efe1ffff640 1 -- 172.21.15.55:0/1836080006 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28041020 con 0x7efe3806f0f0 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.259+0000 7efe1ffff640 1 -- 172.21.15.55:0/1836080006 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe280306c0 con 0x7efe3806f0f0 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.259+0000 7efe3df2c640 1 -- 172.21.15.55:0/1836080006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe380b1290 con 0x7efe3806f0f0 2024-05-20T22:09:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.259+0000 7efe1ffff640 1 -- 172.21.15.55:0/1836080006 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28037040 con 0x7efe3806f0f0 2024-05-20T22:09:29.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.259+0000 7efe3df2c640 1 -- 172.21.15.55:0/1836080006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3815a3a0 con 0x7efe3806f0f0 2024-05-20T22:09:29.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.260+0000 7efe3df2c640 1 -- 172.21.15.55:0/1836080006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3804f720 con 0x7efe3806f0f0 2024-05-20T22:09:29.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.260+0000 7efe1ffff640 1 -- 172.21.15.55:0/1836080006 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe28044020 con 0x7efe3806f0f0 2024-05-20T22:09:29.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.261+0000 7efe1ffff640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe2406be30 0x7efe2406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:29.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.261+0000 7efe36ffd640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe2406be30 0x7efe2406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:29.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.261+0000 7efe1ffff640 1 -- 172.21.15.55:0/1836080006 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6187+0+0 (secure 0 0 0) 0x7efe280ab9d0 con 0x7efe3806f0f0 2024-05-20T22:09:29.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.262+0000 7efe36ffd640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe2406be30 0x7efe2406e340 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efe380b7080 tx=0x7efe2c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:29.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.266+0000 7efe1ffff640 1 -- 172.21.15.55:0/1836080006 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efe2804b030 con 0x7efe3806f0f0 2024-05-20T22:09:29.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.487+0000 7efe3df2c640 1 -- 172.21.15.55:0/1836080006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe380b2050 con 0x7efe3806f0f0 2024-05-20T22:09:29.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.487+0000 7efe1ffff640 1 -- 172.21.15.55:0/1836080006 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5182 (secure 0 0 0) 0x7efe2807a070 con 0x7efe3806f0f0 2024-05-20T22:09:29.488 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:29.488 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3873372908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3873372908},{"type":"v1","addr":"172.21.15.55:6836","nonce":3873372908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3417305427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":3417305427},{"type":"v1","addr":"172.21.15.55:6837","nonce":3417305427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":35081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4211704780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4211704780},{"type":"v1","addr":"172.21.15.138:6833","nonce":4211704780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:09:14.957632+0000","modified":"2024-05-20T22:09:29.223402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":35093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35093":{"gid":35093,"name":"b","rank":0,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.138:6835/270300569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":270300569},{"type":"v1","addr":"172.21.15.138:6835","nonce":270300569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-05-20T22:09:29.489 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 22 2024-05-20T22:09:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 -- 172.21.15.55:0/1836080006 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe2406be30 msgr2=0x7efe2406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe2406be30 0x7efe2406e340 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efe380b7080 tx=0x7efe2c008040 comp rx=0 tx=0).stop 2024-05-20T22:09:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 -- 172.21.15.55:0/1836080006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe3806f0f0 msgr2=0x7efe380ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:29.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe3806f0f0 0x7efe380ca1d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efe2802a180 tx=0x7efe280046c0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 -- 172.21.15.55:0/1836080006 shutdown_connections 2024-05-20T22:09:29.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe2406be30 0x7efe2406e340 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe380cd290 0x7efe381e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe380b5b80 0x7efe38159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 --2- 172.21.15.55:0/1836080006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe3806f0f0 0x7efe380ca1d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:29.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 -- 172.21.15.55:0/1836080006 >> 172.21.15.55:0/1836080006 conn(0x7efe380d2910 msgr2=0x7efe380b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:29.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 -- 172.21.15.55:0/1836080006 shutdown_connections 2024-05-20T22:09:29.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:29.489+0000 7efe3df2c640 1 -- 172.21.15.55:0/1836080006 wait complete. 2024-05-20T22:09:29.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:09:14.957632+0000', 'modified': '2024-05-20T22:09:29.223402+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 42, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35093}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35093': {'gid': 35093, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.138:6835/270300569', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 270300569}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 270300569}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T22:09:29.904 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_35093:up:replay 2024-05-20T22:09:30.905 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:09:31.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.034+0000 7f7a6bc1b640 1 Processor -- start 2024-05-20T22:09:31.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.034+0000 7f7a6bc1b640 1 -- start start 2024-05-20T22:09:31.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.035+0000 7f7a6bc1b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a64071bc0 0x7f7a64071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:31.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.035+0000 7f7a6bc1b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a640b70e0 0x7f7a640b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:31.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.035+0000 7f7a6bc1b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a64072510 con 0x7f7a6415ce10 2024-05-20T22:09:31.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.035+0000 7f7a6bc1b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7a640d1540 con 0x7f7a640b70e0 2024-05-20T22:09:31.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.035+0000 7f7a6bc1b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7a640d2190 con 0x7f7a64071bc0 2024-05-20T22:09:31.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.036+0000 7f7a6a1b8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a640b70e0 0x7f7a640b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:31.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.036+0000 7f7a6a1b8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a640b70e0 0x7f7a640b74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:41542/0 (socket says 172.21.15.55:41542) 2024-05-20T22:09:31.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.036+0000 7f7a699b7640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7a6415ce10 0x7f7a64160090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:49466/0 (socket says 172.21.15.55:49466) 2024-05-20T22:09:31.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.036+0000 7f7a6a1b8640 1 -- 172.21.15.55:0/3539492426 learned_addr learned my addr 172.21.15.55:0/3539492426 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:31.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.036+0000 7f7a6a1b8640 1 -- 172.21.15.55:0/3539492426 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a64071bc0 msgr2=0x7f7a64071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:31.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.036+0000 7f7a6a1b8640 1 --2- 172.21.15.55:0/3539492426 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a64071bc0 0x7f7a64071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:31.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.036+0000 7f7a6a1b8640 1 -- 172.21.15.55:0/3539492426 >> v1:172.21.15.138:6789/0 conn(0x7f7a6415ce10 legacy=0x7f7a64160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:31.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.036+0000 7f7a6a1b8640 1 -- 172.21.15.55:0/3539492426 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a640d0930 con 0x7f7a640b70e0 2024-05-20T22:09:31.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.036+0000 7f7a689b5640 1 -- 172.21.15.55:0/3539492426 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4201789251 0 0) 0x7f7a5c001110 con 0x7f7a6415ce10 2024-05-20T22:09:31.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.037+0000 7f7a6a1b8640 1 --2- 172.21.15.55:0/3539492426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a640b70e0 0x7f7a640b74f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009e50 tx=0x7f7a5402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef5e7fbe8ccbce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:31.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.037+0000 7f7a689b5640 1 -- 172.21.15.55:0/3539492426 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54036070 con 0x7f7a640b70e0 2024-05-20T22:09:31.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.037+0000 7f7a689b5640 1 -- 172.21.15.55:0/3539492426 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a54031600 con 0x7f7a640b70e0 2024-05-20T22:09:31.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.037+0000 7f7a689b5640 1 -- 172.21.15.55:0/3539492426 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540319d0 con 0x7f7a640b70e0 2024-05-20T22:09:31.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.038+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/3539492426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a640b70e0 msgr2=0x7f7a640b74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:31.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.038+0000 7f7a6bc1b640 1 --2- 172.21.15.55:0/3539492426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a640b70e0 0x7f7a640b74f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009e50 tx=0x7f7a5402a710 comp rx=0 tx=0).stop 2024-05-20T22:09:31.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.038+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/3539492426 shutdown_connections 2024-05-20T22:09:31.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.038+0000 7f7a6bc1b640 1 --2- 172.21.15.55:0/3539492426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a640b70e0 0x7f7a640b74f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:31.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.038+0000 7f7a6bc1b640 1 --2- 172.21.15.55:0/3539492426 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a64071bc0 0x7f7a64071fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:31.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.038+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/3539492426 >> 172.21.15.55:0/3539492426 conn(0x7f7a640d2910 msgr2=0x7f7a640d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:31.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.038+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/3539492426 shutdown_connections 2024-05-20T22:09:31.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.038+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/3539492426 wait complete. 2024-05-20T22:09:31.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.040+0000 7f7a6bc1b640 1 Processor -- start 2024-05-20T22:09:31.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.040+0000 7f7a6bc1b640 1 -- start start 2024-05-20T22:09:31.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.041+0000 7f7a6bc1b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a64071bc0 0x7f7a6414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:31.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.041+0000 7f7a6bc1b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a640b70e0 0x7f7a640ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:31.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.041+0000 7f7a6bc1b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a6414ef80 0x7f7a640b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:31.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.041+0000 7f7a6bc1b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7a640d11d0 con 0x7f7a6414ef80 2024-05-20T22:09:31.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.041+0000 7f7a6bc1b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a640d0ef0 con 0x7f7a640b70e0 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.041+0000 7f7a6bc1b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7a641607d0 con 0x7f7a64071bc0 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.041+0000 7f7a6a1b8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a6414ef80 0x7f7a640b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.041+0000 7f7a6a1b8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a6414ef80 0x7f7a640b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52146/0 (socket says 172.21.15.55:52146) 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.041+0000 7f7a6a1b8640 1 -- 172.21.15.55:0/1477026018 learned_addr learned my addr 172.21.15.55:0/1477026018 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a699b7640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a64071bc0 0x7f7a6414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a691b6640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a640b70e0 0x7f7a640ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a6a1b8640 1 -- 172.21.15.55:0/1477026018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a64071bc0 msgr2=0x7f7a6414fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a6a1b8640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a64071bc0 0x7f7a6414fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a6a1b8640 1 -- 172.21.15.55:0/1477026018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a640b70e0 msgr2=0x7f7a640ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a6a1b8640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a640b70e0 0x7f7a640ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a6a1b8640 1 -- 172.21.15.55:0/1477026018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a640c9fa0 con 0x7f7a6414ef80 2024-05-20T22:09:31.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a6a1b8640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a6414ef80 0x7f7a640b0db0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7a54034040 tx=0x7f7a54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:31.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a627fc640 1 -- 172.21.15.55:0/1477026018 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54041070 con 0x7f7a6414ef80 2024-05-20T22:09:31.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a627fc640 1 -- 172.21.15.55:0/1477026018 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a54003990 con 0x7f7a6414ef80 2024-05-20T22:09:31.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/1477026018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a640afff0 con 0x7f7a6414ef80 2024-05-20T22:09:31.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a627fc640 1 -- 172.21.15.55:0/1477026018 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54036040 con 0x7f7a6414ef80 2024-05-20T22:09:31.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.042+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/1477026018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a640b05b0 con 0x7f7a6414ef80 2024-05-20T22:09:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.043+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/1477026018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6404f720 con 0x7f7a6414ef80 2024-05-20T22:09:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.044+0000 7f7a627fc640 1 -- 172.21.15.55:0/1477026018 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a54007820 con 0x7f7a6414ef80 2024-05-20T22:09:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.045+0000 7f7a627fc640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7a5006be60 0x7f7a5006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.045+0000 7f7a699b7640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7a5006be60 0x7f7a5006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.045+0000 7f7a627fc640 1 -- 172.21.15.55:0/1477026018 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6187+0+0 (secure 0 0 0) 0x7f7a540abd70 con 0x7f7a6414ef80 2024-05-20T22:09:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.045+0000 7f7a699b7640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7a5006be60 0x7f7a5006e370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c000f60 tx=0x7f7a5c007c00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.050+0000 7f7a627fc640 1 -- 172.21.15.55:0/1477026018 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7a5404b020 con 0x7f7a6414ef80 2024-05-20T22:09:31.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.270+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/1477026018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a640b54f0 con 0x7f7a6414ef80 2024-05-20T22:09:31.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.270+0000 7f7a627fc640 1 -- 172.21.15.55:0/1477026018 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5182 (secure 0 0 0) 0x7f7a5407a410 con 0x7f7a6414ef80 2024-05-20T22:09:31.272 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:31.272 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3873372908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3873372908},{"type":"v1","addr":"172.21.15.55:6836","nonce":3873372908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3417305427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":3417305427},{"type":"v1","addr":"172.21.15.55:6837","nonce":3417305427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":35081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4211704780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4211704780},{"type":"v1","addr":"172.21.15.138:6833","nonce":4211704780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:09:14.957632+0000","modified":"2024-05-20T22:09:30.231605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":35093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35093":{"gid":35093,"name":"b","rank":0,"incarnation":22,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.138:6835/270300569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":270300569},{"type":"v1","addr":"172.21.15.138:6835","nonce":270300569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-05-20T22:09:31.272 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 24 2024-05-20T22:09:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/1477026018 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7a5006be60 msgr2=0x7f7a5006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7a5006be60 0x7f7a5006e370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c000f60 tx=0x7f7a5c007c00 comp rx=0 tx=0).stop 2024-05-20T22:09:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/1477026018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a6414ef80 msgr2=0x7f7a640b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a6414ef80 0x7f7a640b0db0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7a54034040 tx=0x7f7a54004a40 comp rx=0 tx=0).stop 2024-05-20T22:09:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/1477026018 shutdown_connections 2024-05-20T22:09:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7a5006be60 0x7f7a5006e370 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a6414ef80 0x7f7a640b0db0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a640b70e0 0x7f7a640ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 --2- 172.21.15.55:0/1477026018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a64071bc0 0x7f7a6414fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/1477026018 >> 172.21.15.55:0/1477026018 conn(0x7f7a640d2910 msgr2=0x7f7a640b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/1477026018 shutdown_connections 2024-05-20T22:09:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:31.273+0000 7f7a6bc1b640 1 -- 172.21.15.55:0/1477026018 wait complete. 2024-05-20T22:09:31.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:09:14.957632+0000', 'modified': '2024-05-20T22:09:30.231605+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 42, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35093}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35093': {'gid': 35093, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.138:6835/270300569', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 270300569}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 270300569}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T22:09:31.688 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_35093:up:rejoin 2024-05-20T22:09:32.689 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:09:32.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.818+0000 7fccb999d640 1 Processor -- start 2024-05-20T22:09:32.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.818+0000 7fccb999d640 1 -- start start 2024-05-20T22:09:32.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.819+0000 7fccb999d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fccb40d1710 0x7fccb40d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:32.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.819+0000 7fccb999d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccb40ce120 0x7fccb40ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:32.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.819+0000 7fccb999d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb40cea70 con 0x7fccb415bfe0 2024-05-20T22:09:32.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.819+0000 7fccb999d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fccb40b36a0 con 0x7fccb40d1710 2024-05-20T22:09:32.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.819+0000 7fccb999d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fccb40d2160 con 0x7fccb40ce120 2024-05-20T22:09:32.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccb899b640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fccb415bfe0 0x7fccb40b3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35770/0 (socket says 172.21.15.55:35770) 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccb899b640 1 -- 172.21.15.55:0/2201757307 learned_addr learned my addr 172.21.15.55:0/2201757307 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccabfff640 1 --2- 172.21.15.55:0/2201757307 >> v2:172.21.15.138:3300/0 conn(0x7fccb40d1710 0x7fccb40d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccb919c640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccb40ce120 0x7fccb40ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1371451968 0 0) 0x7fccb40cea70 con 0x7fccb415bfe0 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc9c003520 con 0x7fccb415bfe0 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3576876164 0 0) 0x7fcc9c003520 con 0x7fccb415bfe0 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccb40ce120 msgr2=0x7fccb40ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccb40ce120 0x7fccb40ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 >> v2:172.21.15.138:3300/0 conn(0x7fccb40d1710 msgr2=0x7fccb40d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 --2- 172.21.15.55:0/2201757307 >> v2:172.21.15.138:3300/0 conn(0x7fccb40d1710 0x7fccb40d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb40b3050 con 0x7fccb415bfe0 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fccb0001810 con 0x7fccb415bfe0 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 >> v1:172.21.15.55:6789/0 conn(0x7fccb415bfe0 legacy=0x7fccb40b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:32.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc9c0039b0 0x7fcc9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc9c004800 0x7fcc9c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccb40ce120 0x7fcc9c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fccb40b36a0 con 0x7fcc9c0039b0 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fccb40d2160 con 0x7fcc9c004800 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcc9c00dc00 con 0x7fccb40ce120 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.820+0000 7fccb899b640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc9c0039b0 0x7fcc9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.821+0000 7fccabfff640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc9c004800 0x7fcc9c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.821+0000 7fccb919c640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccb40ce120 0x7fcc9c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.821+0000 7fccb899b640 1 -- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccb40ce120 msgr2=0x7fcc9c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.821+0000 7fccb899b640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccb40ce120 0x7fcc9c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.821+0000 7fccb899b640 1 -- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc9c004800 msgr2=0x7fcc9c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.821+0000 7fccb899b640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc9c004800 0x7fcc9c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:32.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.821+0000 7fccb899b640 1 -- 172.21.15.55:0/2201757307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c003520 con 0x7fcc9c0039b0 2024-05-20T22:09:32.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.821+0000 7fccabfff640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc9c004800 0x7fcc9c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:32.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.822+0000 7fccb899b640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc9c0039b0 0x7fcc9c003e20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fccb40d2090 tx=0x7fcc9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e707858fa4d580 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:32.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.822+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0007d70 con 0x7fcc9c0039b0 2024-05-20T22:09:32.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.822+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb0007270 con 0x7fcc9c0039b0 2024-05-20T22:09:32.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.822+0000 7fccab7fe640 1 -- 172.21.15.55:0/2201757307 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00057a0 con 0x7fcc9c0039b0 2024-05-20T22:09:32.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.822+0000 7fccb999d640 1 -- 172.21.15.55:0/2201757307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc9c0039b0 msgr2=0x7fcc9c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:32.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.822+0000 7fccb999d640 1 --2- 172.21.15.55:0/2201757307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc9c0039b0 0x7fcc9c003e20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fccb40d2090 tx=0x7fcc9c005830 comp rx=0 tx=0).stop 2024-05-20T22:09:32.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.822+0000 7fccb919c640 1 -- 172.21.15.55:0/2201757307 reap_dead start 2024-05-20T22:09:32.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.823+0000 7fccb999d640 1 -- 172.21.15.55:0/2201757307 shutdown_connections 2024-05-20T22:09:32.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.823+0000 7fccb999d640 1 -- 172.21.15.55:0/2201757307 >> 172.21.15.55:0/2201757307 conn(0x7fccb409a400 msgr2=0x7fccb409a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:32.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.823+0000 7fccb999d640 1 -- 172.21.15.55:0/2201757307 shutdown_connections 2024-05-20T22:09:32.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.823+0000 7fccb999d640 1 -- 172.21.15.55:0/2201757307 wait complete. 2024-05-20T22:09:32.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.824+0000 7fccb999d640 1 Processor -- start 2024-05-20T22:09:32.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.825+0000 7fccb999d640 1 -- start start 2024-05-20T22:09:32.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccb999d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc9c0039b0 0x7fccb415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccb999d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb41589f0 0x7fccb415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccb999d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccb415a9d0 0x7fccb409d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccb999d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fccb406e570 con 0x7fccb415a9d0 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccb999d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fccb40b2d00 con 0x7fccb41589f0 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccb999d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fccb406e6f0 con 0x7fcc9c0039b0 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccb919c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccb415a9d0 0x7fccb409d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccb919c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccb415a9d0 0x7fccb409d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52154/0 (socket says 172.21.15.55:52154) 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccb919c640 1 -- 172.21.15.55:0/180791763 learned_addr learned my addr 172.21.15.55:0/180791763 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccb899b640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc9c0039b0 0x7fccb415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.826+0000 7fccabfff640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb41589f0 0x7fccb415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fccb919c640 1 -- 172.21.15.55:0/180791763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc9c0039b0 msgr2=0x7fccb415e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fccb919c640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc9c0039b0 0x7fccb415e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:32.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fccb919c640 1 -- 172.21.15.55:0/180791763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb41589f0 msgr2=0x7fccb415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fccb919c640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb41589f0 0x7fccb415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fccb919c640 1 -- 172.21.15.55:0/180791763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb415ade0 con 0x7fccb415a9d0 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fccb899b640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc9c0039b0 0x7fccb415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fccabfff640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb41589f0 0x7fccb415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fccb919c640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccb415a9d0 0x7fccb409d630 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fccb40cbd80 tx=0x7fccac0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fcca97fa640 1 -- 172.21.15.55:0/180791763 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac011e40 con 0x7fccb415a9d0 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fcca97fa640 1 -- 172.21.15.55:0/180791763 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccac01b070 con 0x7fccb415a9d0 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fccb999d640 1 -- 172.21.15.55:0/180791763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb409e6e0 con 0x7fccb415a9d0 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fcca97fa640 1 -- 172.21.15.55:0/180791763 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac003990 con 0x7fccb415a9d0 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.827+0000 7fccb999d640 1 -- 172.21.15.55:0/180791763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb415ee10 con 0x7fccb415a9d0 2024-05-20T22:09:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.828+0000 7fccb999d640 1 -- 172.21.15.55:0/180791763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb4075990 con 0x7fccb415a9d0 2024-05-20T22:09:32.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.829+0000 7fcca97fa640 1 -- 172.21.15.55:0/180791763 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fccac005d10 con 0x7fccb415a9d0 2024-05-20T22:09:32.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.829+0000 7fcca97fa640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcca006be60 0x7fcca006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:32.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.829+0000 7fcca97fa640 1 -- 172.21.15.55:0/180791763 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6187+0+0 (secure 0 0 0) 0x7fccac08b3c0 con 0x7fccb415a9d0 2024-05-20T22:09:32.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.829+0000 7fccb899b640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcca006be60 0x7fcca006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:32.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.830+0000 7fccb899b640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcca006be60 0x7fcca006e370 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fccb0033000 tx=0x7fccb002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:32.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:32.834+0000 7fcca97fa640 1 -- 172.21.15.55:0/180791763 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fccb415ade0 con 0x7fccb415a9d0 2024-05-20T22:09:33.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.055+0000 7fccb999d640 1 -- 172.21.15.55:0/180791763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccb415bb20 con 0x7fccb415a9d0 2024-05-20T22:09:33.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.055+0000 7fcca97fa640 1 -- 172.21.15.55:0/180791763 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5188 (secure 0 0 0) 0x7fccac014350 con 0x7fccb415a9d0 2024-05-20T22:09:33.056 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:33.056 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3873372908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3873372908},{"type":"v1","addr":"172.21.15.55:6836","nonce":3873372908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3417305427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":3417305427},{"type":"v1","addr":"172.21.15.55:6837","nonce":3417305427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":35081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4211704780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4211704780},{"type":"v1","addr":"172.21.15.138:6833","nonce":4211704780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:09:14.957632+0000","modified":"2024-05-20T22:09:31.239510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":35093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35093":{"gid":35093,"name":"b","rank":0,"incarnation":22,"state":"up:clientreplay","state_seq":4,"addr":"172.21.15.138:6835/270300569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":270300569},{"type":"v1","addr":"172.21.15.138:6835","nonce":270300569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-05-20T22:09:33.057 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 25 2024-05-20T22:09:33.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 -- 172.21.15.55:0/180791763 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcca006be60 msgr2=0x7fcca006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:33.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcca006be60 0x7fcca006e370 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fccb0033000 tx=0x7fccb002e000 comp rx=0 tx=0).stop 2024-05-20T22:09:33.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 -- 172.21.15.55:0/180791763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccb415a9d0 msgr2=0x7fccb409d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:33.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccb415a9d0 0x7fccb409d630 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fccb40cbd80 tx=0x7fccac0115a0 comp rx=0 tx=0).stop 2024-05-20T22:09:33.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 -- 172.21.15.55:0/180791763 shutdown_connections 2024-05-20T22:09:33.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcca006be60 0x7fcca006e370 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:33.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccb415a9d0 0x7fccb409d630 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:33.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb41589f0 0x7fccb415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:33.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 --2- 172.21.15.55:0/180791763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc9c0039b0 0x7fccb415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:33.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 -- 172.21.15.55:0/180791763 >> 172.21.15.55:0/180791763 conn(0x7fccb409a400 msgr2=0x7fccb40af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:33.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 -- 172.21.15.55:0/180791763 shutdown_connections 2024-05-20T22:09:33.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:33.057+0000 7fccb999d640 1 -- 172.21.15.55:0/180791763 wait complete. 2024-05-20T22:09:33.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:09:14.957632+0000', 'modified': '2024-05-20T22:09:31.239510+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 42, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35093}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35093': {'gid': 35093, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:clientreplay', 'state_seq': 4, 'addr': '172.21.15.138:6835/270300569', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 270300569}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 270300569}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T22:09:33.472 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_35093:up:clientreplay 2024-05-20T22:09:34.228 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-05-20T22:09:34.227+0000 7f7fa3a1d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T22:09:34.474 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:09:34.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.603+0000 7f79bafce640 1 Processor -- start 2024-05-20T22:09:34.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.603+0000 7f79bafce640 1 -- start start 2024-05-20T22:09:34.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79bafce640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79b40d3400 0x7f79b41600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:34.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79bafce640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f79b415cd50 0x7f79b40cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:34.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79bafce640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b40d44d0 con 0x7f79b40cfce0 2024-05-20T22:09:34.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79bafce640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f79b41605f0 con 0x7f79b415cd50 2024-05-20T22:09:34.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79bafce640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f79b40d00f0 con 0x7f79b40d3400 2024-05-20T22:09:34.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79b3fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f79b415cd50 0x7f79b40cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:34.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79b3fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f79b415cd50 0x7f79b40cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52164/0 (socket says 172.21.15.55:52164) 2024-05-20T22:09:34.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79b3fff640 1 -- 172.21.15.55:0/2537168793 learned_addr learned my addr 172.21.15.55:0/2537168793 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:34.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79b8d6a640 1 --2- 172.21.15.55:0/2537168793 >> v2:172.21.15.138:3300/0 conn(0x7f79b40d3400 0x7f79b41600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:34.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79b3fff640 1 -- 172.21.15.55:0/2537168793 >> v2:172.21.15.138:3300/0 conn(0x7f79b40d3400 msgr2=0x7f79b41600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:34.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79b3fff640 1 --2- 172.21.15.55:0/2537168793 >> v2:172.21.15.138:3300/0 conn(0x7f79b40d3400 0x7f79b41600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:34.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79b3fff640 1 -- 172.21.15.55:0/2537168793 >> v1:172.21.15.138:6789/0 conn(0x7f79b40cfce0 legacy=0x7f79b40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:34.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.605+0000 7f79b3fff640 1 -- 172.21.15.55:0/2537168793 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b40d2fe0 con 0x7f79b415cd50 2024-05-20T22:09:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.606+0000 7f79b3fff640 1 --2- 172.21.15.55:0/2537168793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 0x7f79b40cf7a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f79b40d54f0 tx=0x7f79a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28efe204963e1d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.606+0000 7f79b37fe640 1 -- 172.21.15.55:0/2537168793 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8037070 con 0x7f79b415cd50 2024-05-20T22:09:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.606+0000 7f79b37fe640 1 -- 172.21.15.55:0/2537168793 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a8032440 con 0x7f79b415cd50 2024-05-20T22:09:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.606+0000 7f79b37fe640 1 -- 172.21.15.55:0/2537168793 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8032810 con 0x7f79b415cd50 2024-05-20T22:09:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.607+0000 7f79bafce640 1 -- 172.21.15.55:0/2537168793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 msgr2=0x7f79b40cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.607+0000 7f79bafce640 1 --2- 172.21.15.55:0/2537168793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 0x7f79b40cf7a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f79b40d54f0 tx=0x7f79a802a230 comp rx=0 tx=0).stop 2024-05-20T22:09:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.607+0000 7f79bafce640 1 -- 172.21.15.55:0/2537168793 shutdown_connections 2024-05-20T22:09:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.607+0000 7f79bafce640 1 --2- 172.21.15.55:0/2537168793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 0x7f79b40cf7a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:34.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.607+0000 7f79bafce640 1 --2- 172.21.15.55:0/2537168793 >> v2:172.21.15.138:3300/0 conn(0x7f79b40d3400 0x7f79b41600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:34.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.607+0000 7f79bafce640 1 -- 172.21.15.55:0/2537168793 >> 172.21.15.55:0/2537168793 conn(0x7f79b40d1c80 msgr2=0x7f79b40d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:34.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.607+0000 7f79bafce640 1 -- 172.21.15.55:0/2537168793 shutdown_connections 2024-05-20T22:09:34.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.607+0000 7f79bafce640 1 -- 172.21.15.55:0/2537168793 wait complete. 2024-05-20T22:09:34.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.609+0000 7f79bafce640 1 Processor -- start 2024-05-20T22:09:34.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.609+0000 7f79bafce640 1 -- start start 2024-05-20T22:09:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.610+0000 7f79bafce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b40cfce0 0x7f79b41e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.610+0000 7f79bafce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79b40d3400 0x7f79b4146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.610+0000 7f79bafce640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 0x7f79b4161da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.610+0000 7f79bafce640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f79b40d3f10 con 0x7f79b415cd50 2024-05-20T22:09:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.610+0000 7f79bafce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79b40d3c30 con 0x7f79b40cfce0 2024-05-20T22:09:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.610+0000 7f79bafce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f79b40d41f0 con 0x7f79b40d3400 2024-05-20T22:09:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.610+0000 7f79b956b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 0x7f79b4161da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.610+0000 7f79b956b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 0x7f79b4161da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52170/0 (socket says 172.21.15.55:52170) 2024-05-20T22:09:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.610+0000 7f79b956b640 1 -- 172.21.15.55:0/18707401 learned_addr learned my addr 172.21.15.55:0/18707401 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b8d6a640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b40cfce0 0x7f79b41e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b3fff640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79b40d3400 0x7f79b4146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b956b640 1 -- 172.21.15.55:0/18707401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79b40d3400 msgr2=0x7f79b4146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b956b640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79b40d3400 0x7f79b4146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b956b640 1 -- 172.21.15.55:0/18707401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b40cfce0 msgr2=0x7f79b41e96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b956b640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b40cfce0 0x7f79b41e96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b956b640 1 -- 172.21.15.55:0/18707401 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b41e8c10 con 0x7f79b415cd50 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b956b640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 0x7f79b4161da0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00b970 tx=0x7f79ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b17fa640 1 -- 172.21.15.55:0/18707401 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac019070 con 0x7f79b415cd50 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b17fa640 1 -- 172.21.15.55:0/18707401 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ac00cae0 con 0x7f79b415cd50 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79b17fa640 1 -- 172.21.15.55:0/18707401 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac014440 con 0x7f79b415cd50 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79bafce640 1 -- 172.21.15.55:0/18707401 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b4147840 con 0x7f79b415cd50 2024-05-20T22:09:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.611+0000 7f79bafce640 1 -- 172.21.15.55:0/18707401 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b41e91b0 con 0x7f79b415cd50 2024-05-20T22:09:34.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.613+0000 7f79bafce640 1 -- 172.21.15.55:0/18707401 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b404f720 con 0x7f79b415cd50 2024-05-20T22:09:34.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.613+0000 7f79b17fa640 1 -- 172.21.15.55:0/18707401 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79ac01c070 con 0x7f79b415cd50 2024-05-20T22:09:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.614+0000 7f79b17fa640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79a006be30 0x7f79a006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.614+0000 7f79b17fa640 1 -- 172.21.15.55:0/18707401 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f79ac08ad90 con 0x7f79b415cd50 2024-05-20T22:09:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.614+0000 7f79b8d6a640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79a006be30 0x7f79a006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.615+0000 7f79b8d6a640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79a006be30 0x7f79a006e340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f79a4006fd0 tx=0x7f79a4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:34.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.619+0000 7f79b17fa640 1 -- 172.21.15.55:0/18707401 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f79ac0594c0 con 0x7f79b415cd50 2024-05-20T22:09:34.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.842+0000 7f79bafce640 1 -- 172.21.15.55:0/18707401 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79b40ba790 con 0x7f79b415cd50 2024-05-20T22:09:34.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.842+0000 7f79b17fa640 1 -- 172.21.15.55:0/18707401 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5191 (secure 0 0 0) 0x7f79ac058dc0 con 0x7f79b415cd50 2024-05-20T22:09:34.843 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:09:34.843 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3873372908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3873372908},{"type":"v1","addr":"172.21.15.55:6836","nonce":3873372908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3417305427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":3417305427},{"type":"v1","addr":"172.21.15.55:6837","nonce":3417305427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":35081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4211704780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4211704780},{"type":"v1","addr":"172.21.15.138:6833","nonce":4211704780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:09:14.957632+0000","modified":"2024-05-20T22:09:33.242208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":35093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35093":{"gid":35093,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6835/270300569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":270300569},{"type":"v1","addr":"172.21.15.138:6835","nonce":270300569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":35093,"qdb_cluster":[35093]},"id":2}]} 2024-05-20T22:09:34.844 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2024-05-20T22:09:34.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 -- 172.21.15.55:0/18707401 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79a006be30 msgr2=0x7f79a006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:34.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79a006be30 0x7f79a006e340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f79a4006fd0 tx=0x7f79a4008040 comp rx=0 tx=0).stop 2024-05-20T22:09:34.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 -- 172.21.15.55:0/18707401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 msgr2=0x7f79b4161da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:34.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 0x7f79b4161da0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00b970 tx=0x7f79ac00be30 comp rx=0 tx=0).stop 2024-05-20T22:09:34.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 -- 172.21.15.55:0/18707401 shutdown_connections 2024-05-20T22:09:34.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79a006be30 0x7f79a006e340 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:34.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79b415cd50 0x7f79b4161da0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:34.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79b40d3400 0x7f79b4146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:34.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 --2- 172.21.15.55:0/18707401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b40cfce0 0x7f79b41e96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:34.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 -- 172.21.15.55:0/18707401 >> 172.21.15.55:0/18707401 conn(0x7f79b40d1c80 msgr2=0x7f79b4067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:34.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 -- 172.21.15.55:0/18707401 shutdown_connections 2024-05-20T22:09:34.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:34.845+0000 7f79bafce640 1 -- 172.21.15.55:0/18707401 wait complete. 2024-05-20T22:09:35.261 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:09:14.957632+0000', 'modified': '2024-05-20T22:09:33.242208+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 42, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35093}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35093': {'gid': 35093, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.138:6835/270300569', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 270300569}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 270300569}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 35093, 'qdb_cluster': [35093]} 2024-05-20T22:09:35.261 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T22:09:35.261 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-20T22:09:35.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-05-20T22:09:35.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.382+0000 7f8cc945a640 1 Processor -- start 2024-05-20T22:09:35.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.382+0000 7f8cc945a640 1 -- start start 2024-05-20T22:09:35.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.383+0000 7f8cc945a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8cc40d3460 0x7f8cc415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.383+0000 7f8cc945a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8cc415cd00 0x7f8cc40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.383+0000 7f8cc945a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cc40cfd00 0x7f8cc40b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.383+0000 7f8cc945a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f8cc40d0110 con 0x7f8cc40d3460 2024-05-20T22:09:35.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.383+0000 7f8cc945a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8cc415d110 con 0x7f8cc415cd00 2024-05-20T22:09:35.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.383+0000 7f8cc945a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc40d3a20 con 0x7f8cc40cfd00 2024-05-20T22:09:35.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc2ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8cc40d3460 0x7f8cc415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc2ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8cc40d3460 0x7f8cc415f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52176/0 (socket says 172.21.15.55:52176) 2024-05-20T22:09:35.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc2ffd640 1 -- 172.21.15.55:0/368619227 learned_addr learned my addr 172.21.15.55:0/368619227 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:35.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc37fe640 1 --2- 172.21.15.55:0/368619227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cc40cfd00 0x7f8cc40b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc2ffd640 1 -- 172.21.15.55:0/368619227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cc40cfd00 msgr2=0x7f8cc40b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:35.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/368619227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cc40cfd00 0x7f8cc40b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/368619227 >> v2:172.21.15.138:3300/0 conn(0x7f8cc415cd00 0x7f8cc40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc2ffd640 1 -- 172.21.15.55:0/368619227 >> v2:172.21.15.138:3300/0 conn(0x7f8cc415cd00 msgr2=0x7f8cc40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:35.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/368619227 >> v2:172.21.15.138:3300/0 conn(0x7f8cc415cd00 0x7f8cc40cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc2ffd640 1 -- 172.21.15.55:0/368619227 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc40d3d00 con 0x7f8cc40d3460 2024-05-20T22:09:35.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.384+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/368619227 >> v2:172.21.15.138:3300/0 conn(0x7f8cc415cd00 0x7f8cc40cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:35.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.385+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/368619227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40d3460 0x7f8cc415f6e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40cbb70 tx=0x7f8cb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b5e435625caa2c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.385+0000 7f8cc1ffb640 1 -- 172.21.15.55:0/368619227 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4037070 con 0x7f8cc40d3460 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.385+0000 7f8cc1ffb640 1 -- 172.21.15.55:0/368619227 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb4032440 con 0x7f8cc40d3460 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.385+0000 7f8cc1ffb640 1 -- 172.21.15.55:0/368619227 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4032810 con 0x7f8cc40d3460 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.386+0000 7f8cc945a640 1 -- 172.21.15.55:0/368619227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40d3460 msgr2=0x7f8cc415f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.386+0000 7f8cc945a640 1 --2- 172.21.15.55:0/368619227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40d3460 0x7f8cc415f6e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40cbb70 tx=0x7f8cb402a1e0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.386+0000 7f8cc945a640 1 -- 172.21.15.55:0/368619227 shutdown_connections 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.386+0000 7f8cc945a640 1 --2- 172.21.15.55:0/368619227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cc40cfd00 0x7f8cc40b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.386+0000 7f8cc945a640 1 --2- 172.21.15.55:0/368619227 >> v2:172.21.15.138:3300/0 conn(0x7f8cc415cd00 0x7f8cc40cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.386+0000 7f8cc945a640 1 --2- 172.21.15.55:0/368619227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40d3460 0x7f8cc415f6e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.386+0000 7f8cc945a640 1 -- 172.21.15.55:0/368619227 >> 172.21.15.55:0/368619227 conn(0x7f8cc40d1c70 msgr2=0x7f8cc40d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.386+0000 7f8cc945a640 1 -- 172.21.15.55:0/368619227 shutdown_connections 2024-05-20T22:09:35.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.386+0000 7f8cc945a640 1 -- 172.21.15.55:0/368619227 wait complete. 2024-05-20T22:09:35.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.388+0000 7f8cc945a640 1 Processor -- start 2024-05-20T22:09:35.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.388+0000 7f8cc945a640 1 -- start start 2024-05-20T22:09:35.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc945a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40cfd00 0x7f8cc41f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc945a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cc40d3460 0x7f8cc40caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc945a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cc415cd00 0x7f8cc40cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc945a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc40d04a0 con 0x7f8cc40cfd00 2024-05-20T22:09:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc945a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc40d42c0 con 0x7f8cc415cd00 2024-05-20T22:09:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc945a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc40d0620 con 0x7f8cc40d3460 2024-05-20T22:09:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40cfd00 0x7f8cc41f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40cfd00 0x7f8cc41f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52178/0 (socket says 172.21.15.55:52178) 2024-05-20T22:09:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc2ffd640 1 -- 172.21.15.55:0/587140174 learned_addr learned my addr 172.21.15.55:0/587140174 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc2ffd640 1 -- 172.21.15.55:0/587140174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cc40d3460 msgr2=0x7f8cc40caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cc40d3460 0x7f8cc40caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc2ffd640 1 -- 172.21.15.55:0/587140174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cc415cd00 msgr2=0x7f8cc40cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cc415cd00 0x7f8cc40cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.389+0000 7f8cc2ffd640 1 -- 172.21.15.55:0/587140174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc40cbcf0 con 0x7f8cc40cfd00 2024-05-20T22:09:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.390+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40cfd00 0x7f8cc41f4f70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8cb402a220 tx=0x7f8cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.390+0000 7f8cab7fe640 1 -- 172.21.15.55:0/587140174 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4041020 con 0x7f8cc40cfd00 2024-05-20T22:09:35.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.390+0000 7f8cab7fe640 1 -- 172.21.15.55:0/587140174 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb40306c0 con 0x7f8cc40cfd00 2024-05-20T22:09:35.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.390+0000 7f8cc945a640 1 -- 172.21.15.55:0/587140174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc40cbfd0 con 0x7f8cc40cfd00 2024-05-20T22:09:35.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.390+0000 7f8cab7fe640 1 -- 172.21.15.55:0/587140174 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4037040 con 0x7f8cc40cfd00 2024-05-20T22:09:35.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.390+0000 7f8cc945a640 1 -- 172.21.15.55:0/587140174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc40cc590 con 0x7f8cc40cfd00 2024-05-20T22:09:35.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.392+0000 7f8cab7fe640 1 -- 172.21.15.55:0/587140174 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8cb4044020 con 0x7f8cc40cfd00 2024-05-20T22:09:35.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.392+0000 7f8cab7fe640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8cb006be60 0x7f8cb006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.392+0000 7f8cab7fe640 1 -- 172.21.15.55:0/587140174 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f8cb40ab980 con 0x7f8cc40cfd00 2024-05-20T22:09:35.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.392+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8cb006be60 0x7f8cb006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.393+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8cb006be60 0x7f8cb006e370 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8006fd0 tx=0x7f8cb8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:35.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.393+0000 7f8cd03dc740 1 Processor -- start 2024-05-20T22:09:35.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.393+0000 7f8cd03dc740 1 -- start start 2024-05-20T22:09:35.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 0x5654c6d4b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c69573b0 0x5654c6d480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 0x5654c6d21360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5654c6c844d0 con 0x5654c69573b0 2024-05-20T22:09:35.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654c6c2c3e0 con 0x5654c6980270 2024-05-20T22:09:35.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5654c6c80490 con 0x5654c695f590 2024-05-20T22:09:35.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c69573b0 0x5654c6d480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c69573b0 0x5654c6d480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52186/0 (socket says 172.21.15.55:52186) 2024-05-20T22:09:35.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc27fc640 1 -- 172.21.15.55:0/2700529800 learned_addr learned my addr 172.21.15.55:0/2700529800 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:35.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/2700529800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 0x5654c6d4b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc37fe640 1 --2- 172.21.15.55:0/2700529800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 0x5654c6d21360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc27fc640 1 -- 172.21.15.55:0/2700529800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 msgr2=0x5654c6d21360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:35.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/2700529800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 0x5654c6d21360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc27fc640 1 -- 172.21.15.55:0/2700529800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 msgr2=0x5654c6d4b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:35.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/2700529800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 0x5654c6d4b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc27fc640 1 -- 172.21.15.55:0/2700529800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c69982d0 con 0x5654c69573b0 2024-05-20T22:09:35.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/2700529800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 0x5654c6d4b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:09:35.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/2700529800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c69573b0 0x5654c6d480a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800caf0 tx=0x7f8cb800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dba0d26ac88b65 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:35.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.395+0000 7f8ca8ff9640 1 -- 172.21.15.55:0/2700529800 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8019040 con 0x5654c69573b0 2024-05-20T22:09:35.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.396+0000 7f8ca8ff9640 1 -- 172.21.15.55:0/2700529800 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb8022070 con 0x5654c69573b0 2024-05-20T22:09:35.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.396+0000 7f8ca8ff9640 1 -- 172.21.15.55:0/2700529800 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8013de0 con 0x5654c69573b0 2024-05-20T22:09:35.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.396+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2700529800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c69573b0 msgr2=0x5654c6d480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:35.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.396+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2700529800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c69573b0 0x5654c6d480a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800caf0 tx=0x7f8cb800f000 comp rx=0 tx=0).stop 2024-05-20T22:09:35.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.396+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2700529800 shutdown_connections 2024-05-20T22:09:35.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.396+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2700529800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 0x5654c6d21360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.396+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2700529800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c69573b0 0x5654c6d480a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.396+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2700529800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 0x5654c6d4b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.396+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2700529800 >> 172.21.15.55:0/2700529800 conn(0x5654c69743c0 msgr2=0x5654c6d20af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.396+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2700529800 shutdown_connections 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.397+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2700529800 wait complete. 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.397+0000 7f8cd03dc740 1 -- start start 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.397+0000 7f8cd03dc740 1 Processor -- start 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 0x5654c6d4d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c695f590 0x5654c6d4dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 0x5654c6d64970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5654c6c3f8e0 con 0x5654c695f590 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654c6d65cb0 con 0x5654c6980270 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5654c6d69760 con 0x5654c69573b0 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cd03dc740 10 client.0 _ll_register_callbacks cb 0x5654c6c0c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:09:35.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c695f590 0x5654c6d4dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c695f590 0x5654c6d4dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52190/0 (socket says 172.21.15.55:52190) 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc27fc640 1 -- 172.21.15.55:0/2919445342 learned_addr learned my addr 172.21.15.55:0/2919445342 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 0x5654c6d4d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc37fe640 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 0x5654c6d64970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc27fc640 1 -- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 msgr2=0x5654c6d4d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 0x5654c6d4d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc27fc640 1 -- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 msgr2=0x5654c6d64970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 0x5654c6d64970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc27fc640 1 -- 172.21.15.55:0/2919445342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c6a626b0 con 0x5654c695f590 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 0x5654c6d4d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc37fe640 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 0x5654c6d64970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c695f590 0x5654c6d4dec0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8008010 tx=0x7f8cb8006bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3778def4e3088e05 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.399+0000 7f8ca1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/2919445342 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb802c070 con 0x5654c695f590 2024-05-20T22:09:35.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/2919445342 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f8cb8030600 con 0x5654c695f590 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2919445342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb8022070 con 0x5654c695f590 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/2919445342 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8cb803a050 con 0x5654c695f590 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8cd03dc740 10 client.5787 fetch_fsmap learned FSMap version 26 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2919445342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5654c6d66f10 con 0x5654c695f590 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/2919445342 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb8022070 con 0x5654c695f590 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/2919445342 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f8cb803d020 con 0x5654c695f590 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8cd03dc740 10 client.5787 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8cd03dc740 10 client.5787 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8cd03dc740 20 client.5787 populate_metadata read hostname 'smithi055' 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:6834/270300569,v1:172.21.15.138:6835/270300569] conn(0x5654c693ee40 0x5654c6d83600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8cd03dc740 4 client.5787 mds_command: new command op to 35093 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2919445342 --> [v2:172.21.15.138:6834/270300569,v1:172.21.15.138:6835/270300569] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654c6c91c00 con 0x5654c693ee40 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.400+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:6834/270300569,v1:172.21.15.138:6835/270300569] conn(0x5654c693ee40 0x5654c6d83600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.401+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:6834/270300569,v1:172.21.15.138:6835/270300569] conn(0x5654c693ee40 0x5654c6d83600 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e24aca5faff89b server_cookie=dc8e18c3f7c2e761 in_seq=0 out_seq=0 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.401+0000 7f8ca1ffb640 10 client.5787 ms_handle_connect on v2:172.21.15.138:6834/270300569 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/2919445342 <== mds.0 v2:172.21.15.138:6834/270300569 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654c6c91c00 con 0x5654c693ee40 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8ca1ffb640 10 client.5787 handle_command_reply: tid=0 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 client.5787 shutdown 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2919445342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c695f590 msgr2=0x5654c6d4dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c695f590 0x5654c6d4dec0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8008010 tx=0x7f8cb8006bd0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2919445342 shutdown_connections 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:6834/270300569,v1:172.21.15.138:6835/270300569] conn(0x5654c693ee40 0x5654c6d83600 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6980270 0x5654c6d64970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c695f590 0x5654c6d4dec0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8ca1ffb640 0 client.5787 ms_handle_reset on v2:172.21.15.138:6834/270300569 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2919445342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 0x5654c6d4d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2919445342 >> 172.21.15.55:0/2919445342 conn(0x5654c69743c0 msgr2=0x5654c6d1fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:35.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2919445342 shutdown_connections 2024-05-20T22:09:35.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2919445342 wait complete. 2024-05-20T22:09:35.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.402+0000 7f8cd03dc740 20 client.5787 trim_cache size 0 max 16384 2024-05-20T22:09:35.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.421+0000 7f8cd03dc740 1 Processor -- start 2024-05-20T22:09:35.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.421+0000 7f8cd03dc740 1 -- start start 2024-05-20T22:09:35.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.422+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 0x5654c6d441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.422+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c695f590 0x5654c6d42f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.422+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d43470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.422+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc4201550 con 0x5654c6980270 2024-05-20T22:09:35.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.422+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc4161c90 con 0x5654c695f590 2024-05-20T22:09:35.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.422+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc4200f30 con 0x5654c69573b0 2024-05-20T22:09:35.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.422+0000 7f8cc37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d43470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.422+0000 7f8cc37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d43470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52200/0 (socket says 172.21.15.55:52200) 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.422+0000 7f8cc37fe640 1 -- 172.21.15.55:0/2160117305 learned_addr learned my addr 172.21.15.55:0/2160117305 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/2160117305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c695f590 0x5654c6d42f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/2160117305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 0x5654c6d441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8cc37fe640 1 -- 172.21.15.55:0/2160117305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 msgr2=0x5654c6d441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8cc37fe640 1 --2- 172.21.15.55:0/2160117305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 0x5654c6d441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8cc37fe640 1 -- 172.21.15.55:0/2160117305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c695f590 msgr2=0x5654c6d42f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8cc37fe640 1 --2- 172.21.15.55:0/2160117305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c695f590 0x5654c6d42f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8cc37fe640 1 -- 172.21.15.55:0/2160117305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c6d64ce0 con 0x5654c6980270 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8cc37fe640 1 --2- 172.21.15.55:0/2160117305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d43470 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0041d0 tx=0x7f8cbc004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3928091c05f418 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8ca3fff640 1 -- 172.21.15.55:0/2160117305 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc009ce0 con 0x5654c6980270 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8ca3fff640 1 -- 172.21.15.55:0/2160117305 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cbc01a070 con 0x5654c6980270 2024-05-20T22:09:35.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.423+0000 7f8ca3fff640 1 -- 172.21.15.55:0/2160117305 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc015640 con 0x5654c6980270 2024-05-20T22:09:35.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.424+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2160117305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 msgr2=0x5654c6d43470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:35.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.424+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2160117305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d43470 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0041d0 tx=0x7f8cbc004690 comp rx=0 tx=0).stop 2024-05-20T22:09:35.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.424+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2160117305 shutdown_connections 2024-05-20T22:09:35.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.424+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2160117305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d43470 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.424+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2160117305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c695f590 0x5654c6d42f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.424+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/2160117305 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c69573b0 0x5654c6d441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.424+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2160117305 >> 172.21.15.55:0/2160117305 conn(0x5654c69743c0 msgr2=0x5654c6d4abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:35.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.424+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2160117305 shutdown_connections 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.424+0000 7f8cd03dc740 1 -- 172.21.15.55:0/2160117305 wait complete. 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.425+0000 7f8cd03dc740 1 -- start start 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.425+0000 7f8cd03dc740 1 Processor -- start 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c69573b0 0x5654c6d47a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 0x5654c6de2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cd03dc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d5c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5654c6d46b90 con 0x5654c6980270 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654c6d659f0 con 0x5654c69573b0 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cd03dc740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5654c6d65400 con 0x5654c695f590 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cd03dc740 10 client.0 _ll_register_callbacks cb 0x7f8cb800c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:09:35.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d5c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d5c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52202/0 (socket says 172.21.15.55:52202) 2024-05-20T22:09:35.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc37fe640 1 -- 172.21.15.55:0/4202144242 learned_addr learned my addr 172.21.15.55:0/4202144242 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:35.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c69573b0 0x5654c6d47a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 0x5654c6de2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc37fe640 1 -- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 msgr2=0x5654c6de2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:35.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc37fe640 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 0x5654c6de2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc37fe640 1 -- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c69573b0 msgr2=0x5654c6d47a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc37fe640 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c69573b0 0x5654c6d47a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc37fe640 1 -- 172.21.15.55:0/4202144242 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c6d1a220 con 0x5654c6980270 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c69573b0 0x5654c6d47a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.426+0000 7f8cc27fc640 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 0x5654c6de2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8cc37fe640 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d5c360 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc000f80 tx=0x7f8cbc007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cf6ca2e88f96c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8ca1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/4202144242 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc015730 con 0x5654c6980270 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/4202144242 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f8cbc0158d0 con 0x5654c6980270 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8cd03dc740 1 -- 172.21.15.55:0/4202144242 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cbc01a070 con 0x5654c6980270 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/4202144242 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8cbc02f050 con 0x5654c6980270 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8cd03dc740 10 client.5793 fetch_fsmap learned FSMap version 26 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8cd03dc740 1 -- 172.21.15.55:0/4202144242 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5654c6d5bf40 con 0x5654c6980270 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/4202144242 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cbc01a070 con 0x5654c6980270 2024-05-20T22:09:35.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/4202144242 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f8cbc032020 con 0x5654c6980270 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8cd03dc740 10 client.5793 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8cd03dc740 10 client.5793 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8cd03dc740 20 client.5793 populate_metadata read hostname 'smithi055' 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:6834/270300569,v1:172.21.15.138:6835/270300569] conn(0x5654c6d67fe0 0x5654c6d683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8cd03dc740 4 client.5793 mds_command: new command op to 35093 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.427+0000 7f8cd03dc740 1 -- 172.21.15.55:0/4202144242 --> [v2:172.21.15.138:6834/270300569,v1:172.21.15.138:6835/270300569] -- command(tid 0: {"prefix": "status"}) -- 0x5654c6c8bf50 con 0x5654c6d67fe0 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.428+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:6834/270300569,v1:172.21.15.138:6835/270300569] conn(0x5654c6d67fe0 0x5654c6d683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.428+0000 7f8ca1ffb640 10 client.5793 ms_handle_connect on v2:172.21.15.138:6834/270300569 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.428+0000 7f8cc2ffd640 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:6834/270300569,v1:172.21.15.138:6835/270300569] conn(0x5654c6d67fe0 0x5654c6d683f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2741aae1736d7540 server_cookie=6bda6cd2f2d524de in_seq=0 out_seq=0 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.428+0000 7f8ca1ffb640 1 -- 172.21.15.55:0/4202144242 <== mds.0 v2:172.21.15.138:6834/270300569 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5654c6c8bf50 con 0x5654c6d67fe0 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.428+0000 7f8ca1ffb640 10 client.5793 handle_command_reply: tid=0 2024-05-20T22:09:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.428+0000 7f8cd03dc740 1 client.5793 shutdown 2024-05-20T22:09:35.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 1 -- 172.21.15.55:0/4202144242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 msgr2=0x5654c6d5c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:35.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d5c360 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc000f80 tx=0x7f8cbc007040 comp rx=0 tx=0).stop 2024-05-20T22:09:35.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 1 -- 172.21.15.55:0/4202144242 shutdown_connections 2024-05-20T22:09:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:6834/270300569,v1:172.21.15.138:6835/270300569] conn(0x5654c6d67fe0 0x5654c6d683f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5654c6980270 0x5654c6d5c360 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654c695f590 0x5654c6de2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8ca1ffb640 0 client.5793 ms_handle_reset on v2:172.21.15.138:6834/270300569 2024-05-20T22:09:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 1 --2- 172.21.15.55:0/4202144242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c69573b0 0x5654c6d47a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 1 -- 172.21.15.55:0/4202144242 >> 172.21.15.55:0/4202144242 conn(0x5654c69743c0 msgr2=0x5654c6d627d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 1 -- 172.21.15.55:0/4202144242 shutdown_connections 2024-05-20T22:09:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 1 -- 172.21.15.55:0/4202144242 wait complete. 2024-05-20T22:09:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cd03dc740 20 client.5793 trim_cache size 0 max 16384 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "de8ca569-3fd1-4cb2-8bdc-99f367745927", 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 35093, 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 6.2027997959999999, 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 26, 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 44, 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 42, 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 6.2140218369999998 2024-05-20T22:09:35.436 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:09:35.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cc945a640 1 -- 172.21.15.55:0/587140174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8cb006be60 msgr2=0x7f8cb006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:35.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cc945a640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8cb006be60 0x7f8cb006e370 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8006fd0 tx=0x7f8cb8008040 comp rx=0 tx=0).stop 2024-05-20T22:09:35.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cc945a640 1 -- 172.21.15.55:0/587140174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40cfd00 msgr2=0x7f8cc41f4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:35.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cc945a640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40cfd00 0x7f8cc41f4f70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8cb402a220 tx=0x7f8cb40046c0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cc945a640 1 -- 172.21.15.55:0/587140174 shutdown_connections 2024-05-20T22:09:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cc945a640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8cb006be60 0x7f8cb006e370 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cc945a640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cc415cd00 0x7f8cc40cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cc945a640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8cc40d3460 0x7f8cc40caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cc945a640 1 --2- 172.21.15.55:0/587140174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc40cfd00 0x7f8cc41f4f70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.429+0000 7f8cc945a640 1 -- 172.21.15.55:0/587140174 >> 172.21.15.55:0/587140174 conn(0x7f8cc40d1c70 msgr2=0x7f8cc4161fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.430+0000 7f8cc945a640 1 -- 172.21.15.55:0/587140174 shutdown_connections 2024-05-20T22:09:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:35.430+0000 7f8cc945a640 1 -- 172.21.15.55:0/587140174 wait complete. 2024-05-20T22:09:35.442 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:35.442 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> stdin-killer -- python3 -c ' 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> import fcntl 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> import sys 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> import time 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> with open("/home/ubuntu/cephtest/mnt.0/open_file0", '"'"'w'"'"') as f: 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> f.write("content") 2024-05-20T22:09:35.459 DEBUG:teuthology.orchestra.run.smithi055:> f.flush() 2024-05-20T22:09:35.460 DEBUG:teuthology.orchestra.run.smithi055:> while True: 2024-05-20T22:09:35.460 DEBUG:teuthology.orchestra.run.smithi055:> print("open_background: keeping file open", file=sys.stderr) 2024-05-20T22:09:35.460 DEBUG:teuthology.orchestra.run.smithi055:> try: 2024-05-20T22:09:35.460 DEBUG:teuthology.orchestra.run.smithi055:> if os.read(0, 4096) == b"": 2024-05-20T22:09:35.460 DEBUG:teuthology.orchestra.run.smithi055:> break 2024-05-20T22:09:35.460 DEBUG:teuthology.orchestra.run.smithi055:> except BlockingIOError: 2024-05-20T22:09:35.460 DEBUG:teuthology.orchestra.run.smithi055:> pass 2024-05-20T22:09:35.460 DEBUG:teuthology.orchestra.run.smithi055:> time.sleep(2) 2024-05-20T22:09:35.460 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:09:35.501 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file0 2024-05-20T22:09:35.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:09:35.517 INFO:teuthology.orchestra.run.smithi055.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file0': No such file or directory 2024-05-20T22:09:35.574 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:36.518 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file0 2024-05-20T22:09:36.539 INFO:teuthology.orchestra.run.smithi055.stdout:7 2024-05-20T22:09:36.539 INFO:tasks.cephfs.mount:File open_file0 became visible with size 7 from 0 after 1s 2024-05-20T22:09:36.539 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:36.539 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:36.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:36.599 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file0 bs=1M conv=fdatasync count=1 seek=0) 2024-05-20T22:09:36.678 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:09:36.678 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:09:36.678 INFO:teuthology.orchestra.run.smithi055.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129771 s, 80.8 MB/s 2024-05-20T22:09:36.679 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:36.679 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:36.739 DEBUG:teuthology.orchestra.run.smithi055:> stdin-killer -- python3 -c ' 2024-05-20T22:09:36.739 DEBUG:teuthology.orchestra.run.smithi055:> import fcntl 2024-05-20T22:09:36.739 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:09:36.739 DEBUG:teuthology.orchestra.run.smithi055:> import sys 2024-05-20T22:09:36.739 DEBUG:teuthology.orchestra.run.smithi055:> import time 2024-05-20T22:09:36.739 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:36.739 DEBUG:teuthology.orchestra.run.smithi055:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-05-20T22:09:36.739 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> with open("/home/ubuntu/cephtest/mnt.0/open_file1", '"'"'w'"'"') as f: 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> f.write("content") 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> f.flush() 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> while True: 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> print("open_background: keeping file open", file=sys.stderr) 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> try: 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> if os.read(0, 4096) == b"": 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> break 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> except BlockingIOError: 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> pass 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> time.sleep(2) 2024-05-20T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:09:36.783 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file1 2024-05-20T22:09:36.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:09:36.802 INFO:teuthology.orchestra.run.smithi055.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file1': No such file or directory 2024-05-20T22:09:36.858 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:37.576 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:37.804 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file1 2024-05-20T22:09:37.824 INFO:teuthology.orchestra.run.smithi055.stdout:7 2024-05-20T22:09:37.825 INFO:tasks.cephfs.mount:File open_file1 became visible with size 7 from 0 after 1s 2024-05-20T22:09:37.825 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:37.825 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:37.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:37.885 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file1 bs=1M conv=fdatasync count=1 seek=0) 2024-05-20T22:09:37.970 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:09:37.971 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:09:37.971 INFO:teuthology.orchestra.run.smithi055.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134922 s, 77.7 MB/s 2024-05-20T22:09:37.971 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:37.971 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:37.990 DEBUG:teuthology.orchestra.run.smithi055:> stdin-killer -- python3 -c ' 2024-05-20T22:09:37.990 DEBUG:teuthology.orchestra.run.smithi055:> import fcntl 2024-05-20T22:09:37.990 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:09:37.990 DEBUG:teuthology.orchestra.run.smithi055:> import sys 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> import time 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> with open("/home/ubuntu/cephtest/mnt.0/open_file2", '"'"'w'"'"') as f: 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> f.write("content") 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> f.flush() 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> while True: 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> print("open_background: keeping file open", file=sys.stderr) 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> try: 2024-05-20T22:09:37.991 DEBUG:teuthology.orchestra.run.smithi055:> if os.read(0, 4096) == b"": 2024-05-20T22:09:37.992 DEBUG:teuthology.orchestra.run.smithi055:> break 2024-05-20T22:09:37.992 DEBUG:teuthology.orchestra.run.smithi055:> except BlockingIOError: 2024-05-20T22:09:37.992 DEBUG:teuthology.orchestra.run.smithi055:> pass 2024-05-20T22:09:37.992 DEBUG:teuthology.orchestra.run.smithi055:> time.sleep(2) 2024-05-20T22:09:37.992 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:09:38.033 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file2 2024-05-20T22:09:38.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:09:38.052 INFO:teuthology.orchestra.run.smithi055.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file2': No such file or directory 2024-05-20T22:09:38.106 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:38.861 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:39.053 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file2 2024-05-20T22:09:39.074 INFO:teuthology.orchestra.run.smithi055.stdout:7 2024-05-20T22:09:39.074 INFO:tasks.cephfs.mount:File open_file2 became visible with size 7 from 0 after 1s 2024-05-20T22:09:39.074 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:39.074 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:39.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:39.133 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file2 bs=1M conv=fdatasync count=1 seek=0) 2024-05-20T22:09:39.213 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:09:39.213 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:09:39.213 INFO:teuthology.orchestra.run.smithi055.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124597 s, 84.2 MB/s 2024-05-20T22:09:39.213 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:39.213 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:39.272 DEBUG:teuthology.orchestra.run.smithi055:> stdin-killer -- python3 -c ' 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> import fcntl 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> import sys 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> import time 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> with open("/home/ubuntu/cephtest/mnt.0/open_file3", '"'"'w'"'"') as f: 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> f.write("content") 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> f.flush() 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> while True: 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> print("open_background: keeping file open", file=sys.stderr) 2024-05-20T22:09:39.273 DEBUG:teuthology.orchestra.run.smithi055:> try: 2024-05-20T22:09:39.274 DEBUG:teuthology.orchestra.run.smithi055:> if os.read(0, 4096) == b"": 2024-05-20T22:09:39.274 DEBUG:teuthology.orchestra.run.smithi055:> break 2024-05-20T22:09:39.274 DEBUG:teuthology.orchestra.run.smithi055:> except BlockingIOError: 2024-05-20T22:09:39.274 DEBUG:teuthology.orchestra.run.smithi055:> pass 2024-05-20T22:09:39.274 DEBUG:teuthology.orchestra.run.smithi055:> time.sleep(2) 2024-05-20T22:09:39.274 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:09:39.315 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file3 2024-05-20T22:09:39.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:09:39.334 INFO:teuthology.orchestra.run.smithi055.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file3': No such file or directory 2024-05-20T22:09:39.389 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:39.578 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:40.335 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file3 2024-05-20T22:09:40.355 INFO:teuthology.orchestra.run.smithi055.stdout:7 2024-05-20T22:09:40.355 INFO:tasks.cephfs.mount:File open_file3 became visible with size 7 from 0 after 1s 2024-05-20T22:09:40.356 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:40.356 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:40.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:40.416 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file3 bs=1M conv=fdatasync count=1 seek=0) 2024-05-20T22:09:40.495 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:09:40.496 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:09:40.496 INFO:teuthology.orchestra.run.smithi055.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118017 s, 88.8 MB/s 2024-05-20T22:09:40.497 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:40.497 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:40.555 DEBUG:teuthology.orchestra.run.smithi055:> stdin-killer -- python3 -c ' 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> import fcntl 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> import sys 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> import time 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> with open("/home/ubuntu/cephtest/mnt.0/open_file4", '"'"'w'"'"') as f: 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> f.write("content") 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> f.flush() 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> while True: 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> print("open_background: keeping file open", file=sys.stderr) 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> try: 2024-05-20T22:09:40.556 DEBUG:teuthology.orchestra.run.smithi055:> if os.read(0, 4096) == b"": 2024-05-20T22:09:40.557 DEBUG:teuthology.orchestra.run.smithi055:> break 2024-05-20T22:09:40.557 DEBUG:teuthology.orchestra.run.smithi055:> except BlockingIOError: 2024-05-20T22:09:40.557 DEBUG:teuthology.orchestra.run.smithi055:> pass 2024-05-20T22:09:40.557 DEBUG:teuthology.orchestra.run.smithi055:> time.sleep(2) 2024-05-20T22:09:40.557 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:09:40.599 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file4 2024-05-20T22:09:40.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:09:40.618 INFO:teuthology.orchestra.run.smithi055.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file4': No such file or directory 2024-05-20T22:09:40.672 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:40.863 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:41.392 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:41.618 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file4 2024-05-20T22:09:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:7 2024-05-20T22:09:41.640 INFO:tasks.cephfs.mount:File open_file4 became visible with size 7 from 0 after 1s 2024-05-20T22:09:41.640 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:41.640 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:41.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:41.700 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file4 bs=1M conv=fdatasync count=1 seek=0) 2024-05-20T22:09:41.780 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:09:41.780 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:09:41.780 INFO:teuthology.orchestra.run.smithi055.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130152 s, 80.6 MB/s 2024-05-20T22:09:41.780 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:41.780 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:41.838 DEBUG:teuthology.orchestra.run.smithi055:> stdin-killer -- python3 -c ' 2024-05-20T22:09:41.838 DEBUG:teuthology.orchestra.run.smithi055:> import fcntl 2024-05-20T22:09:41.838 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> import sys 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> import time 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> with open("/home/ubuntu/cephtest/mnt.0/open_file5", '"'"'w'"'"') as f: 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> f.write("content") 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> f.flush() 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> while True: 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> print("open_background: keeping file open", file=sys.stderr) 2024-05-20T22:09:41.839 DEBUG:teuthology.orchestra.run.smithi055:> try: 2024-05-20T22:09:41.840 DEBUG:teuthology.orchestra.run.smithi055:> if os.read(0, 4096) == b"": 2024-05-20T22:09:41.840 DEBUG:teuthology.orchestra.run.smithi055:> break 2024-05-20T22:09:41.840 DEBUG:teuthology.orchestra.run.smithi055:> except BlockingIOError: 2024-05-20T22:09:41.840 DEBUG:teuthology.orchestra.run.smithi055:> pass 2024-05-20T22:09:41.840 DEBUG:teuthology.orchestra.run.smithi055:> time.sleep(2) 2024-05-20T22:09:41.840 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:09:41.882 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file5 2024-05-20T22:09:41.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:09:41.900 INFO:teuthology.orchestra.run.smithi055.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file5': No such file or directory 2024-05-20T22:09:41.956 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:42.111 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:42.865 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:42.902 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file5 2024-05-20T22:09:42.922 INFO:teuthology.orchestra.run.smithi055.stdout:7 2024-05-20T22:09:42.922 INFO:tasks.cephfs.mount:File open_file5 became visible with size 7 from 0 after 1s 2024-05-20T22:09:42.922 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:42.922 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:42.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:42.982 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file5 bs=1M conv=fdatasync count=1 seek=0) 2024-05-20T22:09:43.065 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:09:43.065 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:09:43.065 INFO:teuthology.orchestra.run.smithi055.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126026 s, 83.2 MB/s 2024-05-20T22:09:43.065 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:43.065 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:43.123 DEBUG:teuthology.orchestra.run.smithi055:> stdin-killer -- python3 -c ' 2024-05-20T22:09:43.123 DEBUG:teuthology.orchestra.run.smithi055:> import fcntl 2024-05-20T22:09:43.123 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:09:43.123 DEBUG:teuthology.orchestra.run.smithi055:> import sys 2024-05-20T22:09:43.123 DEBUG:teuthology.orchestra.run.smithi055:> import time 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> with open("/home/ubuntu/cephtest/mnt.0/open_file6", '"'"'w'"'"') as f: 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> f.write("content") 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> f.flush() 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> while True: 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> print("open_background: keeping file open", file=sys.stderr) 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> try: 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> if os.read(0, 4096) == b"": 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> break 2024-05-20T22:09:43.124 DEBUG:teuthology.orchestra.run.smithi055:> except BlockingIOError: 2024-05-20T22:09:43.125 DEBUG:teuthology.orchestra.run.smithi055:> pass 2024-05-20T22:09:43.125 DEBUG:teuthology.orchestra.run.smithi055:> time.sleep(2) 2024-05-20T22:09:43.125 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:09:43.166 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file6 2024-05-20T22:09:43.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:09:43.185 INFO:teuthology.orchestra.run.smithi055.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file6': No such file or directory 2024-05-20T22:09:43.240 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:43.385 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:09:43.388 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:09:43.393 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:43.583 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:43.958 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:44.186 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file6 2024-05-20T22:09:44.207 INFO:teuthology.orchestra.run.smithi055.stdout:7 2024-05-20T22:09:44.208 INFO:tasks.cephfs.mount:File open_file6 became visible with size 7 from 0 after 1s 2024-05-20T22:09:44.208 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:44.208 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:44.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:44.267 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file6 bs=1M conv=fdatasync count=1 seek=0) 2024-05-20T22:09:44.344 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:09:44.344 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:09:44.344 INFO:teuthology.orchestra.run.smithi055.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011168 s, 93.9 MB/s 2024-05-20T22:09:44.345 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:44.345 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:44.404 DEBUG:teuthology.orchestra.run.smithi055:> stdin-killer -- python3 -c ' 2024-05-20T22:09:44.404 DEBUG:teuthology.orchestra.run.smithi055:> import fcntl 2024-05-20T22:09:44.404 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:09:44.404 DEBUG:teuthology.orchestra.run.smithi055:> import sys 2024-05-20T22:09:44.404 DEBUG:teuthology.orchestra.run.smithi055:> import time 2024-05-20T22:09:44.404 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:44.404 DEBUG:teuthology.orchestra.run.smithi055:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> with open("/home/ubuntu/cephtest/mnt.0/open_file7", '"'"'w'"'"') as f: 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> f.write("content") 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> f.flush() 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> while True: 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> print("open_background: keeping file open", file=sys.stderr) 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> try: 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> if os.read(0, 4096) == b"": 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> break 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> except BlockingIOError: 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> pass 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> time.sleep(2) 2024-05-20T22:09:44.405 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:09:44.446 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file7 2024-05-20T22:09:44.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:09:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file7': No such file or directory 2024-05-20T22:09:44.522 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:44.676 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:44.867 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:45.242 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:45.395 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:45.466 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file7 2024-05-20T22:09:45.487 INFO:teuthology.orchestra.run.smithi055.stdout:7 2024-05-20T22:09:45.487 INFO:tasks.cephfs.mount:File open_file7 became visible with size 7 from 0 after 1s 2024-05-20T22:09:45.487 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:09:45.487 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:09:45.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:09:45.546 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file7 bs=1M conv=fdatasync count=1 seek=0) 2024-05-20T22:09:45.584 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:45.624 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-20T22:09:45.624 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-20T22:09:45.624 INFO:teuthology.orchestra.run.smithi055.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920598 s, 114 MB/s 2024-05-20T22:09:45.960 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:46.115 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:46.524 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:46.678 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:46.869 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:47.244 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:47.398 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:47.586 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:47.962 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:48.117 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:48.526 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:48.680 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:48.871 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:49.247 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:49.400 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:49.588 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:49.964 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:50.119 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:50.528 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:50.682 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:50.874 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:51.249 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:51.402 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:51.591 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:51.967 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:52.121 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:52.530 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:52.684 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:52.876 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:53.251 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:53.404 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:53.593 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:53.969 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:54.124 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:54.532 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:54.687 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:54.878 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:55.406 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:55.594 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:55.626 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.1 2024-05-20T22:09:55.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.690+0000 7f640200df40 1 Processor -- start 2024-05-20T22:09:55.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.690+0000 7f640200df40 1 -- start start 2024-05-20T22:09:55.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.691+0000 7f640200df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565420c05140 0x565420c06f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:55.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.691+0000 7f640200df40 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420caf400 0x565420c06030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:55.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.691+0000 7f640200df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565420cae9d0 0x565420ca6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:55.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.691+0000 7f640200df40 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x565420c090c0 con 0x565420caf400 2024-05-20T22:09:55.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.691+0000 7f640200df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565420b82c80 con 0x565420cae9d0 2024-05-20T22:09:55.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.691+0000 7f640200df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x565420c09360 con 0x565420c05140 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.691+0000 7f63fa5cb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420caf400 0x565420c06030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.691+0000 7f63fa5cb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420caf400 0x565420c06030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39594/0 (socket says 172.21.15.55:39594) 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.691+0000 7f63fa5cb640 1 -- 172.21.15.55:0/3056872036 learned_addr learned my addr 172.21.15.55:0/3056872036 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63fb5cd640 1 --2- 172.21.15.55:0/3056872036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565420cae9d0 0x565420ca6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63fadcc640 1 --2- 172.21.15.55:0/3056872036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565420c05140 0x565420c06f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63fa5cb640 1 -- 172.21.15.55:0/3056872036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565420c05140 msgr2=0x565420c06f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63fa5cb640 1 --2- 172.21.15.55:0/3056872036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565420c05140 0x565420c06f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63fa5cb640 1 -- 172.21.15.55:0/3056872036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565420cae9d0 msgr2=0x565420ca6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63fa5cb640 1 --2- 172.21.15.55:0/3056872036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565420cae9d0 0x565420ca6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63fa5cb640 1 -- 172.21.15.55:0/3056872036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565420b07cb0 con 0x565420caf400 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63fa5cb640 1 --2- 172.21.15.55:0/3056872036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420caf400 0x565420c06030 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f63f000a460 tx=0x7f63f000a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befa6099d3d18b9a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:55.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63f95c9640 1 -- 172.21.15.55:0/3056872036 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f000b920 con 0x565420caf400 2024-05-20T22:09:55.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63f95c9640 1 -- 172.21.15.55:0/3056872036 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f000f070 con 0x565420caf400 2024-05-20T22:09:55.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.692+0000 7f63f95c9640 1 -- 172.21.15.55:0/3056872036 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f000bd20 con 0x565420caf400 2024-05-20T22:09:55.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.693+0000 7f640200df40 1 -- 172.21.15.55:0/3056872036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420caf400 msgr2=0x565420c06030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:55.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.693+0000 7f640200df40 1 --2- 172.21.15.55:0/3056872036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420caf400 0x565420c06030 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f63f000a460 tx=0x7f63f000a920 comp rx=0 tx=0).stop 2024-05-20T22:09:55.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.693+0000 7f640200df40 1 -- 172.21.15.55:0/3056872036 shutdown_connections 2024-05-20T22:09:55.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.693+0000 7f640200df40 1 --2- 172.21.15.55:0/3056872036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565420cae9d0 0x565420ca6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:55.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.693+0000 7f640200df40 1 --2- 172.21.15.55:0/3056872036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420caf400 0x565420c06030 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:55.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.693+0000 7f640200df40 1 --2- 172.21.15.55:0/3056872036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565420c05140 0x565420c06f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:55.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.693+0000 7f640200df40 1 -- 172.21.15.55:0/3056872036 >> 172.21.15.55:0/3056872036 conn(0x565420b00ac0 msgr2=0x565420c0a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.693+0000 7f640200df40 1 -- 172.21.15.55:0/3056872036 shutdown_connections 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.693+0000 7f640200df40 1 -- 172.21.15.55:0/3056872036 wait complete. 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.694+0000 7f640200df40 1 Processor -- start 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.694+0000 7f640200df40 1 -- start start 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.695+0000 7f640200df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565420c05140 0x565420ca5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.695+0000 7f640200df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565420caf400 0x565420cacdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.695+0000 7f640200df40 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420c06fe0 0x565420cad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.695+0000 7f640200df40 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x565420c25120 con 0x565420c06fe0 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.695+0000 7f640200df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565420c016f0 con 0x565420c05140 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.695+0000 7f640200df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x565420c04e60 con 0x565420caf400 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.695+0000 7f63fb5cd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420c06fe0 0x565420cad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.695+0000 7f63fb5cd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420c06fe0 0x565420cad330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39600/0 (socket says 172.21.15.55:39600) 2024-05-20T22:09:55.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.695+0000 7f63fb5cd640 1 -- 172.21.15.55:0/3539795926 learned_addr learned my addr 172.21.15.55:0/3539795926 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63fa5cb640 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565420caf400 0x565420cacdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63fadcc640 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565420c05140 0x565420ca5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63fb5cd640 1 -- 172.21.15.55:0/3539795926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565420caf400 msgr2=0x565420cacdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63fb5cd640 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565420caf400 0x565420cacdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63fb5cd640 1 -- 172.21.15.55:0/3539795926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565420c05140 msgr2=0x565420ca5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63fb5cd640 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565420c05140 0x565420ca5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63fb5cd640 1 -- 172.21.15.55:0/3539795926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565420cb1870 con 0x565420c06fe0 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63fb5cd640 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420c06fe0 0x565420cad330 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f63f4025b40 tx=0x7f63f4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63e3fff640 1 -- 172.21.15.55:0/3539795926 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4054040 con 0x565420c06fe0 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63e3fff640 1 -- 172.21.15.55:0/3539795926 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f4050530 con 0x565420c06fe0 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f640200df40 1 -- 172.21.15.55:0/3539795926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565420cb1b50 con 0x565420c06fe0 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f63e3fff640 1 -- 172.21.15.55:0/3539795926 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f40596b0 con 0x565420c06fe0 2024-05-20T22:09:55.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.696+0000 7f640200df40 1 -- 172.21.15.55:0/3539795926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565420cac320 con 0x565420c06fe0 2024-05-20T22:09:55.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.698+0000 7f63e3fff640 1 -- 172.21.15.55:0/3539795926 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63f4057070 con 0x565420c06fe0 2024-05-20T22:09:55.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.698+0000 7f63e3fff640 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f63e806a810 0x7f63e806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:55.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.698+0000 7f63e3fff640 1 -- 172.21.15.55:0/3539795926 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f63f40cd930 con 0x565420c06fe0 2024-05-20T22:09:55.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.699+0000 7f63fadcc640 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f63e806a810 0x7f63e806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:55.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.699+0000 7f640200df40 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x565420cab200 0x565420ca7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:09:55.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.699+0000 7f640200df40 1 -- 172.21.15.55:0/3539795926 --> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] -- osd_op(unknown.0.0:1 5.13 5:c9647462:::mds0_openfiles.1:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x565420ca7f80 con 0x565420cab200 2024-05-20T22:09:55.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.699+0000 7f63fa5cb640 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x565420cab200 0x565420ca7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:09:55.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.699+0000 7f63fadcc640 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f63e806a810 0x7f63e806cd20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0075e0 tx=0x7f63ec00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:55.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.699+0000 7f63fa5cb640 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x565420cab200 0x565420ca7a40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:09:55.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f63fa5cb640 1 -- 172.21.15.55:0/3539795926 <== osd.3 v2:172.21.15.55:6808/3969115886 1 ==== osd_op_reply(1 mds0_openfiles.1 [stat out=16b] v0'0 uv9 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7f63f000c8b0 con 0x565420cab200 2024-05-20T22:09:55.702 INFO:teuthology.orchestra.run.smithi055.stdout:cephfs_metadata/mds0_openfiles.1 mtime 2024-05-20T22:09:46.000000+0000, size 0 2024-05-20T22:09:55.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f640200df40 1 -- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x565420cab200 msgr2=0x565420ca7a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:55.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f640200df40 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x565420cab200 0x565420ca7a40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:09:55.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f640200df40 1 -- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f63e806a810 msgr2=0x7f63e806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:55.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f640200df40 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f63e806a810 0x7f63e806cd20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0075e0 tx=0x7f63ec00b040 comp rx=0 tx=0).stop 2024-05-20T22:09:55.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f640200df40 1 -- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420c06fe0 msgr2=0x565420cad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:09:55.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f640200df40 1 --2- 172.21.15.55:0/3539795926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565420c06fe0 0x565420cad330 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f63f4025b40 tx=0x7f63f4003330 comp rx=0 tx=0).stop 2024-05-20T22:09:55.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f63fb5cd640 1 -- 172.21.15.55:0/3539795926 reap_dead start 2024-05-20T22:09:55.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f640200df40 1 -- 172.21.15.55:0/3539795926 shutdown_connections 2024-05-20T22:09:55.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f640200df40 1 -- 172.21.15.55:0/3539795926 >> 172.21.15.55:0/3539795926 conn(0x565420b00ac0 msgr2=0x565420c04720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:09:55.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f640200df40 1 -- 172.21.15.55:0/3539795926 shutdown_connections 2024-05-20T22:09:55.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55.700+0000 7f640200df40 1 -- 172.21.15.55:0/3539795926 wait complete. 2024-05-20T22:09:55.705 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T22:09:55.705 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T22:09:55.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T22:09:55.791 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T22:09:55.791 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T22:09:55.791 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file0", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi055') 2024-05-20T22:09:55.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:55 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-05-20T22:09:55.971 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:56.126 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:56.534 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:56.689 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:56.880 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:57.255 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:57.407 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:57.597 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:57.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:09:57.611 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file1", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi055') 2024-05-20T22:09:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:57 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-05-20T22:09:57.973 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:58.128 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:58.536 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:58.689 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:58.882 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:58.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:09:58.896 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file2", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi055') 2024-05-20T22:09:58.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:09:58 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-05-20T22:09:59.257 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:59.410 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:09:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:00.130 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:00.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:10:00.143 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file3", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi055') 2024-05-20T22:10:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:00 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-05-20T22:10:00.538 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:00.691 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:01.259 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:01.412 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:01.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:10:01.425 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file4", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi055') 2024-05-20T22:10:01.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:01 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-05-20T22:10:01.977 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:02.541 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:02.694 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:02.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:10:02.707 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file5", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi055') 2024-05-20T22:10:02.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:02 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-05-20T22:10:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:03.979 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:03.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:10:03.992 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file6", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi055') 2024-05-20T22:10:04.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:04 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-05-20T22:10:04.543 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:05.263 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:05.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:10:05.277 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file7", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi055') 2024-05-20T22:10:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:05 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-05-20T22:10:06.545 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:06.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:10:06.558 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:10:06.559 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:10:06.620 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:10:06.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.788+0000 7f5ec460d640 1 Processor -- start 2024-05-20T22:10:06.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.789+0000 7f5ec460d640 1 -- start start 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.790+0000 7f5ec460d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5ebc0d0210 0x7f5ebc06fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.790+0000 7f5ec460d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc06c550 0x7f5ebc06c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.790+0000 7f5ec460d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc06cea0 con 0x7f5ebc15d550 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.790+0000 7f5ec460d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc161a10 con 0x7f5ebc0d0210 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.790+0000 7f5ec460d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc160b40 con 0x7f5ebc06c550 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.790+0000 7f5ec1ba8640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5ebc15d550 0x7f5ebc161300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34780/0 (socket says 172.21.15.55:34780) 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.790+0000 7f5ec1ba8640 1 -- 172.21.15.55:0/2341144510 learned_addr learned my addr 172.21.15.55:0/2341144510 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.791+0000 7f5ec2baa640 1 --2- 172.21.15.55:0/2341144510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc06c550 0x7f5ebc06c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.791+0000 7f5ec23a9640 1 --2- 172.21.15.55:0/2341144510 >> v2:172.21.15.138:3300/0 conn(0x7f5ebc0d0210 0x7f5ebc06fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.791+0000 7f5ec13a7640 1 -- 172.21.15.55:0/2341144510 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277783143 0 0) 0x7f5eac001110 con 0x7f5ebc15d550 2024-05-20T22:10:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.791+0000 7f5ec13a7640 1 -- 172.21.15.55:0/2341144510 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ea8003520 con 0x7f5ebc15d550 2024-05-20T22:10:06.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.791+0000 7f5ec13a7640 1 -- 172.21.15.55:0/2341144510 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1352325629 0 0) 0x7f5ea8003520 con 0x7f5ebc15d550 2024-05-20T22:10:06.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.791+0000 7f5ec2baa640 1 -- 172.21.15.55:0/2341144510 >> v2:172.21.15.138:3300/0 conn(0x7f5ebc0d0210 msgr2=0x7f5ebc06fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:06.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.791+0000 7f5ec2baa640 1 --2- 172.21.15.55:0/2341144510 >> v2:172.21.15.138:3300/0 conn(0x7f5ebc0d0210 0x7f5ebc06fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.791+0000 7f5ec2baa640 1 -- 172.21.15.55:0/2341144510 >> v1:172.21.15.55:6789/0 conn(0x7f5ebc15d550 legacy=0x7f5ebc161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:06.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.791+0000 7f5ec2baa640 1 -- 172.21.15.55:0/2341144510 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc15dd80 con 0x7f5ebc06c550 2024-05-20T22:10:06.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.791+0000 7f5ec23a9640 1 --2- 172.21.15.55:0/2341144510 >> v2:172.21.15.138:3300/0 conn(0x7f5ebc0d0210 0x7f5ebc06fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:06.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.792+0000 7f5ec2baa640 1 --2- 172.21.15.55:0/2341144510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc06c550 0x7f5ebc06c960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009d90 tx=0x7f5eb402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=350bb1b55151f8ad server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:06.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.792+0000 7f5ec13a7640 1 -- 172.21.15.55:0/2341144510 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac001860 con 0x7f5ebc06c550 2024-05-20T22:10:06.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.792+0000 7f5ec13a7640 1 -- 172.21.15.55:0/2341144510 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb4007540 con 0x7f5ebc06c550 2024-05-20T22:10:06.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.792+0000 7f5ec13a7640 1 -- 172.21.15.55:0/2341144510 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4007910 con 0x7f5ebc06c550 2024-05-20T22:10:06.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.793+0000 7f5ec460d640 1 -- 172.21.15.55:0/2341144510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc06c550 msgr2=0x7f5ebc06c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:06.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.793+0000 7f5ec460d640 1 --2- 172.21.15.55:0/2341144510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc06c550 0x7f5ebc06c960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009d90 tx=0x7f5eb402a5f0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.793+0000 7f5ec460d640 1 -- 172.21.15.55:0/2341144510 shutdown_connections 2024-05-20T22:10:06.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.793+0000 7f5ec460d640 1 --2- 172.21.15.55:0/2341144510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc06c550 0x7f5ebc06c960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.793+0000 7f5ec460d640 1 --2- 172.21.15.55:0/2341144510 >> v2:172.21.15.138:3300/0 conn(0x7f5ebc0d0210 0x7f5ebc06fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.793+0000 7f5ec460d640 1 -- 172.21.15.55:0/2341144510 >> 172.21.15.55:0/2341144510 conn(0x7f5ebc072310 msgr2=0x7f5ebc072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:06.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.793+0000 7f5ec460d640 1 -- 172.21.15.55:0/2341144510 shutdown_connections 2024-05-20T22:10:06.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.793+0000 7f5ec460d640 1 -- 172.21.15.55:0/2341144510 wait complete. 2024-05-20T22:10:06.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.795+0000 7f5ec460d640 1 Processor -- start 2024-05-20T22:10:06.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.795+0000 7f5ec460d640 1 -- start start 2024-05-20T22:10:06.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec460d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ebc06c550 0x7f5ebc158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:06.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec460d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ebc0d0210 0x7f5ebc15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:06.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec460d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc15d550 0x7f5ebc1f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:06.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec460d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc160de0 con 0x7f5ebc0d0210 2024-05-20T22:10:06.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec460d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc15e340 con 0x7f5ebc06c550 2024-05-20T22:10:06.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec460d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc15d960 con 0x7f5ebc15d550 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec1ba8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ebc0d0210 0x7f5ebc15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec1ba8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ebc0d0210 0x7f5ebc15a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45186/0 (socket says 172.21.15.55:45186) 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec1ba8640 1 -- 172.21.15.55:0/1109444291 learned_addr learned my addr 172.21.15.55:0/1109444291 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec2baa640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc15d550 0x7f5ebc1f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec23a9640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ebc06c550 0x7f5ebc158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec1ba8640 1 -- 172.21.15.55:0/1109444291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc15d550 msgr2=0x7f5ebc1f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec1ba8640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc15d550 0x7f5ebc1f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec1ba8640 1 -- 172.21.15.55:0/1109444291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ebc06c550 msgr2=0x7f5ebc158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.796+0000 7f5ec1ba8640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ebc06c550 0x7f5ebc158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.797+0000 7f5ec1ba8640 1 -- 172.21.15.55:0/1109444291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc158d70 con 0x7f5ebc0d0210 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.797+0000 7f5ec1ba8640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ebc0d0210 0x7f5ebc15a3c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00c530 tx=0x7f5eac00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.797+0000 7f5eb2ffd640 1 -- 172.21.15.55:0/1109444291 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac018070 con 0x7f5ebc0d0210 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.797+0000 7f5eb2ffd640 1 -- 172.21.15.55:0/1109444291 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac0074e0 con 0x7f5ebc0d0210 2024-05-20T22:10:06.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.797+0000 7f5ec460d640 1 -- 172.21.15.55:0/1109444291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc159050 con 0x7f5ebc0d0210 2024-05-20T22:10:06.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.797+0000 7f5eb2ffd640 1 -- 172.21.15.55:0/1109444291 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac004a20 con 0x7f5ebc0d0210 2024-05-20T22:10:06.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.797+0000 7f5ec460d640 1 -- 172.21.15.55:0/1109444291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc0cd6e0 con 0x7f5ebc0d0210 2024-05-20T22:10:06.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.798+0000 7f5ec460d640 1 -- 172.21.15.55:0/1109444291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc06c2a0 con 0x7f5ebc0d0210 2024-05-20T22:10:06.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.799+0000 7f5eb2ffd640 1 -- 172.21.15.55:0/1109444291 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5eac01b070 con 0x7f5ebc0d0210 2024-05-20T22:10:06.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.799+0000 7f5eb2ffd640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ea006be60 0x7f5ea006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:06.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.799+0000 7f5eb2ffd640 1 -- 172.21.15.55:0/1109444291 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f5eac08a430 con 0x7f5ebc0d0210 2024-05-20T22:10:06.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.799+0000 7f5ec23a9640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ea006be60 0x7f5ea006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:06.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.800+0000 7f5ec23a9640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ea006be60 0x7f5ea006e370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009a10 tx=0x7f5eb8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:06.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.804+0000 7f5eb2ffd640 1 -- 172.21.15.55:0/1109444291 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5ebc06c2a0 con 0x7f5ebc0d0210 2024-05-20T22:10:06.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.994+0000 7f5ec460d640 1 -- 172.21.15.55:0/1109444291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ebc068990 con 0x7f5ebc0d0210 2024-05-20T22:10:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.995+0000 7f5eb2ffd640 1 -- 172.21.15.55:0/1109444291 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+17122 (secure 0 0 0) 0x7f5ebc068990 con 0x7f5ebc0d0210 2024-05-20T22:10:06.996 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:06.996 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":44,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:09:34.247518+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:09:12.734355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T22:09:13.936532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6835/4137968836":"2024-05-21T22:09:29.217850+0000","172.21.15.138:6833/4137968836":"2024-05-21T22:09:29.217850+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:10:06.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 -- 172.21.15.55:0/1109444291 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ea006be60 msgr2=0x7f5ea006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:06.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ea006be60 0x7f5ea006e370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009a10 tx=0x7f5eb8008040 comp rx=0 tx=0).stop 2024-05-20T22:10:06.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 -- 172.21.15.55:0/1109444291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ebc0d0210 msgr2=0x7f5ebc15a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:06.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ebc0d0210 0x7f5ebc15a3c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00c530 tx=0x7f5eac00c9f0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 -- 172.21.15.55:0/1109444291 shutdown_connections 2024-05-20T22:10:06.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5ea006be60 0x7f5ea006e370 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ebc15d550 0x7f5ebc1f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ebc0d0210 0x7f5ebc15a3c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 --2- 172.21.15.55:0/1109444291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ebc06c550 0x7f5ebc158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:06.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 -- 172.21.15.55:0/1109444291 >> 172.21.15.55:0/1109444291 conn(0x7f5ebc072310 msgr2=0x7f5ebc161300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:06.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 -- 172.21.15.55:0/1109444291 shutdown_connections 2024-05-20T22:10:06.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:06.997+0000 7f5ec460d640 1 -- 172.21.15.55:0/1109444291 wait complete. 2024-05-20T22:10:07.010 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:09:12.734355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:09:13.936532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T22:10:07.010 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:10:07.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.127+0000 7f06ac867640 1 Processor -- start 2024-05-20T22:10:07.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.127+0000 7f06ac867640 1 -- start start 2024-05-20T22:10:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.128+0000 7f06ac867640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06a80d3460 0x7f06a815f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.128+0000 7f06ac867640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f06a815cd00 0x7f06a80cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.128+0000 7f06ac867640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a80d0110 con 0x7f06a80cfd00 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.128+0000 7f06ac867640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f06a815d110 con 0x7f06a815cd00 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.128+0000 7f06ac867640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f06a80d3a20 con 0x7f06a80d3460 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.128+0000 7f069dd9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f06a815cd00 0x7f06a80cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.128+0000 7f069dd9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f06a815cd00 0x7f06a80cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45190/0 (socket says 172.21.15.55:45190) 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.128+0000 7f069dd9b640 1 -- 172.21.15.55:0/572301892 learned_addr learned my addr 172.21.15.55:0/572301892 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.129+0000 7f06a659c640 1 --2- 172.21.15.55:0/572301892 >> v2:172.21.15.138:3300/0 conn(0x7f06a80d3460 0x7f06a815f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.129+0000 7f069dd9b640 1 -- 172.21.15.55:0/572301892 >> v2:172.21.15.138:3300/0 conn(0x7f06a80d3460 msgr2=0x7f06a815f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.129+0000 7f069dd9b640 1 --2- 172.21.15.55:0/572301892 >> v2:172.21.15.138:3300/0 conn(0x7f06a80d3460 0x7f06a815f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.129+0000 7f069dd9b640 1 -- 172.21.15.55:0/572301892 >> v1:172.21.15.138:6789/0 conn(0x7f06a80cfd00 legacy=0x7f06a80b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.129+0000 7f069dd9b640 1 -- 172.21.15.55:0/572301892 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a80d3d00 con 0x7f06a815cd00 2024-05-20T22:10:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.129+0000 7f06a659c640 1 --2- 172.21.15.55:0/572301892 >> v2:172.21.15.138:3300/0 conn(0x7f06a80d3460 0x7f06a815f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:07.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.129+0000 7f069dd9b640 1 --2- 172.21.15.55:0/572301892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a815cd00 0x7f06a80cf7c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f06a80d5590 tx=0x7f069002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c16299d35caf21 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:07.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06a5d9b640 1 -- 172.21.15.55:0/572301892 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690037070 con 0x7f06a815cd00 2024-05-20T22:10:07.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06a5d9b640 1 -- 172.21.15.55:0/572301892 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0690032440 con 0x7f06a815cd00 2024-05-20T22:10:07.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06a5d9b640 1 -- 172.21.15.55:0/572301892 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690032810 con 0x7f06a815cd00 2024-05-20T22:10:07.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06ac867640 1 -- 172.21.15.55:0/572301892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a815cd00 msgr2=0x7f06a80cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06ac867640 1 --2- 172.21.15.55:0/572301892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a815cd00 0x7f06a80cf7c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f06a80d5590 tx=0x7f069002a270 comp rx=0 tx=0).stop 2024-05-20T22:10:07.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06ac867640 1 -- 172.21.15.55:0/572301892 shutdown_connections 2024-05-20T22:10:07.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06ac867640 1 --2- 172.21.15.55:0/572301892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a815cd00 0x7f06a80cf7c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06ac867640 1 --2- 172.21.15.55:0/572301892 >> v2:172.21.15.138:3300/0 conn(0x7f06a80d3460 0x7f06a815f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06ac867640 1 -- 172.21.15.55:0/572301892 >> 172.21.15.55:0/572301892 conn(0x7f06a80d1c70 msgr2=0x7f06a80d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:07.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06ac867640 1 -- 172.21.15.55:0/572301892 shutdown_connections 2024-05-20T22:10:07.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.130+0000 7f06ac867640 1 -- 172.21.15.55:0/572301892 wait complete. 2024-05-20T22:10:07.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.132+0000 7f06ac867640 1 Processor -- start 2024-05-20T22:10:07.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.133+0000 7f06ac867640 1 -- start start 2024-05-20T22:10:07.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06ac867640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a80cfd00 0x7f06a80cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06ac867640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a80d3460 0x7f06a81e9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06ac867640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06a815cd00 0x7f06a81e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06ac867640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f06a80d04a0 con 0x7f06a80cfd00 2024-05-20T22:10:07.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06ac867640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06a80d42c0 con 0x7f06a80d3460 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06ac867640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f06a80d0620 con 0x7f06a815cd00 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06a659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a80cfd00 0x7f06a80cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06a659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a80cfd00 0x7f06a80cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45200/0 (socket says 172.21.15.55:45200) 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06a659c640 1 -- 172.21.15.55:0/4217346886 learned_addr learned my addr 172.21.15.55:0/4217346886 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06a6d9d640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06a815cd00 0x7f06a81e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f069dd9b640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a80d3460 0x7f06a81e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06a659c640 1 -- 172.21.15.55:0/4217346886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06a815cd00 msgr2=0x7f06a81e9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06a659c640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06a815cd00 0x7f06a81e9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06a659c640 1 -- 172.21.15.55:0/4217346886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a80d3460 msgr2=0x7f06a81e9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06a659c640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a80d3460 0x7f06a81e9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06a659c640 1 -- 172.21.15.55:0/4217346886 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a81ea3d0 con 0x7f06a80cfd00 2024-05-20T22:10:07.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f06a6d9d640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06a815cd00 0x7f06a81e9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:07.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.134+0000 7f069dd9b640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a80d3460 0x7f06a81e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:07.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.135+0000 7f06a659c640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a80cfd00 0x7f06a80cac30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f069800d800 tx=0x7f069800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:07.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.135+0000 7f069f7fe640 1 -- 172.21.15.55:0/4217346886 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698004030 con 0x7f06a80cfd00 2024-05-20T22:10:07.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.135+0000 7f069f7fe640 1 -- 172.21.15.55:0/4217346886 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06980041d0 con 0x7f06a80cfd00 2024-05-20T22:10:07.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.135+0000 7f06ac867640 1 -- 172.21.15.55:0/4217346886 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a81ea6b0 con 0x7f06a80cfd00 2024-05-20T22:10:07.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.135+0000 7f069f7fe640 1 -- 172.21.15.55:0/4217346886 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06980096a0 con 0x7f06a80cfd00 2024-05-20T22:10:07.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.135+0000 7f06ac867640 1 -- 172.21.15.55:0/4217346886 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a81eac70 con 0x7f06a80cfd00 2024-05-20T22:10:07.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.136+0000 7f06ac867640 1 -- 172.21.15.55:0/4217346886 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a804f720 con 0x7f06a80cfd00 2024-05-20T22:10:07.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.137+0000 7f069f7fe640 1 -- 172.21.15.55:0/4217346886 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0698009840 con 0x7f06a80cfd00 2024-05-20T22:10:07.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.137+0000 7f069f7fe640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f068806be60 0x7f068806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.137+0000 7f069f7fe640 1 -- 172.21.15.55:0/4217346886 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f069808b420 con 0x7f06a80cfd00 2024-05-20T22:10:07.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.137+0000 7f069dd9b640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f068806be60 0x7f068806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.138+0000 7f069dd9b640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f068806be60 0x7f068806e370 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0690034000 tx=0x7f069002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:07.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.142+0000 7f069f7fe640 1 -- 172.21.15.55:0/4217346886 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0698059ac0 con 0x7f06a80cfd00 2024-05-20T22:10:07.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.332+0000 7f06ac867640 1 -- 172.21.15.55:0/4217346886 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f06a80b5e20 con 0x7f06a80cfd00 2024-05-20T22:10:07.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.332+0000 7f069f7fe640 1 -- 172.21.15.55:0/4217346886 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) ==== 93+0+10 (secure 0 0 0) 0x7f06980593c0 con 0x7f06a80cfd00 2024-05-20T22:10:07.333 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.334+0000 7f06ac867640 1 -- 172.21.15.55:0/4217346886 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f068806be60 msgr2=0x7f068806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.334+0000 7f06ac867640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f068806be60 0x7f068806e370 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0690034000 tx=0x7f069002e000 comp rx=0 tx=0).stop 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.334+0000 7f06ac867640 1 -- 172.21.15.55:0/4217346886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a80cfd00 msgr2=0x7f06a80cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.334+0000 7f06ac867640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a80cfd00 0x7f06a80cac30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f069800d800 tx=0x7f069800dcc0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.334+0000 7f06ac867640 1 -- 172.21.15.55:0/4217346886 shutdown_connections 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.334+0000 7f06ac867640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f068806be60 0x7f068806e370 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.334+0000 7f06ac867640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06a815cd00 0x7f06a81e9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.334+0000 7f06ac867640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a80d3460 0x7f06a81e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.334+0000 7f06ac867640 1 --2- 172.21.15.55:0/4217346886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f06a80cfd00 0x7f06a80cac30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.334+0000 7f06ac867640 1 -- 172.21.15.55:0/4217346886 >> 172.21.15.55:0/4217346886 conn(0x7f06a80d1c70 msgr2=0x7f06a806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.335+0000 7f06ac867640 1 -- 172.21.15.55:0/4217346886 shutdown_connections 2024-05-20T22:10:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.335+0000 7f06ac867640 1 -- 172.21.15.55:0/4217346886 wait complete. 2024-05-20T22:10:07.348 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:10:07.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.465+0000 7fa17bfff640 1 Processor -- start 2024-05-20T22:10:07.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.465+0000 7fa17bfff640 1 -- start start 2024-05-20T22:10:07.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.466+0000 7fa17bfff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa17c06cb60 0x7fa17c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.466+0000 7fa17bfff640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa17c0cea20 con 0x7fa17c070170 2024-05-20T22:10:07.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.466+0000 7fa17bfff640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa17c06c080 con 0x7fa17c0ce540 2024-05-20T22:10:07.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.466+0000 7fa17bfff640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa17c06c360 con 0x7fa17c06cb60 2024-05-20T22:10:07.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.466+0000 7fa17b7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa17c06cb60 0x7fa17c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.466+0000 7fa173fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa17c070170 0x7fa17c0cafc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34782/0 (socket says 172.21.15.55:34782) 2024-05-20T22:10:07.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.466+0000 7fa17b7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa17c06cb60 0x7fa17c15f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45202/0 (socket says 172.21.15.55:45202) 2024-05-20T22:10:07.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.466+0000 7fa173fff640 1 -- 172.21.15.55:0/1792939083 learned_addr learned my addr 172.21.15.55:0/1792939083 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:07.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.467+0000 7fa17b7fe640 1 -- 172.21.15.55:0/1792939083 >> v1:172.21.15.138:6789/0 conn(0x7fa17c0ce540 legacy=0x7fa17c06fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.467+0000 7fa17b7fe640 1 -- 172.21.15.55:0/1792939083 >> v1:172.21.15.55:6789/0 conn(0x7fa17c070170 legacy=0x7fa17c0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.467+0000 7fa17b7fe640 1 -- 172.21.15.55:0/1792939083 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c06c640 con 0x7fa17c06cb60 2024-05-20T22:10:07.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.467+0000 7fa17b7fe640 1 --2- 172.21.15.55:0/1792939083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c06cb60 0x7fa17c15f0c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa174009d90 tx=0x7fa17402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15df0a5f74f972c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:07.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.468+0000 7fa17a7fc640 1 -- 172.21.15.55:0/1792939083 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174031600 con 0x7fa17c06cb60 2024-05-20T22:10:07.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.468+0000 7fa17a7fc640 1 -- 172.21.15.55:0/1792939083 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1740317a0 con 0x7fa17c06cb60 2024-05-20T22:10:07.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.468+0000 7fa17a7fc640 1 -- 172.21.15.55:0/1792939083 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174031b70 con 0x7fa17c06cb60 2024-05-20T22:10:07.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.468+0000 7fa17bfff640 1 -- 172.21.15.55:0/1792939083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c06cb60 msgr2=0x7fa17c15f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.468+0000 7fa17bfff640 1 --2- 172.21.15.55:0/1792939083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c06cb60 0x7fa17c15f0c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa174009d90 tx=0x7fa17402a610 comp rx=0 tx=0).stop 2024-05-20T22:10:07.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.468+0000 7fa17bfff640 1 -- 172.21.15.55:0/1792939083 shutdown_connections 2024-05-20T22:10:07.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.468+0000 7fa17bfff640 1 --2- 172.21.15.55:0/1792939083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c06cb60 0x7fa17c15f0c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.468+0000 7fa17bfff640 1 -- 172.21.15.55:0/1792939083 >> 172.21.15.55:0/1792939083 conn(0x7fa17c0cd3f0 msgr2=0x7fa17c0cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:07.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.468+0000 7fa17bfff640 1 -- 172.21.15.55:0/1792939083 shutdown_connections 2024-05-20T22:10:07.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.468+0000 7fa17bfff640 1 -- 172.21.15.55:0/1792939083 wait complete. 2024-05-20T22:10:07.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.470+0000 7fa17bfff640 1 Processor -- start 2024-05-20T22:10:07.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.470+0000 7fa17bfff640 1 -- start start 2024-05-20T22:10:07.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.471+0000 7fa17bfff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa17c06cb60 0x7fa17c158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.471+0000 7fa17bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c0ce540 0x7fa17c1e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.471+0000 7fa17bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c0cb9a0 0x7fa17c0cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.471+0000 7fa17bfff640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c0cdf10 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.471+0000 7fa17bfff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c0d0540 con 0x7fa17c0ce540 2024-05-20T22:10:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.471+0000 7fa17bfff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa17c0ce090 con 0x7fa17c06cb60 2024-05-20T22:10:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c0cb9a0 0x7fa17c0cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c0cb9a0 0x7fa17c0cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45212/0 (socket says 172.21.15.55:45212) 2024-05-20T22:10:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17b7fe640 1 -- 172.21.15.55:0/4083686162 learned_addr learned my addr 172.21.15.55:0/4083686162 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa173fff640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c0ce540 0x7fa17c1e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17affd640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa17c06cb60 0x7fa17c158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17b7fe640 1 -- 172.21.15.55:0/4083686162 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa17c06cb60 msgr2=0x7fa17c158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17b7fe640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa17c06cb60 0x7fa17c158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17b7fe640 1 -- 172.21.15.55:0/4083686162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c0ce540 msgr2=0x7fa17c1e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17b7fe640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c0ce540 0x7fa17c1e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17b7fe640 1 -- 172.21.15.55:0/4083686162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c0cc2f0 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17affd640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa17c06cb60 0x7fa17c158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:07.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17b7fe640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c0cb9a0 0x7fa17c0cbdb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa17402ac00 tx=0x7fa174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:07.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa1737fe640 1 -- 172.21.15.55:0/4083686162 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174040070 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa1737fe640 1 -- 172.21.15.55:0/4083686162 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa174030b60 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa1737fe640 1 -- 172.21.15.55:0/4083686162 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17403c420 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17bfff640 1 -- 172.21.15.55:0/4083686162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17c1e4080 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.472+0000 7fa17bfff640 1 -- 172.21.15.55:0/4083686162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17c1e4640 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.473+0000 7fa17bfff640 1 -- 172.21.15.55:0/4083686162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17c070a30 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.474+0000 7fa1737fe640 1 -- 172.21.15.55:0/4083686162 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa174039040 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.475+0000 7fa1737fe640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa15c06be60 0x7fa15c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.475+0000 7fa17affd640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa15c06be60 0x7fa15c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.475+0000 7fa1737fe640 1 -- 172.21.15.55:0/4083686162 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7fa1740abf80 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.475+0000 7fa17affd640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa15c06be60 0x7fa15c06e370 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa16c000f60 tx=0x7fa16c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.479+0000 7fa1737fe640 1 -- 172.21.15.55:0/4083686162 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa17c0cc2f0 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.662+0000 7fa17bfff640 1 -- 172.21.15.55:0/4083686162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa17c09eb80 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.663+0000 7fa1737fe640 1 -- 172.21.15.55:0/4083686162 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) ==== 104+0+11 (secure 0 0 0) 0x7fa17407a7f0 con 0x7fa17c0cb9a0 2024-05-20T22:10:07.664 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:10:07.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 -- 172.21.15.55:0/4083686162 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa15c06be60 msgr2=0x7fa15c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa15c06be60 0x7fa15c06e370 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa16c000f60 tx=0x7fa16c009040 comp rx=0 tx=0).stop 2024-05-20T22:10:07.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 -- 172.21.15.55:0/4083686162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c0cb9a0 msgr2=0x7fa17c0cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c0cb9a0 0x7fa17c0cbdb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa17402ac00 tx=0x7fa174004a40 comp rx=0 tx=0).stop 2024-05-20T22:10:07.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 -- 172.21.15.55:0/4083686162 shutdown_connections 2024-05-20T22:10:07.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa15c06be60 0x7fa15c06e370 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa17c0cb9a0 0x7fa17c0cbdb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c0ce540 0x7fa17c1e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 --2- 172.21.15.55:0/4083686162 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa17c06cb60 0x7fa17c158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 -- 172.21.15.55:0/4083686162 >> 172.21.15.55:0/4083686162 conn(0x7fa17c0cd3f0 msgr2=0x7fa17c15d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:07.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 -- 172.21.15.55:0/4083686162 shutdown_connections 2024-05-20T22:10:07.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.665+0000 7fa17bfff640 1 -- 172.21.15.55:0/4083686162 wait complete. 2024-05-20T22:10:07.677 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:10:07.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.795+0000 7f7f623c7640 1 Processor -- start 2024-05-20T22:10:07.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.795+0000 7f7f623c7640 1 -- start start 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f623c7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0d3460 0x7f7f5c15f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f623c7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7f5c15cd00 0x7f7f5c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f623c7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f5c0cfd00 0x7f7f5c0b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f623c7640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f7f5c0d0110 con 0x7f7f5c15cd00 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f623c7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7f5c15d110 con 0x7f7f5c0cfd00 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f623c7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7f5c0d3a20 con 0x7f7f5c0d3460 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f5b7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7f5c15cd00 0x7f7f5c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f5b7fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7f5c15cd00 0x7f7f5c0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45228/0 (socket says 172.21.15.55:45228) 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f5b7fe640 1 -- 172.21.15.55:0/4139115185 learned_addr learned my addr 172.21.15.55:0/4139115185 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f5bfff640 1 --2- 172.21.15.55:0/4139115185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0d3460 0x7f7f5c15f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.796+0000 7f7f60964640 1 --2- 172.21.15.55:0/4139115185 >> v2:172.21.15.138:3300/0 conn(0x7f7f5c0cfd00 0x7f7f5c0b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.797+0000 7f7f5b7fe640 1 -- 172.21.15.55:0/4139115185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0d3460 msgr2=0x7f7f5c15f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:07.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.797+0000 7f7f5b7fe640 1 --2- 172.21.15.55:0/4139115185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0d3460 0x7f7f5c15f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.797+0000 7f7f5b7fe640 1 -- 172.21.15.55:0/4139115185 >> v2:172.21.15.138:3300/0 conn(0x7f7f5c0cfd00 msgr2=0x7f7f5c0b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:07.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.797+0000 7f7f5b7fe640 1 --2- 172.21.15.55:0/4139115185 >> v2:172.21.15.138:3300/0 conn(0x7f7f5c0cfd00 0x7f7f5c0b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.797+0000 7f7f5b7fe640 1 -- 172.21.15.55:0/4139115185 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c0d3d00 con 0x7f7f5c15cd00 2024-05-20T22:10:07.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.797+0000 7f7f60964640 1 --2- 172.21.15.55:0/4139115185 >> v2:172.21.15.138:3300/0 conn(0x7f7f5c0cfd00 0x7f7f5c0b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:07.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.797+0000 7f7f5bfff640 1 --2- 172.21.15.55:0/4139115185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0d3460 0x7f7f5c15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:07.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.797+0000 7f7f5b7fe640 1 --2- 172.21.15.55:0/4139115185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c15cd00 0x7f7f5c0cf7c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0cf0f0 tx=0x7f7f5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f1d7bcdb518ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:07.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f5affd640 1 -- 172.21.15.55:0/4139115185 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50037070 con 0x7f7f5c15cd00 2024-05-20T22:10:07.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f5affd640 1 -- 172.21.15.55:0/4139115185 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f50032440 con 0x7f7f5c15cd00 2024-05-20T22:10:07.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f5affd640 1 -- 172.21.15.55:0/4139115185 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50032810 con 0x7f7f5c15cd00 2024-05-20T22:10:07.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f623c7640 1 -- 172.21.15.55:0/4139115185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c15cd00 msgr2=0x7f7f5c0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f623c7640 1 --2- 172.21.15.55:0/4139115185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c15cd00 0x7f7f5c0cf7c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0cf0f0 tx=0x7f7f5002a1e0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f623c7640 1 -- 172.21.15.55:0/4139115185 shutdown_connections 2024-05-20T22:10:07.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f623c7640 1 --2- 172.21.15.55:0/4139115185 >> v2:172.21.15.138:3300/0 conn(0x7f7f5c0cfd00 0x7f7f5c0b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f623c7640 1 --2- 172.21.15.55:0/4139115185 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c15cd00 0x7f7f5c0cf7c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f623c7640 1 --2- 172.21.15.55:0/4139115185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0d3460 0x7f7f5c15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f623c7640 1 -- 172.21.15.55:0/4139115185 >> 172.21.15.55:0/4139115185 conn(0x7f7f5c0d1c70 msgr2=0x7f7f5c0d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:07.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.798+0000 7f7f623c7640 1 -- 172.21.15.55:0/4139115185 shutdown_connections 2024-05-20T22:10:07.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.799+0000 7f7f623c7640 1 -- 172.21.15.55:0/4139115185 wait complete. 2024-05-20T22:10:07.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.800+0000 7f7f623c7640 1 Processor -- start 2024-05-20T22:10:07.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.800+0000 7f7f623c7640 1 -- start start 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f623c7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0cfd00 0x7f7f5c0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f623c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0d3460 0x7f7f5c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f623c7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f5c15cd00 0x7f7f5c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f623c7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c0d04a0 con 0x7f7f5c0d3460 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f623c7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c0d42c0 con 0x7f7f5c15cd00 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f623c7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7f5c0d0620 con 0x7f7f5c0cfd00 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f5b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0d3460 0x7f7f5c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f5b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0d3460 0x7f7f5c0cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45244/0 (socket says 172.21.15.55:45244) 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f5b7fe640 1 -- 172.21.15.55:0/1926326541 learned_addr learned my addr 172.21.15.55:0/1926326541 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f5bfff640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0cfd00 0x7f7f5c0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f60964640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f5c15cd00 0x7f7f5c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f5b7fe640 1 -- 172.21.15.55:0/1926326541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0cfd00 msgr2=0x7f7f5c0cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f5b7fe640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0cfd00 0x7f7f5c0cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f5b7fe640 1 -- 172.21.15.55:0/1926326541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f5c15cd00 msgr2=0x7f7f5c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f5b7fe640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f5c15cd00 0x7f7f5c0ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f5b7fe640 1 -- 172.21.15.55:0/1926326541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c0cbea0 con 0x7f7f5c0d3460 2024-05-20T22:10:07.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.802+0000 7f7f5b7fe640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0d3460 0x7f7f5c0cde10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7f5002a220 tx=0x7f7f500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:07.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.803+0000 7f7f58ff9640 1 -- 172.21.15.55:0/1926326541 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50041020 con 0x7f7f5c0d3460 2024-05-20T22:10:07.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.803+0000 7f7f58ff9640 1 -- 172.21.15.55:0/1926326541 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f500306c0 con 0x7f7f5c0d3460 2024-05-20T22:10:07.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.803+0000 7f7f623c7640 1 -- 172.21.15.55:0/1926326541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5c156ba0 con 0x7f7f5c0d3460 2024-05-20T22:10:07.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.803+0000 7f7f58ff9640 1 -- 172.21.15.55:0/1926326541 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50037040 con 0x7f7f5c0d3460 2024-05-20T22:10:07.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.803+0000 7f7f623c7640 1 -- 172.21.15.55:0/1926326541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5c157160 con 0x7f7f5c0d3460 2024-05-20T22:10:07.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.804+0000 7f7f623c7640 1 -- 172.21.15.55:0/1926326541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f5c04f720 con 0x7f7f5c0d3460 2024-05-20T22:10:07.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.804+0000 7f7f58ff9640 1 -- 172.21.15.55:0/1926326541 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f50044020 con 0x7f7f5c0d3460 2024-05-20T22:10:07.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.805+0000 7f7f58ff9640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7f4806be60 0x7f7f4806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:07.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.805+0000 7f7f58ff9640 1 -- 172.21.15.55:0/1926326541 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f7f500ab820 con 0x7f7f5c0d3460 2024-05-20T22:10:07.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.805+0000 7f7f5bfff640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7f4806be60 0x7f7f4806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:07.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.805+0000 7f7f5bfff640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7f4806be60 0x7f7f4806e370 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c006fd0 tx=0x7f7f4c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:07.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.810+0000 7f7f58ff9640 1 -- 172.21.15.55:0/1926326541 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7f5004b030 con 0x7f7f5c0d3460 2024-05-20T22:10:07.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.994+0000 7f7f623c7640 1 -- 172.21.15.55:0/1926326541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7f5c0cbc10 con 0x7f7f5c0d3460 2024-05-20T22:10:07.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.994+0000 7f7f58ff9640 1 -- 172.21.15.55:0/1926326541 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) ==== 100+0+11 (secure 0 0 0) 0x7f7f500798e0 con 0x7f7f5c0d3460 2024-05-20T22:10:07.995 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:10:07.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 -- 172.21.15.55:0/1926326541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7f4806be60 msgr2=0x7f7f4806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7f4806be60 0x7f7f4806e370 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c006fd0 tx=0x7f7f4c008040 comp rx=0 tx=0).stop 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 -- 172.21.15.55:0/1926326541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0d3460 msgr2=0x7f7f5c0cde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0d3460 0x7f7f5c0cde10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7f5002a220 tx=0x7f7f500046c0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 -- 172.21.15.55:0/1926326541 shutdown_connections 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7f4806be60 0x7f7f4806e370 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f5c15cd00 0x7f7f5c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0d3460 0x7f7f5c0cde10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 --2- 172.21.15.55:0/1926326541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f5c0cfd00 0x7f7f5c0cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 -- 172.21.15.55:0/1926326541 >> 172.21.15.55:0/1926326541 conn(0x7f7f5c0d1c70 msgr2=0x7f7f5c15f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 -- 172.21.15.55:0/1926326541 shutdown_connections 2024-05-20T22:10:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:07.996+0000 7f7f623c7640 1 -- 172.21.15.55:0/1926326541 wait complete. 2024-05-20T22:10:08.009 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:10:08.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.125+0000 7f771a1b7640 1 Processor -- start 2024-05-20T22:10:08.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.125+0000 7f771a1b7640 1 -- start start 2024-05-20T22:10:08.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.126+0000 7f771a1b7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77140cdb50 0x7f77140d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.126+0000 7f771a1b7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 0x7f771415b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.126+0000 7f771a1b7640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77140d21c0 con 0x7f77140d1db0 2024-05-20T22:10:08.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.126+0000 7f771a1b7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f77140ce2b0 con 0x7f77140cdb50 2024-05-20T22:10:08.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.126+0000 7f771a1b7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f77140ce550 con 0x7f771409abc0 2024-05-20T22:10:08.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.126+0000 7f7712ffd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f77140d1db0 0x7f771409a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34786/0 (socket says 172.21.15.55:34786) 2024-05-20T22:10:08.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.126+0000 7f7712ffd640 1 -- 172.21.15.55:0/2332900704 learned_addr learned my addr 172.21.15.55:0/2332900704 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:08.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.127+0000 7f77137fe640 1 --2- 172.21.15.55:0/2332900704 >> v2:172.21.15.138:3300/0 conn(0x7f77140cdb50 0x7f77140d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.127+0000 7f7713fff640 1 --2- 172.21.15.55:0/2332900704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 0x7f771415b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.127+0000 7f77127fc640 1 -- 172.21.15.55:0/2332900704 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294758438 0 0) 0x7f7708001110 con 0x7f77140d1db0 2024-05-20T22:10:08.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.127+0000 7f77127fc640 1 -- 172.21.15.55:0/2332900704 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76fc003520 con 0x7f77140d1db0 2024-05-20T22:10:08.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.127+0000 7f7713fff640 1 -- 172.21.15.55:0/2332900704 >> v2:172.21.15.138:3300/0 conn(0x7f77140cdb50 msgr2=0x7f77140d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:08.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.127+0000 7f7713fff640 1 --2- 172.21.15.55:0/2332900704 >> v2:172.21.15.138:3300/0 conn(0x7f77140cdb50 0x7f77140d1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.127+0000 7f7713fff640 1 -- 172.21.15.55:0/2332900704 >> v1:172.21.15.55:6789/0 conn(0x7f77140d1db0 legacy=0x7f771409a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:08.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.127+0000 7f77127fc640 1 -- 172.21.15.55:0/2332900704 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3513324395 0 0) 0x7f76fc003520 con 0x7f77140d1db0 2024-05-20T22:10:08.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.127+0000 7f7713fff640 1 -- 172.21.15.55:0/2332900704 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77140ce830 con 0x7f771409abc0 2024-05-20T22:10:08.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.127+0000 7f77137fe640 1 --2- 172.21.15.55:0/2332900704 >> v2:172.21.15.138:3300/0 conn(0x7f77140cdb50 0x7f77140d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:08.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.128+0000 7f7713fff640 1 --2- 172.21.15.55:0/2332900704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 0x7f771415b550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f770c009d90 tx=0x7f770c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=307cd838e6bc6083 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:08.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.128+0000 7f77127fc640 1 -- 172.21.15.55:0/2332900704 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708001840 con 0x7f771409abc0 2024-05-20T22:10:08.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.128+0000 7f77127fc640 1 -- 172.21.15.55:0/2332900704 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f770c007540 con 0x7f771409abc0 2024-05-20T22:10:08.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.128+0000 7f77127fc640 1 -- 172.21.15.55:0/2332900704 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c0078f0 con 0x7f771409abc0 2024-05-20T22:10:08.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.129+0000 7f771a1b7640 1 -- 172.21.15.55:0/2332900704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 msgr2=0x7f771415b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:08.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.129+0000 7f771a1b7640 1 --2- 172.21.15.55:0/2332900704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 0x7f771415b550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f770c009d90 tx=0x7f770c02a5f0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.129+0000 7f771a1b7640 1 -- 172.21.15.55:0/2332900704 shutdown_connections 2024-05-20T22:10:08.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.129+0000 7f771a1b7640 1 --2- 172.21.15.55:0/2332900704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 0x7f771415b550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.129+0000 7f771a1b7640 1 --2- 172.21.15.55:0/2332900704 >> v2:172.21.15.138:3300/0 conn(0x7f77140cdb50 0x7f77140d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.129+0000 7f771a1b7640 1 -- 172.21.15.55:0/2332900704 >> 172.21.15.55:0/2332900704 conn(0x7f77140b65f0 msgr2=0x7f77140b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:08.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.129+0000 7f771a1b7640 1 -- 172.21.15.55:0/2332900704 shutdown_connections 2024-05-20T22:10:08.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.129+0000 7f771a1b7640 1 -- 172.21.15.55:0/2332900704 wait complete. 2024-05-20T22:10:08.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.131+0000 7f771a1b7640 1 Processor -- start 2024-05-20T22:10:08.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.131+0000 7f771a1b7640 1 -- start start 2024-05-20T22:10:08.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f771a1b7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 0x7f771415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f771a1b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77140cdb50 0x7f7714156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f771a1b7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77140d1db0 0x7f771415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f771a1b7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f771415ba90 con 0x7f77140d1db0 2024-05-20T22:10:08.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f771a1b7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77140cedf0 con 0x7f77140cdb50 2024-05-20T22:10:08.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f771a1b7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f771415bc10 con 0x7f771409abc0 2024-05-20T22:10:08.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f7713fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77140d1db0 0x7f771415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f7713fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77140d1db0 0x7f771415de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45256/0 (socket says 172.21.15.55:45256) 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f7713fff640 1 -- 172.21.15.55:0/2212726974 learned_addr learned my addr 172.21.15.55:0/2212726974 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f77137fe640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 0x7f771415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f7712ffd640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77140cdb50 0x7f7714156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f7713fff640 1 -- 172.21.15.55:0/2212726974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 msgr2=0x7f771415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f7713fff640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 0x7f771415ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f7713fff640 1 -- 172.21.15.55:0/2212726974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77140cdb50 msgr2=0x7f7714156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f7713fff640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77140cdb50 0x7f7714156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.132+0000 7f7713fff640 1 -- 172.21.15.55:0/2212726974 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7714159640 con 0x7f77140d1db0 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.133+0000 7f7713fff640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77140d1db0 0x7f771415de60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f770c009eb0 tx=0x7f770c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.133+0000 7f76fbfff640 1 -- 172.21.15.55:0/2212726974 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c03f070 con 0x7f77140d1db0 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.133+0000 7f76fbfff640 1 -- 172.21.15.55:0/2212726974 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f770c031c40 con 0x7f77140d1db0 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.133+0000 7f76fbfff640 1 -- 172.21.15.55:0/2212726974 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c009450 con 0x7f77140d1db0 2024-05-20T22:10:08.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.133+0000 7f771a1b7640 1 -- 172.21.15.55:0/2212726974 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77141572a0 con 0x7f77140d1db0 2024-05-20T22:10:08.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.133+0000 7f771a1b7640 1 -- 172.21.15.55:0/2212726974 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77140ae7f0 con 0x7f77140d1db0 2024-05-20T22:10:08.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.134+0000 7f771a1b7640 1 -- 172.21.15.55:0/2212726974 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7714159b60 con 0x7f77140d1db0 2024-05-20T22:10:08.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.135+0000 7f76fbfff640 1 -- 172.21.15.55:0/2212726974 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f770c043410 con 0x7f77140d1db0 2024-05-20T22:10:08.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.135+0000 7f76fbfff640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f770006be60 0x7f770006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.135+0000 7f76fbfff640 1 -- 172.21.15.55:0/2212726974 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f770c0ac3c0 con 0x7f77140d1db0 2024-05-20T22:10:08.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.135+0000 7f77137fe640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f770006be60 0x7f770006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.136+0000 7f77137fe640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f770006be60 0x7f770006e370 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7704009500 tx=0x7f7704008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:08.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.140+0000 7f76fbfff640 1 -- 172.21.15.55:0/2212726974 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7714159b60 con 0x7f77140d1db0 2024-05-20T22:10:08.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.337+0000 7f771a1b7640 1 -- 172.21.15.55:0/2212726974 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f771404f720 con 0x7f77140d1db0 2024-05-20T22:10:08.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.337+0000 7f76fbfff640 1 -- 172.21.15.55:0/2212726974 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v44) ==== 80+0+124 (secure 0 0 0) 0x7f770c07ab70 con 0x7f77140d1db0 2024-05-20T22:10:08.338 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.138:6835/4137968836 2024-05-21T22:09:29.217850+0000 2024-05-20T22:10:08.338 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.138:6833/4137968836 2024-05-21T22:09:29.217850+0000 2024-05-20T22:10:08.339 INFO:teuthology.orchestra.run.smithi055.stderr:listed 2 entries 2024-05-20T22:10:08.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 -- 172.21.15.55:0/2212726974 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f770006be60 msgr2=0x7f770006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:08.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f770006be60 0x7f770006e370 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7704009500 tx=0x7f7704008040 comp rx=0 tx=0).stop 2024-05-20T22:10:08.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 -- 172.21.15.55:0/2212726974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77140d1db0 msgr2=0x7f771415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:08.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77140d1db0 0x7f771415de60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f770c009eb0 tx=0x7f770c004a40 comp rx=0 tx=0).stop 2024-05-20T22:10:08.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 -- 172.21.15.55:0/2212726974 shutdown_connections 2024-05-20T22:10:08.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f770006be60 0x7f770006e370 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77140d1db0 0x7f771415de60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77140cdb50 0x7f7714156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 --2- 172.21.15.55:0/2212726974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f771409abc0 0x7f771415ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 -- 172.21.15.55:0/2212726974 >> 172.21.15.55:0/2212726974 conn(0x7f77140b65f0 msgr2=0x7f771415f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:08.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 -- 172.21.15.55:0/2212726974 shutdown_connections 2024-05-20T22:10:08.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.339+0000 7f771a1b7640 1 -- 172.21.15.55:0/2212726974 wait complete. 2024-05-20T22:10:08.352 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi055.front.sepia.ceph.com... 2024-05-20T22:10:08.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:08.352 DEBUG:teuthology.orchestra.run.smithi055:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-20T22:10:08.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:ceph-fuse[54219]: fuse finished with error 0 and tester_r 0 2024-05-20T22:10:08.379 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:10:08.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:08.379 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T22:10:08.441 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T22:10:08.441 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:10:08.441 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:10:08.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=450 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T22:10:08.461 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:10:08.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:08.461 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-20T22:10:08.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:10:08.518 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-20T22:10:08.518 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T22:10:08.518 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:10:08.519 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:10:08.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3618 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T22:10:08.535 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:10:08.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:08.536 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-20T22:10:08.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:10:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-20T22:10:08.592 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T22:10:08.592 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:10:08.592 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:10:08.608 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=450 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T22:10:08.608 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:10:08.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:08.609 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-20T22:10:08.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:10:08.664 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-20T22:10:08.665 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:08.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.778+0000 7f1f0cf23640 1 Processor -- start 2024-05-20T22:10:08.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.778+0000 7f1f0cf23640 1 -- start start 2024-05-20T22:10:08.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f0cf23640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f080d1ac0 0x7f1f080d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f0cf23640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f08071e70 0x7f1f08072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f0cf23640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f080727c0 con 0x7f1f080b1d60 2024-05-20T22:10:08.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f0cf23640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1f080700c0 con 0x7f1f080d1ac0 2024-05-20T22:10:08.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f0cf23640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1f080d1630 con 0x7f1f08071e70 2024-05-20T22:10:08.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f06d9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f08071e70 0x7f1f08072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f05d9b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f080d1ac0 0x7f1f080d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f0659c640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1f080b1d60 0x7f1f0806f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:46448/0 (socket says 172.21.15.55:46448) 2024-05-20T22:10:08.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f0659c640 1 -- 172.21.15.55:0/4160750173 learned_addr learned my addr 172.21.15.55:0/4160750173 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:08.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f06d9d640 1 -- 172.21.15.55:0/4160750173 >> v2:172.21.15.138:3300/0 conn(0x7f1f080d1ac0 msgr2=0x7f1f080d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:08.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f06d9d640 1 --2- 172.21.15.55:0/4160750173 >> v2:172.21.15.138:3300/0 conn(0x7f1f080d1ac0 0x7f1f080d1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f06d9d640 1 -- 172.21.15.55:0/4160750173 >> v1:172.21.15.138:6789/0 conn(0x7f1f080b1d60 legacy=0x7f1f0806f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:08.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f06d9d640 1 -- 172.21.15.55:0/4160750173 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0806fb00 con 0x7f1f08071e70 2024-05-20T22:10:08.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.780+0000 7f1f05d9b640 1 --2- 172.21.15.55:0/4160750173 >> v2:172.21.15.138:3300/0 conn(0x7f1f080d1ac0 0x7f1f080d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:08.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.781+0000 7f1f06d9d640 1 --2- 172.21.15.55:0/4160750173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f08071e70 0x7f1f08072280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f000097a0 tx=0x7f1f0002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9312314a527dc170 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:08.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.781+0000 7f1f0559a640 1 -- 172.21.15.55:0/4160750173 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00037070 con 0x7f1f08071e70 2024-05-20T22:10:08.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.781+0000 7f1f0559a640 1 -- 172.21.15.55:0/4160750173 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0002ae50 con 0x7f1f08071e70 2024-05-20T22:10:08.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.781+0000 7f1f0559a640 1 -- 172.21.15.55:0/4160750173 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00032660 con 0x7f1f08071e70 2024-05-20T22:10:08.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.782+0000 7f1f0cf23640 1 -- 172.21.15.55:0/4160750173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f08071e70 msgr2=0x7f1f08072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:08.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.782+0000 7f1f0cf23640 1 --2- 172.21.15.55:0/4160750173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f08071e70 0x7f1f08072280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f000097a0 tx=0x7f1f0002a100 comp rx=0 tx=0).stop 2024-05-20T22:10:08.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.782+0000 7f1f0cf23640 1 -- 172.21.15.55:0/4160750173 shutdown_connections 2024-05-20T22:10:08.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.782+0000 7f1f0cf23640 1 --2- 172.21.15.55:0/4160750173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f08071e70 0x7f1f08072280 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.782+0000 7f1f0cf23640 1 --2- 172.21.15.55:0/4160750173 >> v2:172.21.15.138:3300/0 conn(0x7f1f080d1ac0 0x7f1f080d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.782+0000 7f1f0cf23640 1 -- 172.21.15.55:0/4160750173 >> 172.21.15.55:0/4160750173 conn(0x7f1f080d2960 msgr2=0x7f1f080d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:08.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.782+0000 7f1f0cf23640 1 -- 172.21.15.55:0/4160750173 shutdown_connections 2024-05-20T22:10:08.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.782+0000 7f1f0cf23640 1 -- 172.21.15.55:0/4160750173 wait complete. 2024-05-20T22:10:08.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.784+0000 7f1f0cf23640 1 Processor -- start 2024-05-20T22:10:08.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.784+0000 7f1f0cf23640 1 -- start start 2024-05-20T22:10:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.785+0000 7f1f0cf23640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08071e70 0x7f1f081583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.785+0000 7f1f0cf23640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f080b1d60 0x7f1f081e7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f0cf23640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f080d1ac0 0x7f1f081e8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f0cf23640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0806f800 con 0x7f1f080d1ac0 2024-05-20T22:10:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f0cf23640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f080d12c0 con 0x7f1f08071e70 2024-05-20T22:10:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f0cf23640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1f080cd6a0 con 0x7f1f080b1d60 2024-05-20T22:10:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f06d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f080d1ac0 0x7f1f081e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f06d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f080d1ac0 0x7f1f081e8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44650/0 (socket says 172.21.15.55:44650) 2024-05-20T22:10:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f06d9d640 1 -- 172.21.15.55:0/1666209528 learned_addr learned my addr 172.21.15.55:0/1666209528 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f0659c640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08071e70 0x7f1f081583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f05d9b640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f080b1d60 0x7f1f081e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f06d9d640 1 -- 172.21.15.55:0/1666209528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f080b1d60 msgr2=0x7f1f081e7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f06d9d640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f080b1d60 0x7f1f081e7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f06d9d640 1 -- 172.21.15.55:0/1666209528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08071e70 msgr2=0x7f1f081583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f06d9d640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08071e70 0x7f1f081583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f06d9d640 1 -- 172.21.15.55:0/1666209528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f080cc310 con 0x7f1f080d1ac0 2024-05-20T22:10:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f06d9d640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f080d1ac0 0x7f1f081e8700 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1f00035040 tx=0x7f1f000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1efeffd640 1 -- 172.21.15.55:0/1666209528 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00041070 con 0x7f1f080d1ac0 2024-05-20T22:10:08.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1efeffd640 1 -- 172.21.15.55:0/1666209528 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0003c460 con 0x7f1f080d1ac0 2024-05-20T22:10:08.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f0cf23640 1 -- 172.21.15.55:0/1666209528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f080c9df0 con 0x7f1f080d1ac0 2024-05-20T22:10:08.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1efeffd640 1 -- 172.21.15.55:0/1666209528 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00037040 con 0x7f1f080d1ac0 2024-05-20T22:10:08.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.786+0000 7f1f0cf23640 1 -- 172.21.15.55:0/1666209528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f080ca3b0 con 0x7f1f080d1ac0 2024-05-20T22:10:08.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.787+0000 7f1f0cf23640 1 -- 172.21.15.55:0/1666209528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f080b3920 con 0x7f1f080d1ac0 2024-05-20T22:10:08.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.788+0000 7f1efeffd640 1 -- 172.21.15.55:0/1666209528 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f00044070 con 0x7f1f080d1ac0 2024-05-20T22:10:08.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.789+0000 7f1efeffd640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1ee806be30 0x7f1ee806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:08.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.789+0000 7f1efeffd640 1 -- 172.21.15.55:0/1666209528 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f1f000ab9d0 con 0x7f1f080d1ac0 2024-05-20T22:10:08.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.789+0000 7f1f0659c640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1ee806be30 0x7f1ee806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:08.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.789+0000 7f1f0659c640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1ee806be30 0x7f1ee806e340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80012e0 tx=0x7f1ef8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:08.793+0000 7f1efeffd640 1 -- 172.21.15.55:0/1666209528 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1f080b3920 con 0x7f1f080d1ac0 2024-05-20T22:10:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.014+0000 7f1f0cf23640 1 -- 172.21.15.55:0/1666209528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f080d0d40 con 0x7f1f080d1ac0 2024-05-20T22:10:09.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.015+0000 7f1efeffd640 1 -- 172.21.15.55:0/1666209528 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5191 (secure 0 0 0) 0x7f1f080cc310 con 0x7f1f080d1ac0 2024-05-20T22:10:09.016 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:09.016 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3873372908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3873372908},{"type":"v1","addr":"172.21.15.55:6836","nonce":3873372908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3417305427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":3417305427},{"type":"v1","addr":"172.21.15.55:6837","nonce":3417305427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":35081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4211704780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4211704780},{"type":"v1","addr":"172.21.15.138:6833","nonce":4211704780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:09:14.957632+0000","modified":"2024-05-20T22:09:33.242208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":35093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35093":{"gid":35093,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6835/270300569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":270300569},{"type":"v1","addr":"172.21.15.138:6835","nonce":270300569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":35093,"qdb_cluster":[35093]},"id":2}]} 2024-05-20T22:10:09.016 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2024-05-20T22:10:09.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 -- 172.21.15.55:0/1666209528 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1ee806be30 msgr2=0x7f1ee806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1ee806be30 0x7f1ee806e340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80012e0 tx=0x7f1ef8009040 comp rx=0 tx=0).stop 2024-05-20T22:10:09.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 -- 172.21.15.55:0/1666209528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f080d1ac0 msgr2=0x7f1f081e8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f080d1ac0 0x7f1f081e8700 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1f00035040 tx=0x7f1f000046c0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 -- 172.21.15.55:0/1666209528 shutdown_connections 2024-05-20T22:10:09.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f1ee806be30 0x7f1ee806e340 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f080d1ac0 0x7f1f081e8700 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f080b1d60 0x7f1f081e7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 --2- 172.21.15.55:0/1666209528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08071e70 0x7f1f081583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 -- 172.21.15.55:0/1666209528 >> 172.21.15.55:0/1666209528 conn(0x7f1f080d2960 msgr2=0x7f1f080b5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 -- 172.21.15.55:0/1666209528 shutdown_connections 2024-05-20T22:10:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.017+0000 7f1f0cf23640 1 -- 172.21.15.55:0/1666209528 wait complete. 2024-05-20T22:10:09.030 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:10:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.143+0000 7f45aea4b640 1 Processor -- start 2024-05-20T22:10:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.143+0000 7f45aea4b640 1 -- start start 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45aea4b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a815f440 0x7f45a80d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45aea4b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45a806d470 0x7f45a80d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45aea4b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45a806b6c0 con 0x7f45a80b3210 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45aea4b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f45a806bfd0 con 0x7f45a806d470 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45aea4b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f45a80d0bc0 con 0x7f45a815f440 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45acfe8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45a806d470 0x7f45a80d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45a7fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a815f440 0x7f45a80d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45a7fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a815f440 0x7f45a80d0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:59086/0 (socket says 172.21.15.55:59086) 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45acfe8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45a806d470 0x7f45a80d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:49782/0 (socket says 172.21.15.55:49782) 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45a77fe640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f45a80b3210 0x7f45a80b09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:46452/0 (socket says 172.21.15.55:46452) 2024-05-20T22:10:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45a7fff640 1 -- 172.21.15.55:0/579048332 learned_addr learned my addr 172.21.15.55:0/579048332 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:09.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45a7fff640 1 -- 172.21.15.55:0/579048332 >> v2:172.21.15.138:3300/0 conn(0x7f45a806d470 msgr2=0x7f45a80d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:09.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45a7fff640 1 --2- 172.21.15.55:0/579048332 >> v2:172.21.15.138:3300/0 conn(0x7f45a806d470 0x7f45a80d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45a7fff640 1 -- 172.21.15.55:0/579048332 >> v1:172.21.15.138:6789/0 conn(0x7f45a80b3210 legacy=0x7f45a80b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45a7fff640 1 -- 172.21.15.55:0/579048332 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a806b980 con 0x7f45a815f440 2024-05-20T22:10:09.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.145+0000 7f45acfe8640 1 --2- 172.21.15.55:0/579048332 >> v2:172.21.15.138:3300/0 conn(0x7f45a806d470 0x7f45a80d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:09.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.146+0000 7f45a7fff640 1 --2- 172.21.15.55:0/579048332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a815f440 0x7f45a80d0140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45a806c150 tx=0x7f459802a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd3d06e98e31a40 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:09.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.146+0000 7f45a6ffd640 1 -- 172.21.15.55:0/579048332 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598037070 con 0x7f45a815f440 2024-05-20T22:10:09.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.146+0000 7f45a6ffd640 1 -- 172.21.15.55:0/579048332 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4598032440 con 0x7f45a815f440 2024-05-20T22:10:09.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.146+0000 7f45a6ffd640 1 -- 172.21.15.55:0/579048332 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598032810 con 0x7f45a815f440 2024-05-20T22:10:09.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.147+0000 7f45aea4b640 1 -- 172.21.15.55:0/579048332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a815f440 msgr2=0x7f45a80d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.147+0000 7f45aea4b640 1 --2- 172.21.15.55:0/579048332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a815f440 0x7f45a80d0140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45a806c150 tx=0x7f459802a1e0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.147+0000 7f45aea4b640 1 -- 172.21.15.55:0/579048332 shutdown_connections 2024-05-20T22:10:09.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.147+0000 7f45aea4b640 1 --2- 172.21.15.55:0/579048332 >> v2:172.21.15.138:3300/0 conn(0x7f45a806d470 0x7f45a80d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.147+0000 7f45aea4b640 1 --2- 172.21.15.55:0/579048332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a815f440 0x7f45a80d0140 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.147+0000 7f45aea4b640 1 -- 172.21.15.55:0/579048332 >> 172.21.15.55:0/579048332 conn(0x7f45a809a400 msgr2=0x7f45a80b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:09.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.147+0000 7f45aea4b640 1 -- 172.21.15.55:0/579048332 shutdown_connections 2024-05-20T22:10:09.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.147+0000 7f45aea4b640 1 -- 172.21.15.55:0/579048332 wait complete. 2024-05-20T22:10:09.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.149+0000 7f45aea4b640 1 Processor -- start 2024-05-20T22:10:09.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.149+0000 7f45aea4b640 1 -- start start 2024-05-20T22:10:09.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.150+0000 7f45aea4b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806d470 0x7f45a8154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.150+0000 7f45aea4b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a80b3210 0x7f45a80cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.150+0000 7f45aea4b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815f440 0x7f45a80cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.150+0000 7f45aea4b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f45a80b2460 con 0x7f45a815f440 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.150+0000 7f45aea4b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45a80b22e0 con 0x7f45a80b3210 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.150+0000 7f45aea4b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f45a80b6510 con 0x7f45a806d470 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45acfe8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815f440 0x7f45a80cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45acfe8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815f440 0x7f45a80cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44662/0 (socket says 172.21.15.55:44662) 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45acfe8640 1 -- 172.21.15.55:0/3627808469 learned_addr learned my addr 172.21.15.55:0/3627808469 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45a77fe640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a80b3210 0x7f45a80cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45a7fff640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806d470 0x7f45a8154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45acfe8640 1 -- 172.21.15.55:0/3627808469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806d470 msgr2=0x7f45a8154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45acfe8640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806d470 0x7f45a8154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45acfe8640 1 -- 172.21.15.55:0/3627808469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a80b3210 msgr2=0x7f45a80cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45acfe8640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a80b3210 0x7f45a80cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45acfe8640 1 -- 172.21.15.55:0/3627808469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a80cb980 con 0x7f45a815f440 2024-05-20T22:10:09.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45acfe8640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815f440 0x7f45a80cdc20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f45a000efc0 tx=0x7f45a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:09.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45a4ff9640 1 -- 172.21.15.55:0/3627808469 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0018030 con 0x7f45a815f440 2024-05-20T22:10:09.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45a4ff9640 1 -- 172.21.15.55:0/3627808469 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a000cd30 con 0x7f45a815f440 2024-05-20T22:10:09.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.151+0000 7f45aea4b640 1 -- 172.21.15.55:0/3627808469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a80cbc60 con 0x7f45a815f440 2024-05-20T22:10:09.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.152+0000 7f45a4ff9640 1 -- 172.21.15.55:0/3627808469 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0003bb0 con 0x7f45a815f440 2024-05-20T22:10:09.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.152+0000 7f45aea4b640 1 -- 172.21.15.55:0/3627808469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a80cc220 con 0x7f45a815f440 2024-05-20T22:10:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.152+0000 7f45aea4b640 1 -- 172.21.15.55:0/3627808469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a804f720 con 0x7f45a815f440 2024-05-20T22:10:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.153+0000 7f45a4ff9640 1 -- 172.21.15.55:0/3627808469 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45a001b070 con 0x7f45a815f440 2024-05-20T22:10:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.154+0000 7f45a4ff9640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f459406be30 0x7f459406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.154+0000 7f45a4ff9640 1 -- 172.21.15.55:0/3627808469 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f45a008b8a0 con 0x7f45a815f440 2024-05-20T22:10:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.154+0000 7f45a7fff640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f459406be30 0x7f459406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.154+0000 7f45a7fff640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f459406be30 0x7f459406e340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4598034000 tx=0x7f459802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:09.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.158+0000 7f45a4ff9640 1 -- 172.21.15.55:0/3627808469 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f45a0010040 con 0x7f45a815f440 2024-05-20T22:10:09.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.344+0000 7f45aea4b640 1 -- 172.21.15.55:0/3627808469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45a806ed00 con 0x7f45a815f440 2024-05-20T22:10:09.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.345+0000 7f45a4ff9640 1 -- 172.21.15.55:0/3627808469 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+17122 (secure 0 0 0) 0x7f45a0059a00 con 0x7f45a815f440 2024-05-20T22:10:09.346 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:09.346 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":44,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:09:34.247518+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:09:12.734355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T22:09:13.936532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6835/4137968836":"2024-05-21T22:09:29.217850+0000","172.21.15.138:6833/4137968836":"2024-05-21T22:09:29.217850+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:10:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 -- 172.21.15.55:0/3627808469 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f459406be30 msgr2=0x7f459406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f459406be30 0x7f459406e340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4598034000 tx=0x7f459802e000 comp rx=0 tx=0).stop 2024-05-20T22:10:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 -- 172.21.15.55:0/3627808469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815f440 msgr2=0x7f45a80cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815f440 0x7f45a80cdc20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f45a000efc0 tx=0x7f45a000c490 comp rx=0 tx=0).stop 2024-05-20T22:10:09.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 -- 172.21.15.55:0/3627808469 shutdown_connections 2024-05-20T22:10:09.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f459406be30 0x7f459406e340 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815f440 0x7f45a80cdc20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45a80b3210 0x7f45a80cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 --2- 172.21.15.55:0/3627808469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806d470 0x7f45a8154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 -- 172.21.15.55:0/3627808469 >> 172.21.15.55:0/3627808469 conn(0x7f45a809a400 msgr2=0x7f45a80d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:09.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 -- 172.21.15.55:0/3627808469 shutdown_connections 2024-05-20T22:10:09.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.347+0000 7f45aea4b640 1 -- 172.21.15.55:0/3627808469 wait complete. 2024-05-20T22:10:09.360 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:09:12.734355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:09:13.936532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T22:10:09.360 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:10:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.477+0000 7f57d94bc640 1 Processor -- start 2024-05-20T22:10:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.477+0000 7f57d94bc640 1 -- start start 2024-05-20T22:10:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.478+0000 7f57d94bc640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57d40d3460 0x7f57d415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.478+0000 7f57d94bc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40cfed0 0x7f57d40b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.478+0000 7f57d94bc640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d415d110 con 0x7f57d415cd00 2024-05-20T22:10:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.478+0000 7f57d94bc640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f57d40d3a20 con 0x7f57d40d3460 2024-05-20T22:10:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.478+0000 7f57d94bc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57d40d3d00 con 0x7f57d40cfed0 2024-05-20T22:10:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d2ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57d40d3460 0x7f57d415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d37fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40cfed0 0x7f57d40b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d2ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57d40d3460 0x7f57d415f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:49806/0 (socket says 172.21.15.55:49806) 2024-05-20T22:10:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d37fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40cfed0 0x7f57d40b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:59094/0 (socket says 172.21.15.55:59094) 2024-05-20T22:10:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d27fc640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f57d415cd00 0x7f57d40cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:46456/0 (socket says 172.21.15.55:46456) 2024-05-20T22:10:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d2ffd640 1 -- 172.21.15.55:0/4101329811 learned_addr learned my addr 172.21.15.55:0/4101329811 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d2ffd640 1 -- 172.21.15.55:0/4101329811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40cfed0 msgr2=0x7f57d40b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d2ffd640 1 --2- 172.21.15.55:0/4101329811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40cfed0 0x7f57d40b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d2ffd640 1 -- 172.21.15.55:0/4101329811 >> v1:172.21.15.138:6789/0 conn(0x7f57d415cd00 legacy=0x7f57d40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d1ffb640 1 -- 172.21.15.55:0/4101329811 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538718970 0 0) 0x7f57d415d110 con 0x7f57d415cd00 2024-05-20T22:10:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.479+0000 7f57d2ffd640 1 -- 172.21.15.55:0/4101329811 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d40d3fe0 con 0x7f57d40d3460 2024-05-20T22:10:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.480+0000 7f57d2ffd640 1 --2- 172.21.15.55:0/4101329811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40d3460 0x7f57d415f6e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009e50 tx=0x7f57c402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=400c50b9d0df8a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.480+0000 7f57d1ffb640 1 -- 172.21.15.55:0/4101329811 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4036070 con 0x7f57d40d3460 2024-05-20T22:10:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.480+0000 7f57d1ffb640 1 -- 172.21.15.55:0/4101329811 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c4031600 con 0x7f57d40d3460 2024-05-20T22:10:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.480+0000 7f57d1ffb640 1 -- 172.21.15.55:0/4101329811 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c40319d0 con 0x7f57d40d3460 2024-05-20T22:10:09.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.481+0000 7f57d94bc640 1 -- 172.21.15.55:0/4101329811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40d3460 msgr2=0x7f57d415f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.481+0000 7f57d94bc640 1 --2- 172.21.15.55:0/4101329811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40d3460 0x7f57d415f6e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009e50 tx=0x7f57c402a710 comp rx=0 tx=0).stop 2024-05-20T22:10:09.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.481+0000 7f57d94bc640 1 -- 172.21.15.55:0/4101329811 shutdown_connections 2024-05-20T22:10:09.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.481+0000 7f57d94bc640 1 --2- 172.21.15.55:0/4101329811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40cfed0 0x7f57d40b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.481+0000 7f57d94bc640 1 --2- 172.21.15.55:0/4101329811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40d3460 0x7f57d415f6e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.481+0000 7f57d94bc640 1 -- 172.21.15.55:0/4101329811 >> 172.21.15.55:0/4101329811 conn(0x7f57d40d1c70 msgr2=0x7f57d40d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:09.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.481+0000 7f57d94bc640 1 -- 172.21.15.55:0/4101329811 shutdown_connections 2024-05-20T22:10:09.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.481+0000 7f57d94bc640 1 -- 172.21.15.55:0/4101329811 wait complete. 2024-05-20T22:10:09.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.483+0000 7f57d94bc640 1 Processor -- start 2024-05-20T22:10:09.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.483+0000 7f57d94bc640 1 -- start start 2024-05-20T22:10:09.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.484+0000 7f57d94bc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40d3460 0x7f57d40cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.484+0000 7f57d94bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57d415cd00 0x7f57d414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.484+0000 7f57d94bc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40cd100 0x7f57d4161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.484+0000 7f57d94bc640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f57d40d0620 con 0x7f57d415cd00 2024-05-20T22:10:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.484+0000 7f57d94bc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57d40d04a0 con 0x7f57d40cd100 2024-05-20T22:10:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.484+0000 7f57d94bc640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57d40b3bc0 con 0x7f57d40d3460 2024-05-20T22:10:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57d415cd00 0x7f57d414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57d415cd00 0x7f57d414df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44668/0 (socket says 172.21.15.55:44668) 2024-05-20T22:10:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d27fc640 1 -- 172.21.15.55:0/2103487034 learned_addr learned my addr 172.21.15.55:0/2103487034 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d37fe640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40cd100 0x7f57d4161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d2ffd640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40d3460 0x7f57d40cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d27fc640 1 -- 172.21.15.55:0/2103487034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40d3460 msgr2=0x7f57d40cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d27fc640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40d3460 0x7f57d40cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d27fc640 1 -- 172.21.15.55:0/2103487034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40cd100 msgr2=0x7f57d4161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d27fc640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40cd100 0x7f57d4161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d27fc640 1 -- 172.21.15.55:0/2103487034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d40cb510 con 0x7f57d415cd00 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d27fc640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57d415cd00 0x7f57d414df20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c440 tx=0x7f57c800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57bb7fe640 1 -- 172.21.15.55:0/2103487034 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8018070 con 0x7f57d415cd00 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57bb7fe640 1 -- 172.21.15.55:0/2103487034 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c80074e0 con 0x7f57d415cd00 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d94bc640 1 -- 172.21.15.55:0/2103487034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d414e460 con 0x7f57d415cd00 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57bb7fe640 1 -- 172.21.15.55:0/2103487034 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8004a20 con 0x7f57d415cd00 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.485+0000 7f57d94bc640 1 -- 172.21.15.55:0/2103487034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d40cc340 con 0x7f57d415cd00 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.486+0000 7f57d94bc640 1 -- 172.21.15.55:0/2103487034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d404f720 con 0x7f57d415cd00 2024-05-20T22:10:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.487+0000 7f57bb7fe640 1 -- 172.21.15.55:0/2103487034 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57c8004bc0 con 0x7f57d415cd00 2024-05-20T22:10:09.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.487+0000 7f57bb7fe640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57c006be30 0x7f57c006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.487+0000 7f57bb7fe640 1 -- 172.21.15.55:0/2103487034 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f57c808a450 con 0x7f57d415cd00 2024-05-20T22:10:09.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.487+0000 7f57d2ffd640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57c006be30 0x7f57c006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.488+0000 7f57d2ffd640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57c006be30 0x7f57c006e340 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f57c4033000 tx=0x7f57c4009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:09.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.492+0000 7f57bb7fe640 1 -- 172.21.15.55:0/2103487034 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f57c8058af0 con 0x7f57d415cd00 2024-05-20T22:10:09.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.677+0000 7f57d94bc640 1 -- 172.21.15.55:0/2103487034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57d40b7950 con 0x7f57d415cd00 2024-05-20T22:10:09.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.678+0000 7f57bb7fe640 1 -- 172.21.15.55:0/2103487034 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) ==== 93+0+10 (secure 0 0 0) 0x7f57c80583f0 con 0x7f57d415cd00 2024-05-20T22:10:09.679 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:10:09.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 -- 172.21.15.55:0/2103487034 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57c006be30 msgr2=0x7f57c006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57c006be30 0x7f57c006e340 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f57c4033000 tx=0x7f57c4009000 comp rx=0 tx=0).stop 2024-05-20T22:10:09.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 -- 172.21.15.55:0/2103487034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57d415cd00 msgr2=0x7f57d414df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57d415cd00 0x7f57d414df20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c440 tx=0x7f57c800c900 comp rx=0 tx=0).stop 2024-05-20T22:10:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 -- 172.21.15.55:0/2103487034 shutdown_connections 2024-05-20T22:10:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f57c006be30 0x7f57c006e340 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40cd100 0x7f57d4161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57d415cd00 0x7f57d414df20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 --2- 172.21.15.55:0/2103487034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57d40d3460 0x7f57d40cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 -- 172.21.15.55:0/2103487034 >> 172.21.15.55:0/2103487034 conn(0x7f57d40d1c70 msgr2=0x7f57d406c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 -- 172.21.15.55:0/2103487034 shutdown_connections 2024-05-20T22:10:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.680+0000 7f57d94bc640 1 -- 172.21.15.55:0/2103487034 wait complete. 2024-05-20T22:10:09.693 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:10:09.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.809+0000 7facff008640 1 Processor -- start 2024-05-20T22:10:09.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.809+0000 7facff008640 1 -- start start 2024-05-20T22:10:09.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.810+0000 7facff008640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf80d3460 0x7facf815f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facff008640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7facf80cfed0 0x7facf80b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facff008640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf815d110 con 0x7facf815cd00 2024-05-20T22:10:09.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facff008640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7facf80d3a20 con 0x7facf80cfed0 2024-05-20T22:10:09.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facff008640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7facf80d3d00 con 0x7facf80d3460 2024-05-20T22:10:09.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facfd5a5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7facf80cfed0 0x7facf80b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facfd5a5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7facf80cfed0 0x7facf80b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44674/0 (socket says 172.21.15.55:44674) 2024-05-20T22:10:09.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facfd5a5640 1 -- 172.21.15.55:0/1352904085 learned_addr learned my addr 172.21.15.55:0/1352904085 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:09.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facfcda4640 1 --2- 172.21.15.55:0/1352904085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf80d3460 0x7facf815f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facfd5a5640 1 -- 172.21.15.55:0/1352904085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf80d3460 msgr2=0x7facf815f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:09.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facfd5a5640 1 --2- 172.21.15.55:0/1352904085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf80d3460 0x7facf815f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facfd5a5640 1 -- 172.21.15.55:0/1352904085 >> v1:172.21.15.138:6789/0 conn(0x7facf815cd00 legacy=0x7facf80cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.811+0000 7facfd5a5640 1 -- 172.21.15.55:0/1352904085 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf80d3fe0 con 0x7facf80cfed0 2024-05-20T22:10:09.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.812+0000 7facfd5a5640 1 --2- 172.21.15.55:0/1352904085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80cfed0 0x7facf80b3680 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7facf0009e50 tx=0x7facf002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19a53312f30d21e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:09.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.812+0000 7facf77fe640 1 -- 172.21.15.55:0/1352904085 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0036070 con 0x7facf80cfed0 2024-05-20T22:10:09.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.812+0000 7facf77fe640 1 -- 172.21.15.55:0/1352904085 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf0031600 con 0x7facf80cfed0 2024-05-20T22:10:09.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.812+0000 7facf77fe640 1 -- 172.21.15.55:0/1352904085 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00319d0 con 0x7facf80cfed0 2024-05-20T22:10:09.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.813+0000 7facff008640 1 -- 172.21.15.55:0/1352904085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80cfed0 msgr2=0x7facf80b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.813+0000 7facff008640 1 --2- 172.21.15.55:0/1352904085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80cfed0 0x7facf80b3680 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7facf0009e50 tx=0x7facf002a710 comp rx=0 tx=0).stop 2024-05-20T22:10:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.813+0000 7facff008640 1 -- 172.21.15.55:0/1352904085 shutdown_connections 2024-05-20T22:10:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.813+0000 7facff008640 1 --2- 172.21.15.55:0/1352904085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80cfed0 0x7facf80b3680 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.813+0000 7facff008640 1 --2- 172.21.15.55:0/1352904085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf80d3460 0x7facf815f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.813+0000 7facff008640 1 -- 172.21.15.55:0/1352904085 >> 172.21.15.55:0/1352904085 conn(0x7facf80d1c70 msgr2=0x7facf80d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.813+0000 7facff008640 1 -- 172.21.15.55:0/1352904085 shutdown_connections 2024-05-20T22:10:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.813+0000 7facff008640 1 -- 172.21.15.55:0/1352904085 wait complete. 2024-05-20T22:10:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.814+0000 7facff008640 1 Processor -- start 2024-05-20T22:10:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.815+0000 7facff008640 1 -- start start 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facff008640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80d3460 0x7facf815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facff008640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf815cd00 0x7facf81f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facff008640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8146910 0x7facf81573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facff008640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7facf80d0620 con 0x7facf80d3460 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facff008640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facf80d04a0 con 0x7facf8146910 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facff008640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7facf80b3bc0 con 0x7facf815cd00 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facfcda4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80d3460 0x7facf815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facfcda4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80d3460 0x7facf815fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44678/0 (socket says 172.21.15.55:44678) 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facfcda4640 1 -- 172.21.15.55:0/3294650037 learned_addr learned my addr 172.21.15.55:0/3294650037 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facf7fff640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf815cd00 0x7facf81f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facfd5a5640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8146910 0x7facf81573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facfcda4640 1 -- 172.21.15.55:0/3294650037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf815cd00 msgr2=0x7facf81f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facfcda4640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf815cd00 0x7facf81f4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facfcda4640 1 -- 172.21.15.55:0/3294650037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8146910 msgr2=0x7facf81573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facfcda4640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8146910 0x7facf81573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.816+0000 7facfcda4640 1 -- 172.21.15.55:0/3294650037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf8155ed0 con 0x7facf80d3460 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.817+0000 7facfcda4640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80d3460 0x7facf815fa30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7face800d5d0 tx=0x7face800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.817+0000 7facf57fa640 1 -- 172.21.15.55:0/3294650037 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80125f0 con 0x7facf80d3460 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.817+0000 7facff008640 1 -- 172.21.15.55:0/3294650037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf815ff70 con 0x7facf80d3460 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.817+0000 7facf57fa640 1 -- 172.21.15.55:0/3294650037 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face8019070 con 0x7facf80d3460 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.817+0000 7facf57fa640 1 -- 172.21.15.55:0/3294650037 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80129f0 con 0x7facf80d3460 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.817+0000 7facff008640 1 -- 172.21.15.55:0/3294650037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf8156620 con 0x7facf80d3460 2024-05-20T22:10:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.818+0000 7facff008640 1 -- 172.21.15.55:0/3294650037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf804f720 con 0x7facf80d3460 2024-05-20T22:10:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.819+0000 7facf57fa640 1 -- 172.21.15.55:0/3294650037 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7face8012b90 con 0x7facf80d3460 2024-05-20T22:10:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.819+0000 7facf57fa640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7face406be60 0x7face406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.819+0000 7facf57fa640 1 -- 172.21.15.55:0/3294650037 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7face808b460 con 0x7facf80d3460 2024-05-20T22:10:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.819+0000 7facf7fff640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7face406be60 0x7face406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.820+0000 7facf7fff640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7face406be60 0x7face406e370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facf80cf290 tx=0x7facec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:09.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:09.824+0000 7facf57fa640 1 -- 172.21.15.55:0/3294650037 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7face8059b90 con 0x7facf80d3460 2024-05-20T22:10:10.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.011+0000 7facff008640 1 -- 172.21.15.55:0/3294650037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7facf80d2f80 con 0x7facf80d3460 2024-05-20T22:10:10.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.012+0000 7facf57fa640 1 -- 172.21.15.55:0/3294650037 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) ==== 104+0+11 (secure 0 0 0) 0x7face8059490 con 0x7facf80d3460 2024-05-20T22:10:10.013 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:10:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 -- 172.21.15.55:0/3294650037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7face406be60 msgr2=0x7face406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7face406be60 0x7face406e370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facf80cf290 tx=0x7facec009040 comp rx=0 tx=0).stop 2024-05-20T22:10:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 -- 172.21.15.55:0/3294650037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80d3460 msgr2=0x7facf815fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80d3460 0x7facf815fa30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7face800d5d0 tx=0x7face800da90 comp rx=0 tx=0).stop 2024-05-20T22:10:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 -- 172.21.15.55:0/3294650037 shutdown_connections 2024-05-20T22:10:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7face406be60 0x7face406e370 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8146910 0x7facf81573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facf815cd00 0x7facf81f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 --2- 172.21.15.55:0/3294650037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facf80d3460 0x7facf815fa30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 -- 172.21.15.55:0/3294650037 >> 172.21.15.55:0/3294650037 conn(0x7facf80d1c70 msgr2=0x7facf815bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 -- 172.21.15.55:0/3294650037 shutdown_connections 2024-05-20T22:10:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.014+0000 7facff008640 1 -- 172.21.15.55:0/3294650037 wait complete. 2024-05-20T22:10:10.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:10:10.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.143+0000 7fd6e4538640 1 Processor -- start 2024-05-20T22:10:10.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.143+0000 7fd6e4538640 1 -- start start 2024-05-20T22:10:10.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.144+0000 7fd6e4538640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd6dc0b84f0 0x7fd6dc15f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.144+0000 7fd6e4538640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc163700 0x7fd6dc06bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.144+0000 7fd6e4538640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6dc0d3c10 con 0x7fd6dc0b40a0 2024-05-20T22:10:10.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.144+0000 7fd6e4538640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd6dc0d39c0 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.144+0000 7fd6e4538640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc06b750 con 0x7fd6dc163700 2024-05-20T22:10:10.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.145+0000 7fd6e1ad3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd6dc0b84f0 0x7fd6dc15f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.145+0000 7fd6e1ad3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd6dc0b84f0 0x7fd6dc15f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.55:49828/0 (socket says 172.21.15.55:49828) 2024-05-20T22:10:10.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.145+0000 7fd6e22d4640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd6dc0b40a0 0x7fd6dc162ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:46476/0 (socket says 172.21.15.55:46476) 2024-05-20T22:10:10.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.145+0000 7fd6e2ad5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc163700 0x7fd6dc06bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.145+0000 7fd6e1ad3640 1 -- 172.21.15.55:0/1076112784 learned_addr learned my addr 172.21.15.55:0/1076112784 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:10.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.145+0000 7fd6e2ad5640 1 -- 172.21.15.55:0/1076112784 >> v2:172.21.15.138:3300/0 conn(0x7fd6dc0b84f0 msgr2=0x7fd6dc15f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:10.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.145+0000 7fd6e2ad5640 1 --2- 172.21.15.55:0/1076112784 >> v2:172.21.15.138:3300/0 conn(0x7fd6dc0b84f0 0x7fd6dc15f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.145+0000 7fd6e2ad5640 1 -- 172.21.15.55:0/1076112784 >> v1:172.21.15.138:6789/0 conn(0x7fd6dc0b40a0 legacy=0x7fd6dc162ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.145+0000 7fd6e2ad5640 1 -- 172.21.15.55:0/1076112784 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc06ba30 con 0x7fd6dc163700 2024-05-20T22:10:10.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.145+0000 7fd6e1ad3640 1 --2- 172.21.15.55:0/1076112784 >> v2:172.21.15.138:3300/0 conn(0x7fd6dc0b84f0 0x7fd6dc15f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:10.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.146+0000 7fd6e2ad5640 1 --2- 172.21.15.55:0/1076112784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc163700 0x7fd6dc06bf50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009d90 tx=0x7fd6d402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de2db7c59e4d9db server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:10.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.146+0000 7fd6e12d2640 1 -- 172.21.15.55:0/1076112784 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4031600 con 0x7fd6dc163700 2024-05-20T22:10:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.146+0000 7fd6e12d2640 1 -- 172.21.15.55:0/1076112784 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d40317a0 con 0x7fd6dc163700 2024-05-20T22:10:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.146+0000 7fd6e12d2640 1 -- 172.21.15.55:0/1076112784 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4031b70 con 0x7fd6dc163700 2024-05-20T22:10:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.147+0000 7fd6e4538640 1 -- 172.21.15.55:0/1076112784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc163700 msgr2=0x7fd6dc06bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.147+0000 7fd6e4538640 1 --2- 172.21.15.55:0/1076112784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc163700 0x7fd6dc06bf50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009d90 tx=0x7fd6d402a5d0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.147+0000 7fd6e4538640 1 -- 172.21.15.55:0/1076112784 shutdown_connections 2024-05-20T22:10:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.147+0000 7fd6e4538640 1 --2- 172.21.15.55:0/1076112784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc163700 0x7fd6dc06bf50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.147+0000 7fd6e4538640 1 --2- 172.21.15.55:0/1076112784 >> v2:172.21.15.138:3300/0 conn(0x7fd6dc0b84f0 0x7fd6dc15f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.147+0000 7fd6e4538640 1 -- 172.21.15.55:0/1076112784 >> 172.21.15.55:0/1076112784 conn(0x7fd6dc0d15a0 msgr2=0x7fd6dc0d19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.147+0000 7fd6e4538640 1 -- 172.21.15.55:0/1076112784 shutdown_connections 2024-05-20T22:10:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.147+0000 7fd6e4538640 1 -- 172.21.15.55:0/1076112784 wait complete. 2024-05-20T22:10:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.149+0000 7fd6e4538640 1 Processor -- start 2024-05-20T22:10:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.149+0000 7fd6e4538640 1 -- start start 2024-05-20T22:10:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.150+0000 7fd6e4538640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc0b40a0 0x7fd6dc1f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.150+0000 7fd6e4538640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc0b84f0 0x7fd6dc1550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.150+0000 7fd6e4538640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6dc163700 0x7fd6dc15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.150+0000 7fd6e4538640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc163c90 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.150+0000 7fd6e4538640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc163b10 con 0x7fd6dc163700 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.150+0000 7fd6e4538640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc163e10 con 0x7fd6dc0b40a0 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.150+0000 7fd6e1ad3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc0b84f0 0x7fd6dc1550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.150+0000 7fd6e1ad3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc0b84f0 0x7fd6dc1550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44684/0 (socket says 172.21.15.55:44684) 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.150+0000 7fd6e1ad3640 1 -- 172.21.15.55:0/2468799376 learned_addr learned my addr 172.21.15.55:0/2468799376 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6e2ad5640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6dc163700 0x7fd6dc15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6e22d4640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc0b40a0 0x7fd6dc1f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6e1ad3640 1 -- 172.21.15.55:0/2468799376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc0b40a0 msgr2=0x7fd6dc1f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6e1ad3640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc0b40a0 0x7fd6dc1f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6e1ad3640 1 -- 172.21.15.55:0/2468799376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6dc163700 msgr2=0x7fd6dc15a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6e1ad3640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6dc163700 0x7fd6dc15a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6e1ad3640 1 -- 172.21.15.55:0/2468799376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc1625d0 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6e1ad3640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc0b84f0 0x7fd6dc1550c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00e980 tx=0x7fd6cc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:10.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6d2ffd640 1 -- 172.21.15.55:0/2468799376 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc017070 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6d2ffd640 1 -- 172.21.15.55:0/2468799376 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc00ca00 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6e4538640 1 -- 172.21.15.55:0/2468799376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc1628b0 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6d2ffd640 1 -- 172.21.15.55:0/2468799376 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00cdd0 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.151+0000 7fd6e4538640 1 -- 172.21.15.55:0/2468799376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc155600 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.152+0000 7fd6e4538640 1 -- 172.21.15.55:0/2468799376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6dc162340 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.153+0000 7fd6d2ffd640 1 -- 172.21.15.55:0/2468799376 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6cc01a070 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.153+0000 7fd6d2ffd640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd6c006be30 0x7fd6c006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.153+0000 7fd6d2ffd640 1 -- 172.21.15.55:0/2468799376 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7fd6cc08b400 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.154+0000 7fd6e22d4640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd6c006be30 0x7fd6c006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.154+0000 7fd6e22d4640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd6c006be30 0x7fd6c006e340 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8007fd0 tx=0x7fd6d8007c40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:10.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.158+0000 7fd6d2ffd640 1 -- 172.21.15.55:0/2468799376 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd6cc059be0 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.346+0000 7fd6e4538640 1 -- 172.21.15.55:0/2468799376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd6dc155870 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.346+0000 7fd6d2ffd640 1 -- 172.21.15.55:0/2468799376 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) ==== 100+0+11 (secure 0 0 0) 0x7fd6dc155870 con 0x7fd6dc0b84f0 2024-05-20T22:10:10.347 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 -- 172.21.15.55:0/2468799376 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd6c006be30 msgr2=0x7fd6c006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd6c006be30 0x7fd6c006e340 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8007fd0 tx=0x7fd6d8007c40 comp rx=0 tx=0).stop 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 -- 172.21.15.55:0/2468799376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc0b84f0 msgr2=0x7fd6dc1550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc0b84f0 0x7fd6dc1550c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00e980 tx=0x7fd6cc00ee40 comp rx=0 tx=0).stop 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 -- 172.21.15.55:0/2468799376 shutdown_connections 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd6c006be30 0x7fd6c006e340 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6dc163700 0x7fd6dc15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc0b84f0 0x7fd6dc1550c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 --2- 172.21.15.55:0/2468799376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6dc0b40a0 0x7fd6dc1f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 -- 172.21.15.55:0/2468799376 >> 172.21.15.55:0/2468799376 conn(0x7fd6dc0d15a0 msgr2=0x7fd6dc0b8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:10.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 -- 172.21.15.55:0/2468799376 shutdown_connections 2024-05-20T22:10:10.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.349+0000 7fd6e4538640 1 -- 172.21.15.55:0/2468799376 wait complete. 2024-05-20T22:10:10.361 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:10.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.474+0000 7f3c54afe640 1 Processor -- start 2024-05-20T22:10:10.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.475+0000 7f3c54afe640 1 -- start start 2024-05-20T22:10:10.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.476+0000 7f3c54afe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c500d3400 0x7f3c501600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.476+0000 7f3c54afe640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c500d44d0 con 0x7f3c500cfeb0 2024-05-20T22:10:10.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.476+0000 7f3c54afe640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c500d2fe0 con 0x7f3c5015cd50 2024-05-20T22:10:10.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.476+0000 7f3c54afe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c501605f0 con 0x7f3c500d3400 2024-05-20T22:10:10.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.476+0000 7f3c4ed9d640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3c500cfeb0 0x7f3c500b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56406/0 (socket says 172.21.15.55:56406) 2024-05-20T22:10:10.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.476+0000 7f3c4ed9d640 1 -- 172.21.15.55:0/1639863079 learned_addr learned my addr 172.21.15.55:0/1639863079 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:10.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.476+0000 7f3c4e59c640 1 --2- 172.21.15.55:0/1639863079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c500d3400 0x7f3c501600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.476+0000 7f3c4d59a640 1 -- 172.21.15.55:0/1639863079 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1143475218 0 0) 0x7f3c500d44d0 con 0x7f3c500cfeb0 2024-05-20T22:10:10.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.477+0000 7f3c4d59a640 1 -- 172.21.15.55:0/1639863079 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c38003520 con 0x7f3c500cfeb0 2024-05-20T22:10:10.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.477+0000 7f3c4d59a640 1 -- 172.21.15.55:0/1639863079 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1334184175 0 0) 0x7f3c500d2fe0 con 0x7f3c5015cd50 2024-05-20T22:10:10.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.477+0000 7f3c4d59a640 1 -- 172.21.15.55:0/1639863079 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c500d44d0 con 0x7f3c5015cd50 2024-05-20T22:10:10.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.477+0000 7f3c4e59c640 1 -- 172.21.15.55:0/1639863079 >> v1:172.21.15.138:6789/0 conn(0x7f3c5015cd50 legacy=0x7f3c500cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.477+0000 7f3c4e59c640 1 -- 172.21.15.55:0/1639863079 >> v1:172.21.15.55:6789/0 conn(0x7f3c500cfeb0 legacy=0x7f3c500b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.477+0000 7f3c4d59a640 1 -- 172.21.15.55:0/1639863079 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3098094124 0 0) 0x7f3c38003520 con 0x7f3c500cfeb0 2024-05-20T22:10:10.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.477+0000 7f3c4e59c640 1 -- 172.21.15.55:0/1639863079 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c500d3950 con 0x7f3c500d3400 2024-05-20T22:10:10.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.478+0000 7f3c4e59c640 1 --2- 172.21.15.55:0/1639863079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c500d3400 0x7f3c501600b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009dc0 tx=0x7f3c4002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc609f04657c2ec server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:10.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.478+0000 7f3c4d59a640 1 -- 172.21.15.55:0/1639863079 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48001670 con 0x7f3c500d3400 2024-05-20T22:10:10.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.478+0000 7f3c4d59a640 1 -- 172.21.15.55:0/1639863079 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c40007540 con 0x7f3c500d3400 2024-05-20T22:10:10.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.478+0000 7f3c4d59a640 1 -- 172.21.15.55:0/1639863079 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400078f0 con 0x7f3c500d3400 2024-05-20T22:10:10.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.478+0000 7f3c54afe640 1 -- 172.21.15.55:0/1639863079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c500d3400 msgr2=0x7f3c501600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.478+0000 7f3c54afe640 1 --2- 172.21.15.55:0/1639863079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c500d3400 0x7f3c501600b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009dc0 tx=0x7f3c4002a660 comp rx=0 tx=0).stop 2024-05-20T22:10:10.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.479+0000 7f3c54afe640 1 -- 172.21.15.55:0/1639863079 shutdown_connections 2024-05-20T22:10:10.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.479+0000 7f3c54afe640 1 --2- 172.21.15.55:0/1639863079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c500d3400 0x7f3c501600b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.479+0000 7f3c54afe640 1 -- 172.21.15.55:0/1639863079 >> 172.21.15.55:0/1639863079 conn(0x7f3c500d1c80 msgr2=0x7f3c500d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:10.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.479+0000 7f3c54afe640 1 -- 172.21.15.55:0/1639863079 shutdown_connections 2024-05-20T22:10:10.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.479+0000 7f3c54afe640 1 -- 172.21.15.55:0/1639863079 wait complete. 2024-05-20T22:10:10.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.481+0000 7f3c54afe640 1 Processor -- start 2024-05-20T22:10:10.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.481+0000 7f3c54afe640 1 -- start start 2024-05-20T22:10:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c54afe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c500d3400 0x7f3c500c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c54afe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c5015cd50 0x7f3c5014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c54afe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c501f9250 0x7f3c50162820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c54afe640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3c500d41f0 con 0x7f3c500d3400 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c54afe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c500d3f10 con 0x7f3c5015cd50 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c54afe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5006c8a0 con 0x7f3c501f9250 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c4e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c500d3400 0x7f3c500c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c4e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c500d3400 0x7f3c500c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44694/0 (socket says 172.21.15.55:44694) 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c4e59c640 1 -- 172.21.15.55:0/1256406182 learned_addr learned my addr 172.21.15.55:0/1256406182 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c4ed9d640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c501f9250 0x7f3c50162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c4dd9b640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c5015cd50 0x7f3c5014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c4e59c640 1 -- 172.21.15.55:0/1256406182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c501f9250 msgr2=0x7f3c50162820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c4e59c640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c501f9250 0x7f3c50162820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c4e59c640 1 -- 172.21.15.55:0/1256406182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c5015cd50 msgr2=0x7f3c5014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c4e59c640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c5015cd50 0x7f3c5014bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.482+0000 7f3c4e59c640 1 -- 172.21.15.55:0/1256406182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c501617c0 con 0x7f3c500d3400 2024-05-20T22:10:10.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.483+0000 7f3c4e59c640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c500d3400 0x7f3c500c8f80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009ee0 tx=0x7f3c40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:10.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.483+0000 7f3c3effd640 1 -- 172.21.15.55:0/1256406182 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40040070 con 0x7f3c500d3400 2024-05-20T22:10:10.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.483+0000 7f3c3effd640 1 -- 172.21.15.55:0/1256406182 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c40031bc0 con 0x7f3c500d3400 2024-05-20T22:10:10.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.483+0000 7f3c54afe640 1 -- 172.21.15.55:0/1256406182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c50161a60 con 0x7f3c500d3400 2024-05-20T22:10:10.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.483+0000 7f3c3effd640 1 -- 172.21.15.55:0/1256406182 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40009450 con 0x7f3c500d3400 2024-05-20T22:10:10.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.483+0000 7f3c54afe640 1 -- 172.21.15.55:0/1256406182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c50162020 con 0x7f3c500d3400 2024-05-20T22:10:10.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.484+0000 7f3c54afe640 1 -- 172.21.15.55:0/1256406182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c5004f720 con 0x7f3c500d3400 2024-05-20T22:10:10.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.484+0000 7f3c3effd640 1 -- 172.21.15.55:0/1256406182 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c40039040 con 0x7f3c500d3400 2024-05-20T22:10:10.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.485+0000 7f3c3effd640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3c3006be30 0x7f3c3006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.485+0000 7f3c3effd640 1 -- 172.21.15.55:0/1256406182 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f3c400ac0b0 con 0x7f3c500d3400 2024-05-20T22:10:10.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.485+0000 7f3c4dd9b640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3c3006be30 0x7f3c3006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.485+0000 7f3c4dd9b640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3c3006be30 0x7f3c3006e340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c5015df40 tx=0x7f3c44009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:10.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.490+0000 7f3c3effd640 1 -- 172.21.15.55:0/1256406182 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3c4004c070 con 0x7f3c500d3400 2024-05-20T22:10:10.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.713+0000 7f3c54afe640 1 -- 172.21.15.55:0/1256406182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c500b8ce0 con 0x7f3c500d3400 2024-05-20T22:10:10.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.714+0000 7f3c3effd640 1 -- 172.21.15.55:0/1256406182 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5191 (secure 0 0 0) 0x7f3c4007a100 con 0x7f3c500d3400 2024-05-20T22:10:10.715 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:10.715 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3873372908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3873372908},{"type":"v1","addr":"172.21.15.55:6836","nonce":3873372908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3417305427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":3417305427},{"type":"v1","addr":"172.21.15.55:6837","nonce":3417305427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":35081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4211704780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4211704780},{"type":"v1","addr":"172.21.15.138:6833","nonce":4211704780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:09:14.957632+0000","modified":"2024-05-20T22:09:33.242208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":35093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35093":{"gid":35093,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6835/270300569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":270300569},{"type":"v1","addr":"172.21.15.138:6835","nonce":270300569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":35093,"qdb_cluster":[35093]},"id":2}]} 2024-05-20T22:10:10.715 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2024-05-20T22:10:10.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 -- 172.21.15.55:0/1256406182 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3c3006be30 msgr2=0x7f3c3006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3c3006be30 0x7f3c3006e340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c5015df40 tx=0x7f3c44009040 comp rx=0 tx=0).stop 2024-05-20T22:10:10.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 -- 172.21.15.55:0/1256406182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c500d3400 msgr2=0x7f3c500c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c500d3400 0x7f3c500c8f80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009ee0 tx=0x7f3c40004a40 comp rx=0 tx=0).stop 2024-05-20T22:10:10.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 -- 172.21.15.55:0/1256406182 shutdown_connections 2024-05-20T22:10:10.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3c3006be30 0x7f3c3006e340 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c501f9250 0x7f3c50162820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c5015cd50 0x7f3c5014bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 --2- 172.21.15.55:0/1256406182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c500d3400 0x7f3c500c8f80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 -- 172.21.15.55:0/1256406182 >> 172.21.15.55:0/1256406182 conn(0x7f3c500d1c80 msgr2=0x7f3c5015ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:10.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 -- 172.21.15.55:0/1256406182 shutdown_connections 2024-05-20T22:10:10.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.716+0000 7f3c54afe640 1 -- 172.21.15.55:0/1256406182 wait complete. 2024-05-20T22:10:10.728 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:10:10.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.842+0000 7fdcc9dd5640 1 Processor -- start 2024-05-20T22:10:10.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.842+0000 7fdcc9dd5640 1 -- start start 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.843+0000 7fdcc9dd5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc40cd290 0x7fdcc415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.843+0000 7fdcc9dd5640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc415d4a0 con 0x7fdcc40b5b80 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.843+0000 7fdcc9dd5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc4161380 con 0x7fdcc406f0f0 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.843+0000 7fdcc9dd5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc40d1090 con 0x7fdcc40cd290 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.843+0000 7fdcc37fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdcc40b5b80 0x7fdcc4160bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56416/0 (socket says 172.21.15.55:56416) 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.843+0000 7fdcc37fe640 1 -- 172.21.15.55:0/1732259718 learned_addr learned my addr 172.21.15.55:0/1732259718 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 384968005 0 0) 0x7fdcc415d4a0 con 0x7fdcc40b5b80 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc2ffd640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc40cd290 0x7fdcc415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcb0003520 con 0x7fdcc40b5b80 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1936671465 0 0) 0x7fdcb0003520 con 0x7fdcc40b5b80 2024-05-20T22:10:10.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc40cd290 msgr2=0x7fdcc415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc40cd290 0x7fdcc415cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 >> v1:172.21.15.138:6789/0 conn(0x7fdcc406f0f0 legacy=0x7fdcc406f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc4161660 con 0x7fdcc40b5b80 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fdcb40019f0 con 0x7fdcc40b5b80 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 >> v1:172.21.15.55:6789/0 conn(0x7fdcc40b5b80 legacy=0x7fdcc4160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcb00039e0 0x7fdcb0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc40cd290 0x7fdcb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcb0004d40 0x7fdcb0005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc40d1090 con 0x7fdcb0004d40 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb000bba0 con 0x7fdcb00039e0 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb000bda0 con 0x7fdcc40cd290 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc3fff640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcb0004d40 0x7fdcb0005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc37fe640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcb00039e0 0x7fdcb0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.844+0000 7fdcc2ffd640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc40cd290 0x7fdcb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.845+0000 7fdcc3fff640 1 -- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc40cd290 msgr2=0x7fdcb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:10.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.845+0000 7fdcc3fff640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc40cd290 0x7fdcb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.845+0000 7fdcc3fff640 1 -- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcb00039e0 msgr2=0x7fdcb0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:10.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.845+0000 7fdcc3fff640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcb00039e0 0x7fdcb0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.845+0000 7fdcc3fff640 1 -- 172.21.15.55:0/1732259718 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb0003520 con 0x7fdcb0004d40 2024-05-20T22:10:10.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.845+0000 7fdcc3fff640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcb0004d40 0x7fdcb0005170 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc001720 tx=0x7fdcb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b3dd151e226983 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:10.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.845+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc001370 con 0x7fdcb0004d40 2024-05-20T22:10:10.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.845+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc0052d0 con 0x7fdcb0004d40 2024-05-20T22:10:10.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.845+0000 7fdcc27fc640 1 -- 172.21.15.55:0/1732259718 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc02abc0 con 0x7fdcb0004d40 2024-05-20T22:10:10.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.846+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1732259718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcb0004d40 msgr2=0x7fdcb0005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.846+0000 7fdcc9dd5640 1 --2- 172.21.15.55:0/1732259718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcb0004d40 0x7fdcb0005170 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc001720 tx=0x7fdcb0005830 comp rx=0 tx=0).stop 2024-05-20T22:10:10.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.846+0000 7fdcc3fff640 1 -- 172.21.15.55:0/1732259718 reap_dead start 2024-05-20T22:10:10.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.846+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1732259718 shutdown_connections 2024-05-20T22:10:10.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.846+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1732259718 >> 172.21.15.55:0/1732259718 conn(0x7fdcc40d2910 msgr2=0x7fdcc40d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:10.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.846+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1732259718 shutdown_connections 2024-05-20T22:10:10.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.846+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1732259718 wait complete. 2024-05-20T22:10:10.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.848+0000 7fdcc9dd5640 1 Processor -- start 2024-05-20T22:10:10.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.848+0000 7fdcc9dd5640 1 -- start start 2024-05-20T22:10:10.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.849+0000 7fdcc9dd5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb0004d40 0x7fdcc41560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc9dd5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc40c6e30 0x7fdcc40c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc9dd5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc40c8d20 0x7fdcc415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc9dd5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc415cf60 con 0x7fdcc40c6e30 2024-05-20T22:10:10.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc9dd5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc40d12e0 con 0x7fdcc40c8d20 2024-05-20T22:10:10.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc9dd5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc415d0e0 con 0x7fdcb0004d40 2024-05-20T22:10:10.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc40c6e30 0x7fdcc40c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc40c6e30 0x7fdcc40c87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44708/0 (socket says 172.21.15.55:44708) 2024-05-20T22:10:10.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc2ffd640 1 -- 172.21.15.55:0/1957334118 learned_addr learned my addr 172.21.15.55:0/1957334118 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc3fff640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc40c8d20 0x7fdcc415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc37fe640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb0004d40 0x7fdcc41560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc2ffd640 1 -- 172.21.15.55:0/1957334118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb0004d40 msgr2=0x7fdcc41560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc2ffd640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb0004d40 0x7fdcc41560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc2ffd640 1 -- 172.21.15.55:0/1957334118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc40c8d20 msgr2=0x7fdcc415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc2ffd640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc40c8d20 0x7fdcc415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc2ffd640 1 -- 172.21.15.55:0/1957334118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc40c6670 con 0x7fdcc40c6e30 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc3fff640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc40c8d20 0x7fdcc415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc37fe640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb0004d40 0x7fdcc41560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc2ffd640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc40c6e30 0x7fdcc40c87e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0119d0 tx=0x7fdcac011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcbbfff640 1 -- 172.21.15.55:0/1957334118 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00fb50 con 0x7fdcc40c6e30 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcbbfff640 1 -- 172.21.15.55:0/1957334118 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcac00fcf0 con 0x7fdcc40c6e30 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1957334118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc40c6950 con 0x7fdcc40c6e30 2024-05-20T22:10:10.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcbbfff640 1 -- 172.21.15.55:0/1957334118 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00fea0 con 0x7fdcc40c6e30 2024-05-20T22:10:10.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.850+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1957334118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc415fc70 con 0x7fdcc40c6e30 2024-05-20T22:10:10.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.851+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1957334118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc404f720 con 0x7fdcc40c6e30 2024-05-20T22:10:10.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.852+0000 7fdcbbfff640 1 -- 172.21.15.55:0/1957334118 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcac012040 con 0x7fdcc40c6e30 2024-05-20T22:10:10.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.852+0000 7fdcbbfff640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdca406be60 0x7fdca406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:10.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.853+0000 7fdcbbfff640 1 -- 172.21.15.55:0/1957334118 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7fdcac08b6b0 con 0x7fdcc40c6e30 2024-05-20T22:10:10.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.853+0000 7fdcc37fe640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdca406be60 0x7fdca406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:10.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.853+0000 7fdcc37fe640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdca406be60 0x7fdca406e370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4004d00 tx=0x7fdcb4006040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:10.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:10.857+0000 7fdcbbfff640 1 -- 172.21.15.55:0/1957334118 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdcac019030 con 0x7fdcc40c6e30 2024-05-20T22:10:11.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.046+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1957334118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcc40b2050 con 0x7fdcc40c6e30 2024-05-20T22:10:11.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.047+0000 7fdcbbfff640 1 -- 172.21.15.55:0/1957334118 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+17122 (secure 0 0 0) 0x7fdcac0600c0 con 0x7fdcc40c6e30 2024-05-20T22:10:11.048 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:11.048 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":44,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:09:34.247518+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:09:12.734355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T22:09:13.936532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6835/4137968836":"2024-05-21T22:09:29.217850+0000","172.21.15.138:6833/4137968836":"2024-05-21T22:09:29.217850+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:10:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1957334118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdca406be60 msgr2=0x7fdca406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdca406be60 0x7fdca406e370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4004d00 tx=0x7fdcb4006040 comp rx=0 tx=0).stop 2024-05-20T22:10:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1957334118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc40c6e30 msgr2=0x7fdcc40c87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc40c6e30 0x7fdcc40c87e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0119d0 tx=0x7fdcac011e90 comp rx=0 tx=0).stop 2024-05-20T22:10:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1957334118 shutdown_connections 2024-05-20T22:10:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdca406be60 0x7fdca406e370 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc40c8d20 0x7fdcc415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc40c6e30 0x7fdcc40c87e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 --2- 172.21.15.55:0/1957334118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcb0004d40 0x7fdcc41560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1957334118 >> 172.21.15.55:0/1957334118 conn(0x7fdcc40d2910 msgr2=0x7fdcc40cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:11.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1957334118 shutdown_connections 2024-05-20T22:10:11.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.049+0000 7fdcc9dd5640 1 -- 172.21.15.55:0/1957334118 wait complete. 2024-05-20T22:10:11.062 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T22:10:11.062 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:11.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.175+0000 7fcdff9ce640 1 Processor -- start 2024-05-20T22:10:11.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.175+0000 7fcdff9ce640 1 -- start start 2024-05-20T22:10:11.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdff9ce640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcdf80d42a0 0x7fcdf80cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdff9ce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 0x7fcdf80d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdff9ce640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcdf80d3bb0 0x7fcdf80b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdff9ce640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf80d3fc0 con 0x7fcdf80d42a0 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdff9ce640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf8160550 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdff9ce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf815f9d0 con 0x7fcdf815ccc0 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdfd76a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcdf80d42a0 0x7fcdf80cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdfd76a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcdf80d42a0 0x7fcdf80cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44720/0 (socket says 172.21.15.55:44720) 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdfd76a640 1 -- 172.21.15.55:0/414909785 learned_addr learned my addr 172.21.15.55:0/414909785 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdfdf6b640 1 --2- 172.21.15.55:0/414909785 >> v2:172.21.15.138:3300/0 conn(0x7fcdf80d3bb0 0x7fcdf80b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdfcf69640 1 --2- 172.21.15.55:0/414909785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 0x7fcdf80d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdfd76a640 1 -- 172.21.15.55:0/414909785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 msgr2=0x7fcdf80d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdfd76a640 1 --2- 172.21.15.55:0/414909785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 0x7fcdf80d3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdfd76a640 1 -- 172.21.15.55:0/414909785 >> v2:172.21.15.138:3300/0 conn(0x7fcdf80d3bb0 msgr2=0x7fcdf80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:11.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdfd76a640 1 --2- 172.21.15.55:0/414909785 >> v2:172.21.15.138:3300/0 conn(0x7fcdf80d3bb0 0x7fcdf80b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.177+0000 7fcdfd76a640 1 -- 172.21.15.55:0/414909785 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf815fcb0 con 0x7fcdf80d42a0 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.178+0000 7fcdfd76a640 1 --2- 172.21.15.55:0/414909785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d42a0 0x7fcdf80cf710 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80c83e0 tx=0x7fcdf402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61200ddd7a9a8b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.178+0000 7fcdeffff640 1 -- 172.21.15.55:0/414909785 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4036070 con 0x7fcdf80d42a0 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.178+0000 7fcdeffff640 1 -- 172.21.15.55:0/414909785 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf4031600 con 0x7fcdf80d42a0 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.178+0000 7fcdeffff640 1 -- 172.21.15.55:0/414909785 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40319d0 con 0x7fcdf80d42a0 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.179+0000 7fcdff9ce640 1 -- 172.21.15.55:0/414909785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d42a0 msgr2=0x7fcdf80cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.179+0000 7fcdff9ce640 1 --2- 172.21.15.55:0/414909785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d42a0 0x7fcdf80cf710 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80c83e0 tx=0x7fcdf402a610 comp rx=0 tx=0).stop 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.179+0000 7fcdff9ce640 1 -- 172.21.15.55:0/414909785 shutdown_connections 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.179+0000 7fcdff9ce640 1 --2- 172.21.15.55:0/414909785 >> v2:172.21.15.138:3300/0 conn(0x7fcdf80d3bb0 0x7fcdf80b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.179+0000 7fcdff9ce640 1 --2- 172.21.15.55:0/414909785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 0x7fcdf80d3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.179+0000 7fcdff9ce640 1 --2- 172.21.15.55:0/414909785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d42a0 0x7fcdf80cf710 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.179+0000 7fcdff9ce640 1 -- 172.21.15.55:0/414909785 >> 172.21.15.55:0/414909785 conn(0x7fcdf80d1c50 msgr2=0x7fcdf80d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.179+0000 7fcdff9ce640 1 -- 172.21.15.55:0/414909785 shutdown_connections 2024-05-20T22:10:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.179+0000 7fcdff9ce640 1 -- 172.21.15.55:0/414909785 wait complete. 2024-05-20T22:10:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.181+0000 7fcdff9ce640 1 Processor -- start 2024-05-20T22:10:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.181+0000 7fcdff9ce640 1 -- start start 2024-05-20T22:10:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.182+0000 7fcdff9ce640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d3bb0 0x7fcdf81f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.182+0000 7fcdff9ce640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 0x7fcdf81f8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.182+0000 7fcdff9ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf814a810 0x7fcdf81f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.182+0000 7fcdff9ce640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf815f630 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.182+0000 7fcdff9ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf8160270 con 0x7fcdf814a810 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.182+0000 7fcdff9ce640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf815f7b0 con 0x7fcdf815ccc0 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.182+0000 7fcdfd76a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d3bb0 0x7fcdf81f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.182+0000 7fcdfd76a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d3bb0 0x7fcdf81f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44736/0 (socket says 172.21.15.55:44736) 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.182+0000 7fcdfd76a640 1 -- 172.21.15.55:0/522192941 learned_addr learned my addr 172.21.15.55:0/522192941 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdfdf6b640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf814a810 0x7fcdf81f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdfcf69640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 0x7fcdf81f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdfd76a640 1 -- 172.21.15.55:0/522192941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 msgr2=0x7fcdf81f8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdfd76a640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 0x7fcdf81f8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdfd76a640 1 -- 172.21.15.55:0/522192941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf814a810 msgr2=0x7fcdf81f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdfd76a640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf814a810 0x7fcdf81f8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdfd76a640 1 -- 172.21.15.55:0/522192941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf8160890 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdfcf69640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 0x7fcdf81f8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdfdf6b640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf814a810 0x7fcdf81f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdfd76a640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d3bb0 0x7fcdf81f48e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4034040 tx=0x7fcdf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdedffb640 1 -- 172.21.15.55:0/522192941 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf403f020 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdedffb640 1 -- 172.21.15.55:0/522192941 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf4037040 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdedffb640 1 -- 172.21.15.55:0/522192941 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4036070 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdff9ce640 1 -- 172.21.15.55:0/522192941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf8160b70 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.183+0000 7fcdff9ce640 1 -- 172.21.15.55:0/522192941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf8161130 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.184+0000 7fcdff9ce640 1 -- 172.21.15.55:0/522192941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf80d2f40 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.185+0000 7fcdedffb640 1 -- 172.21.15.55:0/522192941 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdf4042020 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.186+0000 7fcdedffb640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcddc06be60 0x7fcddc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.186+0000 7fcdedffb640 1 -- 172.21.15.55:0/522192941 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7fcdf40ab7a0 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.186+0000 7fcdfcf69640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcddc06be60 0x7fcddc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.186+0000 7fcdfcf69640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcddc06be60 0x7fcddc06e370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80b9700 tx=0x7fcde8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:11.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.190+0000 7fcdedffb640 1 -- 172.21.15.55:0/522192941 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcdf4049020 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.412+0000 7fcdff9ce640 1 -- 172.21.15.55:0/522192941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdf80b7430 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.413+0000 7fcdedffb640 1 -- 172.21.15.55:0/522192941 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5191 (secure 0 0 0) 0x7fcdf4079ec0 con 0x7fcdf80d3bb0 2024-05-20T22:10:11.414 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:11.414 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3873372908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3873372908},{"type":"v1","addr":"172.21.15.55:6836","nonce":3873372908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3417305427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":3417305427},{"type":"v1","addr":"172.21.15.55:6837","nonce":3417305427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":35081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4211704780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4211704780},{"type":"v1","addr":"172.21.15.138:6833","nonce":4211704780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:09:14.957632+0000","modified":"2024-05-20T22:09:33.242208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":35093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35093":{"gid":35093,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6835/270300569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":270300569},{"type":"v1","addr":"172.21.15.138:6835","nonce":270300569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":35093,"qdb_cluster":[35093]},"id":2}]} 2024-05-20T22:10:11.414 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2024-05-20T22:10:11.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 -- 172.21.15.55:0/522192941 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcddc06be60 msgr2=0x7fcddc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcddc06be60 0x7fcddc06e370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80b9700 tx=0x7fcde8008040 comp rx=0 tx=0).stop 2024-05-20T22:10:11.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 -- 172.21.15.55:0/522192941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d3bb0 msgr2=0x7fcdf81f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d3bb0 0x7fcdf81f48e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4034040 tx=0x7fcdf4004a40 comp rx=0 tx=0).stop 2024-05-20T22:10:11.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 -- 172.21.15.55:0/522192941 shutdown_connections 2024-05-20T22:10:11.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcddc06be60 0x7fcddc06e370 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf814a810 0x7fcdf81f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdf815ccc0 0x7fcdf81f8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 --2- 172.21.15.55:0/522192941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcdf80d3bb0 0x7fcdf81f48e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 -- 172.21.15.55:0/522192941 >> 172.21.15.55:0/522192941 conn(0x7fcdf80d1c50 msgr2=0x7fcdf815bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:11.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 -- 172.21.15.55:0/522192941 shutdown_connections 2024-05-20T22:10:11.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.415+0000 7fcdff9ce640 1 -- 172.21.15.55:0/522192941 wait complete. 2024-05-20T22:10:11.428 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:11.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.543+0000 7f3344937640 1 Processor -- start 2024-05-20T22:10:11.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.543+0000 7f3344937640 1 -- start start 2024-05-20T22:10:11.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f3344937640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3340160560 0x7f33400d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f3344937640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f33400ced80 0x7f334015b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f3344937640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33400b2e80 con 0x7f33400b1270 2024-05-20T22:10:11.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f3344937640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f3340160970 con 0x7f3340160560 2024-05-20T22:10:11.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f3344937640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f33400d1210 con 0x7f33400ced80 2024-05-20T22:10:11.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f333e59c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3340160560 0x7f33400d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f333dd9b640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f33400b1270 0x7f33400ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56420/0 (socket says 172.21.15.55:56420) 2024-05-20T22:10:11.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f333dd9b640 1 -- 172.21.15.55:0/1730988655 learned_addr learned my addr 172.21.15.55:0/1730988655 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:11.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f333e59c640 1 --2- 172.21.15.55:0/1730988655 >> v2:172.21.15.55:3300/0 conn(0x7f3340160560 0x7f33400d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44752/0 (socket says 172.21.15.55:44752) 2024-05-20T22:10:11.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f333e59c640 1 -- 172.21.15.55:0/1730988655 >> v2:172.21.15.138:3300/0 conn(0x7f33400ced80 msgr2=0x7f334015b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:11.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f333e59c640 1 --2- 172.21.15.55:0/1730988655 >> v2:172.21.15.138:3300/0 conn(0x7f33400ced80 0x7f334015b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f333e59c640 1 -- 172.21.15.55:0/1730988655 >> v1:172.21.15.55:6789/0 conn(0x7f33400b1270 legacy=0x7f33400ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f333d59a640 1 -- 172.21.15.55:0/1730988655 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703867618 0 0) 0x7f33400b2e80 con 0x7f33400b1270 2024-05-20T22:10:11.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.545+0000 7f333e59c640 1 -- 172.21.15.55:0/1730988655 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33400b3160 con 0x7f3340160560 2024-05-20T22:10:11.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.546+0000 7f333e59c640 1 --2- 172.21.15.55:0/1730988655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 0x7f33400d0cd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f334006e3f0 tx=0x7f333002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aea555ab16ba129 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:11.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.546+0000 7f333d59a640 1 -- 172.21.15.55:0/1730988655 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330037070 con 0x7f3340160560 2024-05-20T22:10:11.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.546+0000 7f333d59a640 1 -- 172.21.15.55:0/1730988655 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3330032440 con 0x7f3340160560 2024-05-20T22:10:11.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.546+0000 7f333d59a640 1 -- 172.21.15.55:0/1730988655 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330032810 con 0x7f3340160560 2024-05-20T22:10:11.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.547+0000 7f3344937640 1 -- 172.21.15.55:0/1730988655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 msgr2=0x7f33400d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.547+0000 7f3344937640 1 --2- 172.21.15.55:0/1730988655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 0x7f33400d0cd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f334006e3f0 tx=0x7f333002a1a0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.547+0000 7f3344937640 1 -- 172.21.15.55:0/1730988655 shutdown_connections 2024-05-20T22:10:11.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.547+0000 7f3344937640 1 --2- 172.21.15.55:0/1730988655 >> v2:172.21.15.138:3300/0 conn(0x7f33400ced80 0x7f334015b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.547+0000 7f3344937640 1 --2- 172.21.15.55:0/1730988655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 0x7f33400d0cd0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.547+0000 7f3344937640 1 -- 172.21.15.55:0/1730988655 >> 172.21.15.55:0/1730988655 conn(0x7f33400b65a0 msgr2=0x7f33400b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:11.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.547+0000 7f3344937640 1 -- 172.21.15.55:0/1730988655 shutdown_connections 2024-05-20T22:10:11.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.547+0000 7f3344937640 1 -- 172.21.15.55:0/1730988655 wait complete. 2024-05-20T22:10:11.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.549+0000 7f3344937640 1 Processor -- start 2024-05-20T22:10:11.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.549+0000 7f3344937640 1 -- start start 2024-05-20T22:10:11.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.550+0000 7f3344937640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33400b1270 0x7f33401f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.550+0000 7f3344937640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 0x7f334014be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.550+0000 7f3344937640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3340157e90 0x7f33401582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.550+0000 7f3344937640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f33400b1680 con 0x7f3340160560 2024-05-20T22:10:11.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.550+0000 7f3344937640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33400b3720 con 0x7f3340157e90 2024-05-20T22:10:11.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.550+0000 7f3344937640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f33400b1800 con 0x7f33400b1270 2024-05-20T22:10:11.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.550+0000 7f333dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 0x7f334014be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.550+0000 7f333dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 0x7f334014be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44760/0 (socket says 172.21.15.55:44760) 2024-05-20T22:10:11.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.550+0000 7f333dd9b640 1 -- 172.21.15.55:0/1763446995 learned_addr learned my addr 172.21.15.55:0/1763446995 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:11.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f333ed9d640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3340157e90 0x7f33401582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f333e59c640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33400b1270 0x7f33401f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f333dd9b640 1 -- 172.21.15.55:0/1763446995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33400b1270 msgr2=0x7f33401f3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f333dd9b640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33400b1270 0x7f33401f3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f333dd9b640 1 -- 172.21.15.55:0/1763446995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3340157e90 msgr2=0x7f33401582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f333dd9b640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3340157e90 0x7f33401582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f333dd9b640 1 -- 172.21.15.55:0/1763446995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334015b080 con 0x7f3340160560 2024-05-20T22:10:11.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f333e59c640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33400b1270 0x7f33401f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:11.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f333dd9b640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 0x7f334014be60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f333400c430 tx=0x7f333400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:11.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f332effd640 1 -- 172.21.15.55:0/1763446995 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340074e0 con 0x7f3340160560 2024-05-20T22:10:11.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f332effd640 1 -- 172.21.15.55:0/1763446995 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33340047e0 con 0x7f3340160560 2024-05-20T22:10:11.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f332effd640 1 -- 172.21.15.55:0/1763446995 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334004bd0 con 0x7f3340160560 2024-05-20T22:10:11.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f3344937640 1 -- 172.21.15.55:0/1763446995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33401f4090 con 0x7f3340160560 2024-05-20T22:10:11.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.551+0000 7f3344937640 1 -- 172.21.15.55:0/1763446995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3340159380 con 0x7f3340160560 2024-05-20T22:10:11.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.552+0000 7f3344937640 1 -- 172.21.15.55:0/1763446995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33401590f0 con 0x7f3340160560 2024-05-20T22:10:11.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.553+0000 7f332effd640 1 -- 172.21.15.55:0/1763446995 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3334011070 con 0x7f3340160560 2024-05-20T22:10:11.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.553+0000 7f332effd640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f332006be30 0x7f332006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.553+0000 7f332effd640 1 -- 172.21.15.55:0/1763446995 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f333408acd0 con 0x7f3340160560 2024-05-20T22:10:11.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.553+0000 7f333e59c640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f332006be30 0x7f332006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.554+0000 7f333e59c640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f332006be30 0x7f332006e340 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3330034000 tx=0x7f333002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:11.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.558+0000 7f332effd640 1 -- 172.21.15.55:0/1763446995 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f33401f4090 con 0x7f3340160560 2024-05-20T22:10:11.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.782+0000 7f3344937640 1 -- 172.21.15.55:0/1763446995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33401587e0 con 0x7f3340160560 2024-05-20T22:10:11.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.782+0000 7f332effd640 1 -- 172.21.15.55:0/1763446995 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5191 (secure 0 0 0) 0x7f33401587e0 con 0x7f3340160560 2024-05-20T22:10:11.783 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:11.784 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3873372908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3873372908},{"type":"v1","addr":"172.21.15.55:6836","nonce":3873372908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3417305427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":3417305427},{"type":"v1","addr":"172.21.15.55:6837","nonce":3417305427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":35081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4211704780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4211704780},{"type":"v1","addr":"172.21.15.138:6833","nonce":4211704780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:09:14.957632+0000","modified":"2024-05-20T22:09:33.242208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":35093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35093":{"gid":35093,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6835/270300569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":270300569},{"type":"v1","addr":"172.21.15.138:6835","nonce":270300569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":35093,"qdb_cluster":[35093]},"id":2}]} 2024-05-20T22:10:11.784 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2024-05-20T22:10:11.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 -- 172.21.15.55:0/1763446995 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f332006be30 msgr2=0x7f332006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f332006be30 0x7f332006e340 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3330034000 tx=0x7f333002e000 comp rx=0 tx=0).stop 2024-05-20T22:10:11.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 -- 172.21.15.55:0/1763446995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 msgr2=0x7f334014be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 0x7f334014be60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f333400c430 tx=0x7f333400c8f0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 -- 172.21.15.55:0/1763446995 shutdown_connections 2024-05-20T22:10:11.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f332006be30 0x7f332006e340 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3340157e90 0x7f33401582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3340160560 0x7f334014be60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 --2- 172.21.15.55:0/1763446995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33400b1270 0x7f33401f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 -- 172.21.15.55:0/1763446995 >> 172.21.15.55:0/1763446995 conn(0x7f33400b65a0 msgr2=0x7f334015f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:11.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 -- 172.21.15.55:0/1763446995 shutdown_connections 2024-05-20T22:10:11.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.785+0000 7f3344937640 1 -- 172.21.15.55:0/1763446995 wait complete. 2024-05-20T22:10:11.797 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:10:11.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.911+0000 7f74257aa640 1 Processor -- start 2024-05-20T22:10:11.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.911+0000 7f74257aa640 1 -- start start 2024-05-20T22:10:11.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f74257aa640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f74200d3d80 0x7f74200cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f74257aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f742015ccc0 0x7f74200b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f74257aa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7420160550 con 0x7f74200d42a0 2024-05-20T22:10:11.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f74257aa640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f742015f9d0 con 0x7f74200d3d80 2024-05-20T22:10:11.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f74257aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f742015fcb0 con 0x7f742015ccc0 2024-05-20T22:10:11.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f741e7fc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f74200d3d80 0x7f74200cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f741e7fc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f74200d3d80 0x7f74200cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44772/0 (socket says 172.21.15.55:44772) 2024-05-20T22:10:11.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f741e7fc640 1 -- 172.21.15.55:0/3736056139 learned_addr learned my addr 172.21.15.55:0/3736056139 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:11.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f741f7fe640 1 --2- 172.21.15.55:0/3736056139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f742015ccc0 0x7f74200b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f741e7fc640 1 -- 172.21.15.55:0/3736056139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f742015ccc0 msgr2=0x7f74200b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:11.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f741e7fc640 1 --2- 172.21.15.55:0/3736056139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f742015ccc0 0x7f74200b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f741e7fc640 1 -- 172.21.15.55:0/3736056139 >> v1:172.21.15.138:6789/0 conn(0x7f74200d42a0 legacy=0x7f74200d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f741e7fc640 1 -- 172.21.15.55:0/3736056139 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742015ff90 con 0x7f74200d3d80 2024-05-20T22:10:11.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.913+0000 7f741f7fe640 1 --2- 172.21.15.55:0/3736056139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f742015ccc0 0x7f74200b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:11.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.914+0000 7f741e7fc640 1 --2- 172.21.15.55:0/3736056139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f74200d3d80 0x7f74200cf710 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7414009870 tx=0x7f741402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68254cde311f853e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:11.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.914+0000 7f741dffb640 1 -- 172.21.15.55:0/3736056139 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414037070 con 0x7f74200d3d80 2024-05-20T22:10:11.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.914+0000 7f741dffb640 1 -- 172.21.15.55:0/3736056139 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7414032440 con 0x7f74200d3d80 2024-05-20T22:10:11.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.914+0000 7f741dffb640 1 -- 172.21.15.55:0/3736056139 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414032810 con 0x7f74200d3d80 2024-05-20T22:10:11.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.915+0000 7f74257aa640 1 -- 172.21.15.55:0/3736056139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f74200d3d80 msgr2=0x7f74200cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.915+0000 7f74257aa640 1 --2- 172.21.15.55:0/3736056139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f74200d3d80 0x7f74200cf710 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7414009870 tx=0x7f741402a1c0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.915+0000 7f74257aa640 1 -- 172.21.15.55:0/3736056139 shutdown_connections 2024-05-20T22:10:11.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.915+0000 7f74257aa640 1 --2- 172.21.15.55:0/3736056139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f742015ccc0 0x7f74200b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.915+0000 7f74257aa640 1 --2- 172.21.15.55:0/3736056139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f74200d3d80 0x7f74200cf710 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.915+0000 7f74257aa640 1 -- 172.21.15.55:0/3736056139 >> 172.21.15.55:0/3736056139 conn(0x7f74200d1c50 msgr2=0x7f74200d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:11.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.915+0000 7f74257aa640 1 -- 172.21.15.55:0/3736056139 shutdown_connections 2024-05-20T22:10:11.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.915+0000 7f74257aa640 1 -- 172.21.15.55:0/3736056139 wait complete. 2024-05-20T22:10:11.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.917+0000 7f74257aa640 1 Processor -- start 2024-05-20T22:10:11.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.917+0000 7f74257aa640 1 -- start start 2024-05-20T22:10:11.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f74257aa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f742015ccc0 0x7f74200ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f74257aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74200b5f90 0x7f7420156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f74257aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74200b4f20 0x7f74200b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f74257aa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f742015f7b0 con 0x7f742015ccc0 2024-05-20T22:10:11.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f74257aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f742015f630 con 0x7f74200b4f20 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f74257aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f74200b3aa0 con 0x7f74200b5f90 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f741effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f742015ccc0 0x7f74200ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f741effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f742015ccc0 0x7f74200ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44774/0 (socket says 172.21.15.55:44774) 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f741effd640 1 -- 172.21.15.55:0/770065286 learned_addr learned my addr 172.21.15.55:0/770065286 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f741f7fe640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74200b4f20 0x7f74200b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.918+0000 7f741e7fc640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74200b5f90 0x7f7420156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f741effd640 1 -- 172.21.15.55:0/770065286 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74200b5f90 msgr2=0x7f7420156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f741effd640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74200b5f90 0x7f7420156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f741effd640 1 -- 172.21.15.55:0/770065286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74200b4f20 msgr2=0x7f74200b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f741effd640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74200b4f20 0x7f74200b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f741effd640 1 -- 172.21.15.55:0/770065286 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74200b5870 con 0x7f742015ccc0 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f741effd640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f742015ccc0 0x7f74200ca7e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f741000c730 tx=0x7f741000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f74077fe640 1 -- 172.21.15.55:0/770065286 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410016020 con 0x7f742015ccc0 2024-05-20T22:10:11.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f74077fe640 1 -- 172.21.15.55:0/770065286 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7410006d50 con 0x7f742015ccc0 2024-05-20T22:10:11.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f74257aa640 1 -- 172.21.15.55:0/770065286 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7420154930 con 0x7f742015ccc0 2024-05-20T22:10:11.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f74077fe640 1 -- 172.21.15.55:0/770065286 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410003bb0 con 0x7f742015ccc0 2024-05-20T22:10:11.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.919+0000 7f74257aa640 1 -- 172.21.15.55:0/770065286 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74200cb3a0 con 0x7f742015ccc0 2024-05-20T22:10:11.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.920+0000 7f74257aa640 1 -- 172.21.15.55:0/770065286 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742004f720 con 0x7f742015ccc0 2024-05-20T22:10:11.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.921+0000 7f74077fe640 1 -- 172.21.15.55:0/770065286 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f741000a850 con 0x7f742015ccc0 2024-05-20T22:10:11.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.921+0000 7f74077fe640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f740c06be30 0x7f740c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:11.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.921+0000 7f74077fe640 1 -- 172.21.15.55:0/770065286 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f741008aca0 con 0x7f742015ccc0 2024-05-20T22:10:11.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.921+0000 7f741e7fc640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f740c06be30 0x7f740c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:11.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.922+0000 7f741e7fc640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f740c06be30 0x7f740c06e340 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7414034000 tx=0x7f741402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:11.926+0000 7f74077fe640 1 -- 172.21.15.55:0/770065286 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7410020020 con 0x7f742015ccc0 2024-05-20T22:10:12.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.110+0000 7f74257aa640 1 -- 172.21.15.55:0/770065286 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74200d2f40 con 0x7f742015ccc0 2024-05-20T22:10:12.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.112+0000 7f74077fe640 1 -- 172.21.15.55:0/770065286 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+17122 (secure 0 0 0) 0x7f74200b5870 con 0x7f742015ccc0 2024-05-20T22:10:12.113 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:12.113 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":44,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:09:34.247518+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:09:12.734355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T22:09:13.936532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1d","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6835/4137968836":"2024-05-21T22:09:29.217850+0000","172.21.15.138:6833/4137968836":"2024-05-21T22:09:29.217850+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:10:12.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 -- 172.21.15.55:0/770065286 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f740c06be30 msgr2=0x7f740c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f740c06be30 0x7f740c06e340 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7414034000 tx=0x7f741402e000 comp rx=0 tx=0).stop 2024-05-20T22:10:12.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 -- 172.21.15.55:0/770065286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f742015ccc0 msgr2=0x7f74200ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f742015ccc0 0x7f74200ca7e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f741000c730 tx=0x7f741000cbf0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 -- 172.21.15.55:0/770065286 shutdown_connections 2024-05-20T22:10:12.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f740c06be30 0x7f740c06e340 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74200b4f20 0x7f74200b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74200b5f90 0x7f7420156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 --2- 172.21.15.55:0/770065286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f742015ccc0 0x7f74200ca7e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 -- 172.21.15.55:0/770065286 >> 172.21.15.55:0/770065286 conn(0x7f74200d1c50 msgr2=0x7f74200d3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:12.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 -- 172.21.15.55:0/770065286 shutdown_connections 2024-05-20T22:10:12.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.114+0000 7f74257aa640 1 -- 172.21.15.55:0/770065286 wait complete. 2024-05-20T22:10:12.127 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T22:10:12.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.242+0000 7f59cf9d5640 1 Processor -- start 2024-05-20T22:10:12.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.242+0000 7f59cf9d5640 1 -- start start 2024-05-20T22:10:12.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.243+0000 7f59cf9d5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f59c80b2c00 0x7f59c80b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.243+0000 7f59cf9d5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f59c80cffd0 0x7f59c80d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.243+0000 7f59cf9d5640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c80cb840 con 0x7f59c815f420 2024-05-20T22:10:12.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.243+0000 7f59cf9d5640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f59c815f830 con 0x7f59c80b2c00 2024-05-20T22:10:12.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.243+0000 7f59cf9d5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f59c80cb5f0 con 0x7f59c80cffd0 2024-05-20T22:10:12.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.244+0000 7f59ccf70640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f59c80b2c00 0x7f59c80b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.244+0000 7f59cd771640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f59c815f420 0x7f59c80ade70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56422/0 (socket says 172.21.15.55:56422) 2024-05-20T22:10:12.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.244+0000 7f59cd771640 1 -- 172.21.15.55:0/3496992651 learned_addr learned my addr 172.21.15.55:0/3496992651 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:12.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.244+0000 7f59cdf72640 1 --2- 172.21.15.55:0/3496992651 >> v2:172.21.15.138:3300/0 conn(0x7f59c80cffd0 0x7f59c80d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.244+0000 7f59ccf70640 1 -- 172.21.15.55:0/3496992651 >> v2:172.21.15.138:3300/0 conn(0x7f59c80cffd0 msgr2=0x7f59c80d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:12.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.244+0000 7f59ccf70640 1 --2- 172.21.15.55:0/3496992651 >> v2:172.21.15.138:3300/0 conn(0x7f59c80cffd0 0x7f59c80d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.244+0000 7f59ccf70640 1 -- 172.21.15.55:0/3496992651 >> v1:172.21.15.55:6789/0 conn(0x7f59c815f420 legacy=0x7f59c80ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.244+0000 7f59ccf70640 1 -- 172.21.15.55:0/3496992651 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c80cbb20 con 0x7f59c80b2c00 2024-05-20T22:10:12.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.244+0000 7f59bffff640 1 -- 172.21.15.55:0/3496992651 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 510852921 0 0) 0x7f59c80cb840 con 0x7f59c815f420 2024-05-20T22:10:12.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.245+0000 7f59ccf70640 1 --2- 172.21.15.55:0/3496992651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c80b2c00 0x7f59c80b3010 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f59c806eb10 tx=0x7f59b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7791b24c9e5c7281 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:12.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.245+0000 7f59bffff640 1 -- 172.21.15.55:0/3496992651 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8037070 con 0x7f59c80b2c00 2024-05-20T22:10:12.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.245+0000 7f59bffff640 1 -- 172.21.15.55:0/3496992651 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b8032440 con 0x7f59c80b2c00 2024-05-20T22:10:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.245+0000 7f59bffff640 1 -- 172.21.15.55:0/3496992651 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8032810 con 0x7f59c80b2c00 2024-05-20T22:10:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.245+0000 7f59cf9d5640 1 -- 172.21.15.55:0/3496992651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c80b2c00 msgr2=0x7f59c80b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.245+0000 7f59cf9d5640 1 --2- 172.21.15.55:0/3496992651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c80b2c00 0x7f59c80b3010 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f59c806eb10 tx=0x7f59b802a230 comp rx=0 tx=0).stop 2024-05-20T22:10:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.246+0000 7f59cf9d5640 1 -- 172.21.15.55:0/3496992651 shutdown_connections 2024-05-20T22:10:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.246+0000 7f59cf9d5640 1 --2- 172.21.15.55:0/3496992651 >> v2:172.21.15.138:3300/0 conn(0x7f59c80cffd0 0x7f59c80d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.246+0000 7f59cf9d5640 1 --2- 172.21.15.55:0/3496992651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c80b2c00 0x7f59c80b3010 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.246+0000 7f59cf9d5640 1 -- 172.21.15.55:0/3496992651 >> 172.21.15.55:0/3496992651 conn(0x7f59c809a400 msgr2=0x7f59c80b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.246+0000 7f59cf9d5640 1 -- 172.21.15.55:0/3496992651 shutdown_connections 2024-05-20T22:10:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.246+0000 7f59cf9d5640 1 -- 172.21.15.55:0/3496992651 wait complete. 2024-05-20T22:10:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.247+0000 7f59cf9d5640 1 Processor -- start 2024-05-20T22:10:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.247+0000 7f59cf9d5640 1 -- start start 2024-05-20T22:10:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cf9d5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59c80b2c00 0x7f59c81534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cf9d5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c80cffd0 0x7f59c80c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cf9d5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c815f420 0x7f59c80ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cf9d5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80d0920 con 0x7f59c815f420 2024-05-20T22:10:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cf9d5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80cc0e0 con 0x7f59c80cffd0 2024-05-20T22:10:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cf9d5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f59c80cd5d0 con 0x7f59c80b2c00 2024-05-20T22:10:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cdf72640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c815f420 0x7f59c80ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cdf72640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c815f420 0x7f59c80ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44804/0 (socket says 172.21.15.55:44804) 2024-05-20T22:10:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cdf72640 1 -- 172.21.15.55:0/1085987681 learned_addr learned my addr 172.21.15.55:0/1085987681 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cd771640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59c80b2c00 0x7f59c81534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59ccf70640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c80cffd0 0x7f59c80c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cdf72640 1 -- 172.21.15.55:0/1085987681 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59c80b2c00 msgr2=0x7f59c81534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cdf72640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59c80b2c00 0x7f59c81534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cdf72640 1 -- 172.21.15.55:0/1085987681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c80cffd0 msgr2=0x7f59c80c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cdf72640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c80cffd0 0x7f59c80c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cdf72640 1 -- 172.21.15.55:0/1085987681 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c80c5920 con 0x7f59c815f420 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cdf72640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c815f420 0x7f59c80ca050 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f59c0007d70 tx=0x7f59c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59bdffb640 1 -- 172.21.15.55:0/1085987681 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0014420 con 0x7f59c815f420 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59bdffb640 1 -- 172.21.15.55:0/1085987681 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c0019070 con 0x7f59c815f420 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.249+0000 7f59cf9d5640 1 -- 172.21.15.55:0/1085987681 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c80c8fb0 con 0x7f59c815f420 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.250+0000 7f59bdffb640 1 -- 172.21.15.55:0/1085987681 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0014820 con 0x7f59c815f420 2024-05-20T22:10:12.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.250+0000 7f59cf9d5640 1 -- 172.21.15.55:0/1085987681 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c80c9570 con 0x7f59c815f420 2024-05-20T22:10:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.251+0000 7f59cf9d5640 1 -- 172.21.15.55:0/1085987681 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c80c5ba0 con 0x7f59c815f420 2024-05-20T22:10:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.251+0000 7f59bdffb640 1 -- 172.21.15.55:0/1085987681 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59c00149c0 con 0x7f59c815f420 2024-05-20T22:10:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.252+0000 7f59bdffb640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f59ac06be30 0x7f59ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.252+0000 7f59bdffb640 1 -- 172.21.15.55:0/1085987681 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6216+0+0 (secure 0 0 0) 0x7f59c008b7b0 con 0x7f59c815f420 2024-05-20T22:10:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.252+0000 7f59cd771640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f59ac06be30 0x7f59ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.252+0000 7f59cd771640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f59ac06be30 0x7f59ac06e340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f59c4000f60 tx=0x7f59c4007c00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:12.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.256+0000 7f59bdffb640 1 -- 172.21.15.55:0/1085987681 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f59c0059ff0 con 0x7f59c815f420 2024-05-20T22:10:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.480+0000 7f59cf9d5640 1 -- 172.21.15.55:0/1085987681 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f59c804f720 con 0x7f59c815f420 2024-05-20T22:10:12.481 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-20T22:10:12.481+0000 7f0b3ef60640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T22:10:12.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.487+0000 7f59bdffb640 1 -- 172.21.15.55:0/1085987681 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f59c00598f0 con 0x7f59c815f420 2024-05-20T22:10:12.487 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T22:10:12.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 -- 172.21.15.55:0/1085987681 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f59ac06be30 msgr2=0x7f59ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f59ac06be30 0x7f59ac06e340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f59c4000f60 tx=0x7f59c4007c00 comp rx=0 tx=0).stop 2024-05-20T22:10:12.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 -- 172.21.15.55:0/1085987681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c815f420 msgr2=0x7f59c80ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c815f420 0x7f59c80ca050 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f59c0007d70 tx=0x7f59c000e4b0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 -- 172.21.15.55:0/1085987681 shutdown_connections 2024-05-20T22:10:12.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f59ac06be30 0x7f59ac06e340 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59c815f420 0x7f59c80ca050 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c80cffd0 0x7f59c80c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 --2- 172.21.15.55:0/1085987681 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59c80b2c00 0x7f59c81534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 -- 172.21.15.55:0/1085987681 >> 172.21.15.55:0/1085987681 conn(0x7f59c809a400 msgr2=0x7f59c806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:12.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 -- 172.21.15.55:0/1085987681 shutdown_connections 2024-05-20T22:10:12.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.489+0000 7f59cf9d5640 1 -- 172.21.15.55:0/1085987681 wait complete. 2024-05-20T22:10:12.503 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T22:10:12.558 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-05-20T22:10:12.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.621+0000 7fa44b23d640 1 Processor -- start 2024-05-20T22:10:12.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.621+0000 7fa44b23d640 1 -- start start 2024-05-20T22:10:12.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.622+0000 7fa44b23d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa4440cf7c0 0x7fa4440cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.622+0000 7fa44b23d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4440d0110 con 0x7fa4440d03d0 2024-05-20T22:10:12.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.622+0000 7fa44b23d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4440d38b0 con 0x7fa44415e760 2024-05-20T22:10:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.622+0000 7fa44b23d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa4440d44c0 con 0x7fa4440cf7c0 2024-05-20T22:10:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa448fd9640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa4440d03d0 0x7fa4440b3e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56430/0 (socket says 172.21.15.55:56430) 2024-05-20T22:10:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa448fd9640 1 -- 172.21.15.55:0/3772776150 learned_addr learned my addr 172.21.15.55:0/3772776150 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3900819977 0 0) 0x7fa438001110 con 0x7fa4440d03d0 2024-05-20T22:10:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa434003520 con 0x7fa4440d03d0 2024-05-20T22:10:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa443fff640 1 --2- 172.21.15.55:0/3772776150 >> v2:172.21.15.138:3300/0 conn(0x7fa4440cf7c0 0x7fa4440cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 155333135 0 0) 0x7fa434003520 con 0x7fa4440d03d0 2024-05-20T22:10:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 >> v2:172.21.15.138:3300/0 conn(0x7fa4440cf7c0 msgr2=0x7fa4440cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 --2- 172.21.15.55:0/3772776150 >> v2:172.21.15.138:3300/0 conn(0x7fa4440cf7c0 0x7fa4440cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 >> v1:172.21.15.138:6789/0 conn(0x7fa44415e760 legacy=0x7fa44415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4440d3b90 con 0x7fa4440d03d0 2024-05-20T22:10:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fa438001840 con 0x7fa4440d03d0 2024-05-20T22:10:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 >> v1:172.21.15.55:6789/0 conn(0x7fa4440d03d0 legacy=0x7fa4440b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4340039b0 0x7fa434003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa434004800 0x7fa434004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa43400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa4440d44c0 con 0x7fa4340051c0 2024-05-20T22:10:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa43400c160 con 0x7fa4340039b0 2024-05-20T22:10:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa43400c360 con 0x7fa434004800 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa4497da640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa43400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa448fd9640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4340039b0 0x7fa434003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.623+0000 7fa443fff640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa434004800 0x7fa434004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.624+0000 7fa4497da640 1 -- 172.21.15.55:0/3772776150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa434004800 msgr2=0x7fa434004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.624+0000 7fa4497da640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa434004800 0x7fa434004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.624+0000 7fa4497da640 1 -- 172.21.15.55:0/3772776150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4340039b0 msgr2=0x7fa434003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.624+0000 7fa4497da640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4340039b0 0x7fa434003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.624+0000 7fa4497da640 1 -- 172.21.15.55:0/3772776150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa434003520 con 0x7fa4340051c0 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.624+0000 7fa448fd9640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4340039b0 0x7fa434003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.624+0000 7fa4497da640 1 -- 172.21.15.55:0/3772776150 reap_dead start 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.624+0000 7fa4497da640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa43400b9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa43c001e50 tx=0x7fa434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31df996d5626189c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.625+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43800c0e0 con 0x7fa4340051c0 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.625+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c02adc0 con 0x7fa4340051c0 2024-05-20T22:10:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.625+0000 7fa4437fe640 1 -- 172.21.15.55:0/3772776150 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c001a70 con 0x7fa4340051c0 2024-05-20T22:10:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.625+0000 7fa44b23d640 1 -- 172.21.15.55:0/3772776150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 msgr2=0x7fa43400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.625+0000 7fa44b23d640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa43400b9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa43c001e50 tx=0x7fa434005830 comp rx=0 tx=0).stop 2024-05-20T22:10:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.625+0000 7fa44b23d640 1 -- 172.21.15.55:0/3772776150 shutdown_connections 2024-05-20T22:10:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.625+0000 7fa44b23d640 1 --2- 172.21.15.55:0/3772776150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa43400b9c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.625+0000 7fa44b23d640 1 -- 172.21.15.55:0/3772776150 >> 172.21.15.55:0/3772776150 conn(0x7fa4440d1c20 msgr2=0x7fa4440d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.625+0000 7fa44b23d640 1 -- 172.21.15.55:0/3772776150 shutdown_connections 2024-05-20T22:10:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.625+0000 7fa44b23d640 1 -- 172.21.15.55:0/3772776150 wait complete. 2024-05-20T22:10:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.627+0000 7fa44b23d640 1 Processor -- start 2024-05-20T22:10:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.627+0000 7fa44b23d640 1 -- start start 2024-05-20T22:10:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa44b23d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa4441f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa44b23d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa444150fe0 0x7fa4441e9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa44b23d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4441e9c40 0x7fa4440cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa44b23d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa4440d3440 con 0x7fa4340051c0 2024-05-20T22:10:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa44b23d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4440d4150 con 0x7fa4441e9c40 2024-05-20T22:10:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa44b23d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa4440d35c0 con 0x7fa444150fe0 2024-05-20T22:10:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa448fd9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa4441f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa448fd9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa4441f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44818/0 (socket says 172.21.15.55:44818) 2024-05-20T22:10:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa448fd9640 1 -- 172.21.15.55:0/4186691489 learned_addr learned my addr 172.21.15.55:0/4186691489 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa443fff640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa444150fe0 0x7fa4441e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa448fd9640 1 -- 172.21.15.55:0/4186691489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa444150fe0 msgr2=0x7fa4441e9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa448fd9640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa444150fe0 0x7fa4441e9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa448fd9640 1 -- 172.21.15.55:0/4186691489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4441e9c40 msgr2=0x7fa4440cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa4497da640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4441e9c40 0x7fa4440cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa448fd9640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4441e9c40 0x7fa4440cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa448fd9640 1 -- 172.21.15.55:0/4186691489 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4440cbbc0 con 0x7fa4340051c0 2024-05-20T22:10:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa443fff640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa444150fe0 0x7fa4441e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa448fd9640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa4441f4ff0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa438007290 tx=0x7fa438007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.629+0000 7fa4417fa640 1 -- 172.21.15.55:0/4186691489 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43800f940 con 0x7fa4340051c0 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.630+0000 7fa4417fa640 1 -- 172.21.15.55:0/4186691489 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa438018070 con 0x7fa4340051c0 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.630+0000 7fa44b23d640 1 -- 172.21.15.55:0/4186691489 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4441ea280 con 0x7fa4340051c0 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.630+0000 7fa4417fa640 1 -- 172.21.15.55:0/4186691489 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43800fce0 con 0x7fa4340051c0 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.630+0000 7fa44b23d640 1 -- 172.21.15.55:0/4186691489 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4440cbe80 con 0x7fa4340051c0 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.631+0000 7fa44b23d640 1 -- 172.21.15.55:0/4186691489 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44404f720 con 0x7fa4340051c0 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.631+0000 7fa4417fa640 1 -- 172.21.15.55:0/4186691489 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa43801c410 con 0x7fa4340051c0 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.632+0000 7fa4417fa640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa42806be60 0x7fa42806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.632+0000 7fa4417fa640 1 -- 172.21.15.55:0/4186691489 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6302+0+0 (secure 0 0 0) 0x7fa43808ad00 con 0x7fa4340051c0 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.632+0000 7fa443fff640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa42806be60 0x7fa42806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.632+0000 7fa443fff640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa42806be60 0x7fa42806e370 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa4300093d0 tx=0x7fa430008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.636+0000 7fa4417fa640 1 -- 172.21.15.55:0/4186691489 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa4440d0110 con 0x7fa4340051c0 2024-05-20T22:10:12.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:12.859+0000 7fa44b23d640 1 -- 172.21.15.55:0/4186691489 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa4440d2f60 con 0x7fa4340051c0 2024-05-20T22:10:13.446 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:10:13.448 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:10:13.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.523+0000 7fa4417fa640 1 -- 172.21.15.55:0/4186691489 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fa438059490 con 0x7fa4340051c0 2024-05-20T22:10:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.525+0000 7fa44b23d640 1 -- 172.21.15.55:0/4186691489 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa42806be60 msgr2=0x7fa42806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.525+0000 7fa44b23d640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa42806be60 0x7fa42806e370 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa4300093d0 tx=0x7fa430008040 comp rx=0 tx=0).stop 2024-05-20T22:10:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.526+0000 7fa44b23d640 1 -- 172.21.15.55:0/4186691489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 msgr2=0x7fa4441f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.526+0000 7fa44b23d640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa4441f4ff0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa438007290 tx=0x7fa438007750 comp rx=0 tx=0).stop 2024-05-20T22:10:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.526+0000 7fa44b23d640 1 -- 172.21.15.55:0/4186691489 shutdown_connections 2024-05-20T22:10:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.526+0000 7fa44b23d640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa42806be60 0x7fa42806e370 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.526+0000 7fa44b23d640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4441e9c40 0x7fa4440cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.526+0000 7fa44b23d640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa444150fe0 0x7fa4441e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.526+0000 7fa44b23d640 1 --2- 172.21.15.55:0/4186691489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4340051c0 0x7fa4441f4ff0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.526+0000 7fa44b23d640 1 -- 172.21.15.55:0/4186691489 >> 172.21.15.55:0/4186691489 conn(0x7fa4440d1c20 msgr2=0x7fa4440b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.526+0000 7fa44b23d640 1 -- 172.21.15.55:0/4186691489 shutdown_connections 2024-05-20T22:10:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.526+0000 7fa44b23d640 1 -- 172.21.15.55:0/4186691489 wait complete. 2024-05-20T22:10:13.541 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T22:10:13.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.711+0000 7f05a55c7640 1 Processor -- start 2024-05-20T22:10:13.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.711+0000 7f05a55c7640 1 -- start start 2024-05-20T22:10:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.712+0000 7f05a55c7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a006e320 0x7f05a0159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.712+0000 7f05a55c7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f05a00d1690 0x7f05a00d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.712+0000 7f05a55c7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f05a00ce2e0 0x7f05a00ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.712+0000 7f05a55c7640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f05a00b31c0 con 0x7f05a00d1690 2024-05-20T22:10:13.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.712+0000 7f05a55c7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f05a0159fa0 con 0x7f05a00ce2e0 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.712+0000 7f05a55c7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05a015d4a0 con 0x7f05a006e320 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.712+0000 7f0597fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f05a00d1690 0x7f05a00d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f0597fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f05a00d1690 0x7f05a00d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44834/0 (socket says 172.21.15.55:44834) 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f0597fff640 1 -- 172.21.15.55:0/3458740218 learned_addr learned my addr 172.21.15.55:0/3458740218 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f059effd640 1 --2- 172.21.15.55:0/3458740218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a006e320 0x7f05a0159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f059f7fe640 1 --2- 172.21.15.55:0/3458740218 >> v2:172.21.15.138:3300/0 conn(0x7f05a00ce2e0 0x7f05a00ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f0597fff640 1 -- 172.21.15.55:0/3458740218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a006e320 msgr2=0x7f05a0159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f0597fff640 1 --2- 172.21.15.55:0/3458740218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a006e320 0x7f05a0159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f0597fff640 1 -- 172.21.15.55:0/3458740218 >> v2:172.21.15.138:3300/0 conn(0x7f05a00ce2e0 msgr2=0x7f05a00ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f0597fff640 1 --2- 172.21.15.55:0/3458740218 >> v2:172.21.15.138:3300/0 conn(0x7f05a00ce2e0 0x7f05a00ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f0597fff640 1 -- 172.21.15.55:0/3458740218 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a006cf80 con 0x7f05a00d1690 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f059effd640 1 --2- 172.21.15.55:0/3458740218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a006e320 0x7f05a0159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.713+0000 7f059f7fe640 1 --2- 172.21.15.55:0/3458740218 >> v2:172.21.15.138:3300/0 conn(0x7f05a00ce2e0 0x7f05a00ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:13.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.714+0000 7f0597fff640 1 --2- 172.21.15.55:0/3458740218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 0x7f05a00d1aa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f05a0071e30 tx=0x7f058802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6ed7b8c46ec9d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.714+0000 7f059e7fc640 1 -- 172.21.15.55:0/3458740218 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588037070 con 0x7f05a00d1690 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.714+0000 7f059e7fc640 1 -- 172.21.15.55:0/3458740218 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0588032440 con 0x7f05a00d1690 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.714+0000 7f059e7fc640 1 -- 172.21.15.55:0/3458740218 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588032810 con 0x7f05a00d1690 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.715+0000 7f05a55c7640 1 -- 172.21.15.55:0/3458740218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 msgr2=0x7f05a00d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.715+0000 7f05a55c7640 1 --2- 172.21.15.55:0/3458740218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 0x7f05a00d1aa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f05a0071e30 tx=0x7f058802a140 comp rx=0 tx=0).stop 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.715+0000 7f05a55c7640 1 -- 172.21.15.55:0/3458740218 shutdown_connections 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.715+0000 7f05a55c7640 1 --2- 172.21.15.55:0/3458740218 >> v2:172.21.15.138:3300/0 conn(0x7f05a00ce2e0 0x7f05a00ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.715+0000 7f05a55c7640 1 --2- 172.21.15.55:0/3458740218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 0x7f05a00d1aa0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.715+0000 7f05a55c7640 1 --2- 172.21.15.55:0/3458740218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a006e320 0x7f05a0159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.715+0000 7f05a55c7640 1 -- 172.21.15.55:0/3458740218 >> 172.21.15.55:0/3458740218 conn(0x7f05a00b65c0 msgr2=0x7f05a00b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:13.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.715+0000 7f05a55c7640 1 -- 172.21.15.55:0/3458740218 shutdown_connections 2024-05-20T22:10:13.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.715+0000 7f05a55c7640 1 -- 172.21.15.55:0/3458740218 wait complete. 2024-05-20T22:10:13.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.716+0000 7f05a55c7640 1 Processor -- start 2024-05-20T22:10:13.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.717+0000 7f05a55c7640 1 -- start start 2024-05-20T22:10:13.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f05a55c7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05a006e320 0x7f05a00cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:13.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f05a55c7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00ce2e0 0x7f05a009e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:13.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f05a55c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 0x7f05a01e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f05a55c7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f05a00d1fe0 con 0x7f05a00d1690 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f05a55c7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05a006d540 con 0x7f05a006e320 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f05a55c7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05a006e730 con 0x7f05a00ce2e0 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 0x7f05a01e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 0x7f05a01e8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44850/0 (socket says 172.21.15.55:44850) 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f059f7fe640 1 -- 172.21.15.55:0/2742717866 learned_addr learned my addr 172.21.15.55:0/2742717866 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f059effd640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05a006e320 0x7f05a00cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f0597fff640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00ce2e0 0x7f05a009e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f059f7fe640 1 -- 172.21.15.55:0/2742717866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00ce2e0 msgr2=0x7f05a009e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f059f7fe640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00ce2e0 0x7f05a009e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f059f7fe640 1 -- 172.21.15.55:0/2742717866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05a006e320 msgr2=0x7f05a00cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:13.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f059f7fe640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05a006e320 0x7f05a00cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:13.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f059f7fe640 1 -- 172.21.15.55:0/2742717866 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a009f730 con 0x7f05a00d1690 2024-05-20T22:10:13.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.718+0000 7f0597fff640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00ce2e0 0x7f05a009e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:13.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.719+0000 7f059f7fe640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 0x7f05a01e8aa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0598007e20 tx=0x7f059800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:13.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.719+0000 7f05977fe640 1 -- 172.21.15.55:0/2742717866 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059801a070 con 0x7f05a00d1690 2024-05-20T22:10:13.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.719+0000 7f05977fe640 1 -- 172.21.15.55:0/2742717866 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0598003b90 con 0x7f05a00d1690 2024-05-20T22:10:13.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.719+0000 7f05a55c7640 1 -- 172.21.15.55:0/2742717866 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a00cdbd0 con 0x7f05a00d1690 2024-05-20T22:10:13.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.719+0000 7f05977fe640 1 -- 172.21.15.55:0/2742717866 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059800de80 con 0x7f05a00d1690 2024-05-20T22:10:13.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.719+0000 7f05a55c7640 1 -- 172.21.15.55:0/2742717866 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a009f450 con 0x7f05a00d1690 2024-05-20T22:10:13.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.720+0000 7f05a55c7640 1 -- 172.21.15.55:0/2742717866 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a009eee0 con 0x7f05a00d1690 2024-05-20T22:10:13.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.720+0000 7f05977fe640 1 -- 172.21.15.55:0/2742717866 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f059801d070 con 0x7f05a00d1690 2024-05-20T22:10:13.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.721+0000 7f05977fe640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058006be30 0x7f058006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:13.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.721+0000 7f05977fe640 1 -- 172.21.15.55:0/2742717866 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6302+0+0 (secure 0 0 0) 0x7f059808b280 con 0x7f05a00d1690 2024-05-20T22:10:13.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.721+0000 7f059effd640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058006be30 0x7f058006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:13.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.721+0000 7f059effd640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058006be30 0x7f058006e340 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f05a015d700 tx=0x7f0590004730 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:13.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.726+0000 7f05977fe640 1 -- 172.21.15.55:0/2742717866 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0598010070 con 0x7f05a00d1690 2024-05-20T22:10:13.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:13.912+0000 7f05a55c7640 1 -- 172.21.15.55:0/2742717866 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f05a004f800 con 0x7f05a00d1690 2024-05-20T22:10:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.538+0000 7f05977fe640 1 -- 172.21.15.55:0/2742717866 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) ==== 181+0+0 (secure 0 0 0) 0x7f05980593a0 con 0x7f05a00d1690 2024-05-20T22:10:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-05-20T22:10:14.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.542+0000 7f05a55c7640 1 -- 172.21.15.55:0/2742717866 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058006be30 msgr2=0x7f058006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.542+0000 7f05a55c7640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058006be30 0x7f058006e340 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f05a015d700 tx=0x7f0590004730 comp rx=0 tx=0).stop 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.542+0000 7f05a55c7640 1 -- 172.21.15.55:0/2742717866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 msgr2=0x7f05a01e8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.542+0000 7f05a55c7640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 0x7f05a01e8aa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0598007e20 tx=0x7f059800d4a0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.542+0000 7f05a55c7640 1 -- 172.21.15.55:0/2742717866 shutdown_connections 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.542+0000 7f05a55c7640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058006be30 0x7f058006e340 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.543+0000 7f05a55c7640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a00d1690 0x7f05a01e8aa0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.543+0000 7f05a55c7640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00ce2e0 0x7f05a009e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.543+0000 7f05a55c7640 1 --2- 172.21.15.55:0/2742717866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05a006e320 0x7f05a00cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.543+0000 7f05a55c7640 1 -- 172.21.15.55:0/2742717866 >> 172.21.15.55:0/2742717866 conn(0x7f05a00b65c0 msgr2=0x7f05a00b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.543+0000 7f05a55c7640 1 -- 172.21.15.55:0/2742717866 shutdown_connections 2024-05-20T22:10:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.543+0000 7f05a55c7640 1 -- 172.21.15.55:0/2742717866 wait complete. 2024-05-20T22:10:14.559 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T22:10:14.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.682+0000 7f48f7537640 1 Processor -- start 2024-05-20T22:10:14.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.682+0000 7f48f7537640 1 -- start start 2024-05-20T22:10:14.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.683+0000 7f48f7537640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f0161590 0x7f48f0072690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:14.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.683+0000 7f48f7537640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48f00b4b40 0x7f48f015d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:14.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.683+0000 7f48f7537640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f015e010 con 0x7f48f015d550 2024-05-20T22:10:14.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.683+0000 7f48f7537640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f48f0157a00 con 0x7f48f00b4b40 2024-05-20T22:10:14.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.683+0000 7f48f7537640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48f0160b40 con 0x7f48f0161590 2024-05-20T22:10:14.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.683+0000 7f48f5ad4640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48f00b4b40 0x7f48f015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:14.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.683+0000 7f48f4ad2640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f48f015d550 0x7f48f00b4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56438/0 (socket says 172.21.15.55:56438) 2024-05-20T22:10:14.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.683+0000 7f48f4ad2640 1 -- 172.21.15.55:0/3728888235 learned_addr learned my addr 172.21.15.55:0/3728888235 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:14.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.684+0000 7f48f52d3640 1 --2- 172.21.15.55:0/3728888235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f0161590 0x7f48f0072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:14.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.684+0000 7f48f5ad4640 1 -- 172.21.15.55:0/3728888235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f0161590 msgr2=0x7f48f0072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:14.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.684+0000 7f48f5ad4640 1 --2- 172.21.15.55:0/3728888235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f0161590 0x7f48f0072690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.684+0000 7f48f5ad4640 1 -- 172.21.15.55:0/3728888235 >> v1:172.21.15.55:6789/0 conn(0x7f48f015d550 legacy=0x7f48f00b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:14.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.684+0000 7f48effff640 1 -- 172.21.15.55:0/3728888235 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3684070157 0 0) 0x7f48f015e010 con 0x7f48f015d550 2024-05-20T22:10:14.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.684+0000 7f48f5ad4640 1 -- 172.21.15.55:0/3728888235 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f015e2f0 con 0x7f48f00b4b40 2024-05-20T22:10:14.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.684+0000 7f48f5ad4640 1 --2- 172.21.15.55:0/3728888235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f00b4b40 0x7f48f015d960 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f48e80097a0 tx=0x7f48e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f719c2b99c3c5d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:14.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.685+0000 7f48effff640 1 -- 172.21.15.55:0/3728888235 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8037070 con 0x7f48f00b4b40 2024-05-20T22:10:14.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.685+0000 7f48effff640 1 -- 172.21.15.55:0/3728888235 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e8032440 con 0x7f48f00b4b40 2024-05-20T22:10:14.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.685+0000 7f48effff640 1 -- 172.21.15.55:0/3728888235 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8032810 con 0x7f48f00b4b40 2024-05-20T22:10:14.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.685+0000 7f48f7537640 1 -- 172.21.15.55:0/3728888235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f00b4b40 msgr2=0x7f48f015d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:14.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.685+0000 7f48f7537640 1 --2- 172.21.15.55:0/3728888235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f00b4b40 0x7f48f015d960 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f48e80097a0 tx=0x7f48e802a1a0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.685+0000 7f48f7537640 1 -- 172.21.15.55:0/3728888235 shutdown_connections 2024-05-20T22:10:14.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.685+0000 7f48f7537640 1 --2- 172.21.15.55:0/3728888235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f00b4b40 0x7f48f015d960 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.685+0000 7f48f7537640 1 --2- 172.21.15.55:0/3728888235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f0161590 0x7f48f0072690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.685+0000 7f48f7537640 1 -- 172.21.15.55:0/3728888235 >> 172.21.15.55:0/3728888235 conn(0x7f48f00cf430 msgr2=0x7f48f00cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:14.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.685+0000 7f48f7537640 1 -- 172.21.15.55:0/3728888235 shutdown_connections 2024-05-20T22:10:14.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.686+0000 7f48f7537640 1 -- 172.21.15.55:0/3728888235 wait complete. 2024-05-20T22:10:14.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.687+0000 7f48f7537640 1 Processor -- start 2024-05-20T22:10:14.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.687+0000 7f48f7537640 1 -- start start 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f7537640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f015d550 0x7f48f014b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f7537640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f0161590 0x7f48f014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f7537640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f01f3000 0x7f48f014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f7537640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0072bd0 con 0x7f48f015d550 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f7537640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0161100 con 0x7f48f01f3000 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f7537640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48f01619a0 con 0x7f48f0161590 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f52d3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f015d550 0x7f48f014b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f52d3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f015d550 0x7f48f014b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44864/0 (socket says 172.21.15.55:44864) 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f52d3640 1 -- 172.21.15.55:0/1172377089 learned_addr learned my addr 172.21.15.55:0/1172377089 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f4ad2640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f0161590 0x7f48f014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f5ad4640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f01f3000 0x7f48f014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:14.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f52d3640 1 -- 172.21.15.55:0/1172377089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f0161590 msgr2=0x7f48f014c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f52d3640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f0161590 0x7f48f014c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f52d3640 1 -- 172.21.15.55:0/1172377089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f01f3000 msgr2=0x7f48f014daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f52d3640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f01f3000 0x7f48f014daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f52d3640 1 -- 172.21.15.55:0/1172377089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f014ca00 con 0x7f48f015d550 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f52d3640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f015d550 0x7f48f014b220 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f48e000d600 tx=0x7f48e000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48edffb640 1 -- 172.21.15.55:0/1172377089 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e00125f0 con 0x7f48f015d550 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48edffb640 1 -- 172.21.15.55:0/1172377089 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e0019070 con 0x7f48f015d550 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48edffb640 1 -- 172.21.15.55:0/1172377089 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e00129d0 con 0x7f48f015d550 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.689+0000 7f48f7537640 1 -- 172.21.15.55:0/1172377089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f014cce0 con 0x7f48f015d550 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.690+0000 7f48f7537640 1 -- 172.21.15.55:0/1172377089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f014d2a0 con 0x7f48f015d550 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.691+0000 7f48f7537640 1 -- 172.21.15.55:0/1172377089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f0071a70 con 0x7f48f015d550 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.691+0000 7f48edffb640 1 -- 172.21.15.55:0/1172377089 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48e0012b70 con 0x7f48f015d550 2024-05-20T22:10:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.692+0000 7f48edffb640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48dc06be60 0x7f48dc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:14.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.692+0000 7f48edffb640 1 -- 172.21.15.55:0/1172377089 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5864+0+0 (secure 0 0 0) 0x7f48e008b2d0 con 0x7f48f015d550 2024-05-20T22:10:14.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.692+0000 7f48f4ad2640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48dc06be60 0x7f48dc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:14.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.692+0000 7f48f4ad2640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48dc06be60 0x7f48dc06e370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f48f00cc4c0 tx=0x7f48e4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:14.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.696+0000 7f48edffb640 1 -- 172.21.15.55:0/1172377089 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f48e0059b50 con 0x7f48f015d550 2024-05-20T22:10:14.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:14.887+0000 7f48f7537640 1 -- 172.21.15.55:0/1172377089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f48f004f800 con 0x7f48f015d550 2024-05-20T22:10:15.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.540+0000 7f48edffb640 1 -- 172.21.15.55:0/1172377089 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) ==== 169+0+0 (secure 0 0 0) 0x7f48e0059450 con 0x7f48f015d550 2024-05-20T22:10:15.542 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-05-20T22:10:15.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.544+0000 7f48f7537640 1 -- 172.21.15.55:0/1172377089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48dc06be60 msgr2=0x7f48dc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:15.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.544+0000 7f48f7537640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48dc06be60 0x7f48dc06e370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f48f00cc4c0 tx=0x7f48e4009040 comp rx=0 tx=0).stop 2024-05-20T22:10:15.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.544+0000 7f48f7537640 1 -- 172.21.15.55:0/1172377089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f015d550 msgr2=0x7f48f014b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:15.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.544+0000 7f48f7537640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f015d550 0x7f48f014b220 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f48e000d600 tx=0x7f48e000dac0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.544+0000 7f48f7537640 1 -- 172.21.15.55:0/1172377089 shutdown_connections 2024-05-20T22:10:15.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.544+0000 7f48f7537640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f48dc06be60 0x7f48dc06e370 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.544+0000 7f48f7537640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f01f3000 0x7f48f014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.544+0000 7f48f7537640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48f0161590 0x7f48f014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.544+0000 7f48f7537640 1 --2- 172.21.15.55:0/1172377089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48f015d550 0x7f48f014b220 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.544+0000 7f48f7537640 1 -- 172.21.15.55:0/1172377089 >> 172.21.15.55:0/1172377089 conn(0x7f48f00cf430 msgr2=0x7f48f00b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:15.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.545+0000 7f48f7537640 1 -- 172.21.15.55:0/1172377089 shutdown_connections 2024-05-20T22:10:15.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.545+0000 7f48f7537640 1 -- 172.21.15.55:0/1172377089 wait complete. 2024-05-20T22:10:15.561 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T22:10:15.564 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T22:10:15.611 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2024-05-20T22:10:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2024-05-20T22:10:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T22:10:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2024-05-20T22:10:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2024-05-20T22:10:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2024-05-20T22:10:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2024-05-20T22:10:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: mon_allow_pool_size_one = true 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2024-05-20T22:10:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2024-05-20T22:10:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: osd mclock profile = high_recovery_ops 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug trim objects = true 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2024-05-20T22:10:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-20T22:10:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2024-05-20T22:10:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: # 1m isn't quite enough 2024-05-20T22:10:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: mon_down_mkfs_grace = 2m 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_filestore_osds = false 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: mon down mkfs grace = 300 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:10:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = "" 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds balancer = 20 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:10:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2024-05-20T22:10:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2024-05-20T22:10:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2024-05-20T22:10:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2024-05-20T22:10:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-20T22:10:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-20T22:10:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-20T22:10:15.623 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-05-20T22:10:15.623 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-05-20T22:10:15.623 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T22:10:15.623 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-05-20T22:10:15.623 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.623 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-05-20T22:10:15.623 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.623 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-05-20T22:10:15.623 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-05-20T22:10:15.624 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-05-20T22:10:15.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-05-20T22:10:15.626 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-05-20T22:10:15.627 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-05-20T22:10:15.628 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:10:15.629 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-20T22:10:15.630 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T22:10:15.631 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: mon down mkfs grace = 300 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T22:10:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-05-20T22:10:15.633 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-05-20T22:10:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:10:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-05-20T22:10:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-05-20T22:10:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-05-20T22:10:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-05-20T22:10:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-20T22:10:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-20T22:10:15.634 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-20T22:10:15.636 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-20T22:10:15.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.794+0000 7fc53f910640 1 Processor -- start 2024-05-20T22:10:15.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.794+0000 7fc53f910640 1 -- start start 2024-05-20T22:10:15.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc53f910640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc53809b170 0x7fc5380d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:15.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc53f910640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5380cda30 con 0x7fc5380b3e20 2024-05-20T22:10:15.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc53f910640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5380cdd10 con 0x7fc5380b7d20 2024-05-20T22:10:15.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc53f910640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fc53815f930 con 0x7fc53809b170 2024-05-20T22:10:15.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc53dead640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc53809b170 0x7fc5380d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:15.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc53ceab640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc5380b3e20 0x7fc53809aa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56448/0 (socket says 172.21.15.55:56448) 2024-05-20T22:10:15.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc53ceab640 1 -- 172.21.15.55:0/1027454829 learned_addr learned my addr 172.21.15.55:0/1027454829 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:15.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc53dead640 1 -- 172.21.15.55:0/1027454829 >> v1:172.21.15.138:6789/0 conn(0x7fc5380b7d20 legacy=0x7fc5380cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:15.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc53dead640 1 -- 172.21.15.55:0/1027454829 >> v1:172.21.15.55:6789/0 conn(0x7fc5380b3e20 legacy=0x7fc53809aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:15.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc52ffff640 1 -- 172.21.15.55:0/1027454829 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2110301352 0 0) 0x7fc5380cda30 con 0x7fc5380b3e20 2024-05-20T22:10:15.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.796+0000 7fc53dead640 1 -- 172.21.15.55:0/1027454829 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5380cdff0 con 0x7fc53809b170 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.797+0000 7fc53dead640 1 --2- 172.21.15.55:0/1027454829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc53809b170 0x7fc5380d18f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc530009d90 tx=0x7fc53002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5681b7f562a36e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.797+0000 7fc52ffff640 1 -- 172.21.15.55:0/1027454829 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530007740 con 0x7fc53809b170 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.797+0000 7fc52ffff640 1 -- 172.21.15.55:0/1027454829 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5300078e0 con 0x7fc53809b170 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.797+0000 7fc52ffff640 1 -- 172.21.15.55:0/1027454829 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530007cb0 con 0x7fc53809b170 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.798+0000 7fc53f910640 1 -- 172.21.15.55:0/1027454829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc53809b170 msgr2=0x7fc5380d18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.798+0000 7fc53f910640 1 --2- 172.21.15.55:0/1027454829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc53809b170 0x7fc5380d18f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc530009d90 tx=0x7fc53002a6f0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.798+0000 7fc53f910640 1 -- 172.21.15.55:0/1027454829 shutdown_connections 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.798+0000 7fc53f910640 1 --2- 172.21.15.55:0/1027454829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc53809b170 0x7fc5380d18f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.798+0000 7fc53f910640 1 -- 172.21.15.55:0/1027454829 >> 172.21.15.55:0/1027454829 conn(0x7fc5380b43e0 msgr2=0x7fc5380b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.798+0000 7fc53f910640 1 -- 172.21.15.55:0/1027454829 shutdown_connections 2024-05-20T22:10:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.798+0000 7fc53f910640 1 -- 172.21.15.55:0/1027454829 wait complete. 2024-05-20T22:10:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.800+0000 7fc53f910640 1 Processor -- start 2024-05-20T22:10:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.800+0000 7fc53f910640 1 -- start start 2024-05-20T22:10:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53f910640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5380b3e20 0x7fc5381f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53f910640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5380b7d20 0x7fc538071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53f910640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc538071a70 0x7fc53814c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53f910640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380b4230 con 0x7fc5380b3e20 2024-05-20T22:10:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53f910640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380ce5b0 con 0x7fc538071a70 2024-05-20T22:10:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53f910640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc53809bb80 con 0x7fc5380b7d20 2024-05-20T22:10:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53d6ac640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5380b3e20 0x7fc5381f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53d6ac640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5380b3e20 0x7fc5381f94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44876/0 (socket says 172.21.15.55:44876) 2024-05-20T22:10:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53d6ac640 1 -- 172.21.15.55:0/1805114207 learned_addr learned my addr 172.21.15.55:0/1805114207 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53d6ac640 1 -- 172.21.15.55:0/1805114207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5380b7d20 msgr2=0x7fc538071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53d6ac640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5380b7d20 0x7fc538071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53d6ac640 1 -- 172.21.15.55:0/1805114207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc538071a70 msgr2=0x7fc53814c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53d6ac640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc538071a70 0x7fc53814c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53d6ac640 1 -- 172.21.15.55:0/1805114207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5380b4dd0 con 0x7fc5380b3e20 2024-05-20T22:10:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.801+0000 7fc53d6ac640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5380b3e20 0x7fc5381f94d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c3f0 tx=0x7fc53400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.802+0000 7fc52dffb640 1 -- 172.21.15.55:0/1805114207 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534017070 con 0x7fc5380b3e20 2024-05-20T22:10:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.802+0000 7fc52dffb640 1 -- 172.21.15.55:0/1805114207 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5340052d0 con 0x7fc5380b3e20 2024-05-20T22:10:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.802+0000 7fc53f910640 1 -- 172.21.15.55:0/1805114207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5381f55e0 con 0x7fc5380b3e20 2024-05-20T22:10:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.802+0000 7fc52dffb640 1 -- 172.21.15.55:0/1805114207 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534003830 con 0x7fc5380b3e20 2024-05-20T22:10:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.802+0000 7fc53f910640 1 -- 172.21.15.55:0/1805114207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5381f5ba0 con 0x7fc5380b3e20 2024-05-20T22:10:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.803+0000 7fc53f910640 1 -- 172.21.15.55:0/1805114207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc538069090 con 0x7fc5380b3e20 2024-05-20T22:10:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.804+0000 7fc52dffb640 1 -- 172.21.15.55:0/1805114207 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc53401a070 con 0x7fc5380b3e20 2024-05-20T22:10:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.804+0000 7fc52dffb640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc51c06be30 0x7fc51c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.804+0000 7fc52dffb640 1 -- 172.21.15.55:0/1805114207 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5421+0+0 (secure 0 0 0) 0x7fc53408a0d0 con 0x7fc5380b3e20 2024-05-20T22:10:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.804+0000 7fc53ceab640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc51c06be30 0x7fc51c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.805+0000 7fc53ceab640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc51c06be30 0x7fc51c06e340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc528000c00 tx=0x7fc5280079c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:15.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.809+0000 7fc52dffb640 1 -- 172.21.15.55:0/1805114207 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc534058b10 con 0x7fc5380b3e20 2024-05-20T22:10:15.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.992+0000 7fc53f910640 1 -- 172.21.15.55:0/1805114207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc53809b7d0 con 0x7fc5380b3e20 2024-05-20T22:10:15.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.995+0000 7fc52dffb640 1 -- 172.21.15.55:0/1805114207 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fc534058410 con 0x7fc5380b3e20 2024-05-20T22:10:15.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 -- 172.21.15.55:0/1805114207 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc51c06be30 msgr2=0x7fc51c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:15.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc51c06be30 0x7fc51c06e340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc528000c00 tx=0x7fc5280079c0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 -- 172.21.15.55:0/1805114207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5380b3e20 msgr2=0x7fc5381f94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:15.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5380b3e20 0x7fc5381f94d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c3f0 tx=0x7fc53400c8b0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 -- 172.21.15.55:0/1805114207 shutdown_connections 2024-05-20T22:10:15.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc51c06be30 0x7fc51c06e340 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc538071a70 0x7fc53814c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5380b7d20 0x7fc538071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 --2- 172.21.15.55:0/1805114207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5380b3e20 0x7fc5381f94d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:15.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 -- 172.21.15.55:0/1805114207 >> 172.21.15.55:0/1805114207 conn(0x7fc5380b43e0 msgr2=0x7fc53815fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:15.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 -- 172.21.15.55:0/1805114207 shutdown_connections 2024-05-20T22:10:15.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:15.997+0000 7fc53f910640 1 -- 172.21.15.55:0/1805114207 wait complete. 2024-05-20T22:10:16.010 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj' 2024-05-20T22:10:16.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.123+0000 7f197aaf6640 1 Processor -- start 2024-05-20T22:10:16.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.123+0000 7f197aaf6640 1 -- start start 2024-05-20T22:10:16.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f197aaf6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19740b3c90 0x7f19740b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f197aaf6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19740b8550 0x7f197415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f197aaf6640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f197406b870 con 0x7f19740d3390 2024-05-20T22:10:16.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f197aaf6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f197406bb50 con 0x7f19740b8550 2024-05-20T22:10:16.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f197aaf6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f197406be30 con 0x7f19740b3c90 2024-05-20T22:10:16.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f1978892640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f19740d3390 0x7f197406c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56460/0 (socket says 172.21.15.55:56460) 2024-05-20T22:10:16.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f1978892640 1 -- 172.21.15.55:0/3655388684 learned_addr learned my addr 172.21.15.55:0/3655388684 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:16.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f1979093640 1 --2- 172.21.15.55:0/3655388684 >> v2:172.21.15.138:3300/0 conn(0x7f19740b8550 0x7f197415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f1973fff640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19740b3c90 0x7f19740b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 147737341 0 0) 0x7f197406b870 con 0x7f19740d3390 2024-05-20T22:10:16.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f195c003520 con 0x7f19740d3390 2024-05-20T22:10:16.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3147590193 0 0) 0x7f195c003520 con 0x7f19740d3390 2024-05-20T22:10:16.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19740b3c90 msgr2=0x7f19740b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:16.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19740b3c90 0x7f19740b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 >> v2:172.21.15.138:3300/0 conn(0x7f19740b8550 msgr2=0x7f197415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:16.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 --2- 172.21.15.55:0/3655388684 >> v2:172.21.15.138:3300/0 conn(0x7f19740b8550 0x7f197415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197406c110 con 0x7f19740d3390 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f1964001670 con 0x7f19740d3390 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 >> v1:172.21.15.55:6789/0 conn(0x7f19740d3390 legacy=0x7f197406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f195c0039b0 0x7f195c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f195c004800 0x7f195c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c005150 0x7f195c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f197406bb50 con 0x7f195c004800 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f197406be30 con 0x7f195c005150 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f195c00bf80 con 0x7f195c0039b0 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f1973fff640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19740b3c90 0x7f19740b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.125+0000 7f1979093640 1 --2- 172.21.15.55:0/3655388684 >> v2:172.21.15.138:3300/0 conn(0x7f19740b8550 0x7f197415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.126+0000 7f1973fff640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f195c004800 0x7f195c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.126+0000 7f1978892640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f195c0039b0 0x7f195c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.126+0000 7f1979093640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c005150 0x7f195c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.126+0000 7f1973fff640 1 -- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f195c0039b0 msgr2=0x7f195c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:16.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.126+0000 7f1973fff640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f195c0039b0 0x7f195c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.126+0000 7f1973fff640 1 -- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c005150 msgr2=0x7f195c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:16.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.126+0000 7f1973fff640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c005150 0x7f195c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.126+0000 7f1973fff640 1 -- 172.21.15.55:0/3655388684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c003520 con 0x7f195c004800 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.126+0000 7f1979093640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c005150 0x7f195c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.127+0000 7f1973fff640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f195c004800 0x7f195c004c10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f19740d12b0 tx=0x7f195c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0b8c0c0400fde3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.127+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968036070 con 0x7f195c004800 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.127+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1968004030 con 0x7f195c004800 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.127+0000 7f19737fe640 1 -- 172.21.15.55:0/3655388684 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19680097d0 con 0x7f195c004800 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.128+0000 7f197aaf6640 1 -- 172.21.15.55:0/3655388684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f195c004800 msgr2=0x7f195c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.128+0000 7f197aaf6640 1 --2- 172.21.15.55:0/3655388684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f195c004800 0x7f195c004c10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f19740d12b0 tx=0x7f195c005830 comp rx=0 tx=0).stop 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.128+0000 7f1979093640 1 -- 172.21.15.55:0/3655388684 reap_dead start 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.128+0000 7f197aaf6640 1 -- 172.21.15.55:0/3655388684 shutdown_connections 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.128+0000 7f197aaf6640 1 -- 172.21.15.55:0/3655388684 >> 172.21.15.55:0/3655388684 conn(0x7f19740b65f0 msgr2=0x7f19740b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.128+0000 7f197aaf6640 1 -- 172.21.15.55:0/3655388684 shutdown_connections 2024-05-20T22:10:16.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.128+0000 7f197aaf6640 1 -- 172.21.15.55:0/3655388684 wait complete. 2024-05-20T22:10:16.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.129+0000 7f197aaf6640 1 Processor -- start 2024-05-20T22:10:16.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.130+0000 7f197aaf6640 1 -- start start 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f197aaf6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f195c004800 0x7f1974159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f197aaf6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f197414fd50 0x7f1974150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f197aaf6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19741f4910 0x7f19741f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f197aaf6640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f19740b40a0 con 0x7f197414fd50 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f197aaf6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f197406b500 con 0x7f19741f4910 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f197aaf6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f19740b4220 con 0x7f195c004800 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1973fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f197414fd50 0x7f1974150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1973fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f197414fd50 0x7f1974150160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44902/0 (socket says 172.21.15.55:44902) 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1973fff640 1 -- 172.21.15.55:0/3534288624 learned_addr learned my addr 172.21.15.55:0/3534288624 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1979093640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19741f4910 0x7f19741f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1978892640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f195c004800 0x7f1974159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1973fff640 1 -- 172.21.15.55:0/3534288624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f195c004800 msgr2=0x7f1974159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1973fff640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f195c004800 0x7f1974159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1973fff640 1 -- 172.21.15.55:0/3534288624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19741f4910 msgr2=0x7f19741f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1973fff640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19741f4910 0x7f19741f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1973fff640 1 -- 172.21.15.55:0/3534288624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19741f5fc0 con 0x7f197414fd50 2024-05-20T22:10:16.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1979093640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19741f4910 0x7f19741f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:16.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.131+0000 7f1978892640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f195c004800 0x7f1974159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:16.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.132+0000 7f1973fff640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f197414fd50 0x7f1974150160 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1968009bf0 tx=0x7f1968004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:16.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.132+0000 7f19717fa640 1 -- 172.21.15.55:0/3534288624 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968040020 con 0x7f197414fd50 2024-05-20T22:10:16.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.132+0000 7f19717fa640 1 -- 172.21.15.55:0/3534288624 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1968003c60 con 0x7f197414fd50 2024-05-20T22:10:16.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.132+0000 7f197aaf6640 1 -- 172.21.15.55:0/3534288624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19741506a0 con 0x7f197414fd50 2024-05-20T22:10:16.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.132+0000 7f19717fa640 1 -- 172.21.15.55:0/3534288624 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968036040 con 0x7f197414fd50 2024-05-20T22:10:16.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.132+0000 7f197aaf6640 1 -- 172.21.15.55:0/3534288624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741f6530 con 0x7f197414fd50 2024-05-20T22:10:16.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.133+0000 7f197aaf6640 1 -- 172.21.15.55:0/3534288624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1974069090 con 0x7f197414fd50 2024-05-20T22:10:16.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.134+0000 7f19717fa640 1 -- 172.21.15.55:0/3534288624 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1968043020 con 0x7f197414fd50 2024-05-20T22:10:16.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.134+0000 7f19717fa640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f196006be60 0x7f196006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.134+0000 7f19717fa640 1 -- 172.21.15.55:0/3534288624 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5421+0+0 (secure 0 0 0) 0x7f19680ab110 con 0x7f197414fd50 2024-05-20T22:10:16.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.134+0000 7f1978892640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f196006be60 0x7f196006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.135+0000 7f1978892640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f196006be60 0x7f196006e370 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1964001550 tx=0x7f1964009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:16.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.139+0000 7f19717fa640 1 -- 172.21.15.55:0/3534288624 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f196804a030 con 0x7f197414fd50 2024-05-20T22:10:16.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.396+0000 7f197aaf6640 1 -- 172.21.15.55:0/3534288624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]} v 0) -- 0x7f19740d2eb0 con 0x7f197414fd50 2024-05-20T22:10:16.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.552+0000 7f19717fa640 1 -- 172.21.15.55:0/3534288624 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]}]=0 v144) ==== 147+0+0 (secure 0 0 0) 0x7f1968079c70 con 0x7f197414fd50 2024-05-20T22:10:16.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 -- 172.21.15.55:0/3534288624 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f196006be60 msgr2=0x7f196006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f196006be60 0x7f196006e370 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1964001550 tx=0x7f1964009040 comp rx=0 tx=0).stop 2024-05-20T22:10:16.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 -- 172.21.15.55:0/3534288624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f197414fd50 msgr2=0x7f1974150160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f197414fd50 0x7f1974150160 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1968009bf0 tx=0x7f1968004730 comp rx=0 tx=0).stop 2024-05-20T22:10:16.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 -- 172.21.15.55:0/3534288624 shutdown_connections 2024-05-20T22:10:16.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f196006be60 0x7f196006e370 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19741f4910 0x7f19741f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f197414fd50 0x7f1974150160 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 --2- 172.21.15.55:0/3534288624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f195c004800 0x7f1974159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 -- 172.21.15.55:0/3534288624 >> 172.21.15.55:0/3534288624 conn(0x7f19740b65f0 msgr2=0x7f19740cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:16.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 -- 172.21.15.55:0/3534288624 shutdown_connections 2024-05-20T22:10:16.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.555+0000 7f197aaf6640 1 -- 172.21.15.55:0/3534288624 wait complete. 2024-05-20T22:10:16.567 INFO:tasks.cephfs_test_runner:The maximum number of openfiles omap objects keys are now equal to ... ok 2024-05-20T22:10:16.568 INFO:tasks.cephfs_test_runner:Starting test: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) Opening a file should increment omap_total_updates by 1. 2024-05-20T22:10:16.568 INFO:tasks.cephfs_test_runner:test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) 2024-05-20T22:10:16.568 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:10:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.683+0000 7f98dc579640 1 Processor -- start 2024-05-20T22:10:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.683+0000 7f98dc579640 1 -- start start 2024-05-20T22:10:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.684+0000 7f98dc579640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98d40b3b30 0x7f98d40b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.684+0000 7f98dc579640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d40d3dd0 con 0x7f98d415f100 2024-05-20T22:10:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.684+0000 7f98dc579640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d40d40b0 con 0x7f98d406c790 2024-05-20T22:10:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.684+0000 7f98dc579640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f98d40d4390 con 0x7f98d40b3b30 2024-05-20T22:10:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.684+0000 7f98da315640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98d40b3b30 0x7f98d40b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.684+0000 7f98d9b14640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f98d415f100 0x7f98d406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56474/0 (socket says 172.21.15.55:56474) 2024-05-20T22:10:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.684+0000 7f98d9b14640 1 -- 172.21.15.55:0/574852691 learned_addr learned my addr 172.21.15.55:0/574852691 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.685+0000 7f98da315640 1 -- 172.21.15.55:0/574852691 >> v1:172.21.15.138:6789/0 conn(0x7f98d406c790 legacy=0x7f98d40cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.685+0000 7f98da315640 1 -- 172.21.15.55:0/574852691 >> v1:172.21.15.55:6789/0 conn(0x7f98d415f100 legacy=0x7f98d406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.685+0000 7f98da315640 1 -- 172.21.15.55:0/574852691 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d406b5a0 con 0x7f98d40b3b30 2024-05-20T22:10:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.685+0000 7f98d9313640 1 -- 172.21.15.55:0/574852691 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2590529291 0 0) 0x7f98d40d3dd0 con 0x7f98d415f100 2024-05-20T22:10:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.686+0000 7f98da315640 1 --2- 172.21.15.55:0/574852691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 0x7f98d40b7fe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f98d00097a0 tx=0x7f98d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df57ba9473ddbe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.686+0000 7f98d9313640 1 -- 172.21.15.55:0/574852691 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0037070 con 0x7f98d40b3b30 2024-05-20T22:10:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.686+0000 7f98d9313640 1 -- 172.21.15.55:0/574852691 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d0032440 con 0x7f98d40b3b30 2024-05-20T22:10:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.686+0000 7f98d9313640 1 -- 172.21.15.55:0/574852691 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0032810 con 0x7f98d40b3b30 2024-05-20T22:10:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.686+0000 7f98dc579640 1 -- 172.21.15.55:0/574852691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 msgr2=0x7f98d40b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.686+0000 7f98dc579640 1 --2- 172.21.15.55:0/574852691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 0x7f98d40b7fe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f98d00097a0 tx=0x7f98d002a230 comp rx=0 tx=0).stop 2024-05-20T22:10:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.687+0000 7f98dc579640 1 -- 172.21.15.55:0/574852691 shutdown_connections 2024-05-20T22:10:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.687+0000 7f98dc579640 1 --2- 172.21.15.55:0/574852691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 0x7f98d40b7fe0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.687+0000 7f98dc579640 1 -- 172.21.15.55:0/574852691 >> 172.21.15.55:0/574852691 conn(0x7f98d40d51d0 msgr2=0x7f98d40b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.687+0000 7f98dc579640 1 -- 172.21.15.55:0/574852691 shutdown_connections 2024-05-20T22:10:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.687+0000 7f98dc579640 1 -- 172.21.15.55:0/574852691 wait complete. 2024-05-20T22:10:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.688+0000 7f98dc579640 1 Processor -- start 2024-05-20T22:10:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.688+0000 7f98dc579640 1 -- start start 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98dc579640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d406c790 0x7f98d41f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98dc579640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 0x7f98d40abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98dc579640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d415f100 0x7f98d40b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98dc579640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f98d406cba0 con 0x7f98d40b3b30 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98dc579640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98d406bb60 con 0x7f98d406c790 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98dc579640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98d406cd20 con 0x7f98d415f100 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98d9b14640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 0x7f98d40abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98d9b14640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 0x7f98d40abf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44930/0 (socket says 172.21.15.55:44930) 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98d9b14640 1 -- 172.21.15.55:0/4289145429 learned_addr learned my addr 172.21.15.55:0/4289145429 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98da315640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d406c790 0x7f98d41f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98dab16640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d415f100 0x7f98d40b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98d9b14640 1 -- 172.21.15.55:0/4289145429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d415f100 msgr2=0x7f98d40b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98d9b14640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d415f100 0x7f98d40b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98d9b14640 1 -- 172.21.15.55:0/4289145429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d406c790 msgr2=0x7f98d41f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98d9b14640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d406c790 0x7f98d41f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98d9b14640 1 -- 172.21.15.55:0/4289145429 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d40b1c60 con 0x7f98d40b3b30 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98d9b14640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 0x7f98d40abf00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c390 tx=0x7f98c800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.690+0000 7f98c6ffd640 1 -- 172.21.15.55:0/4289145429 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c80074e0 con 0x7f98d40b3b30 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.691+0000 7f98c6ffd640 1 -- 172.21.15.55:0/4289145429 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c80047e0 con 0x7f98d40b3b30 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.691+0000 7f98dc579640 1 -- 172.21.15.55:0/4289145429 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d40b1f40 con 0x7f98d40b3b30 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.691+0000 7f98c6ffd640 1 -- 172.21.15.55:0/4289145429 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8004c10 con 0x7f98d40b3b30 2024-05-20T22:10:16.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.691+0000 7f98dc579640 1 -- 172.21.15.55:0/4289145429 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d40b2500 con 0x7f98d40b3b30 2024-05-20T22:10:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.692+0000 7f98dc579640 1 -- 172.21.15.55:0/4289145429 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d4069090 con 0x7f98d40b3b30 2024-05-20T22:10:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.692+0000 7f98c6ffd640 1 -- 172.21.15.55:0/4289145429 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98c8011070 con 0x7f98d40b3b30 2024-05-20T22:10:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.693+0000 7f98c6ffd640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98b806be30 0x7f98b806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.693+0000 7f98c6ffd640 1 -- 172.21.15.55:0/4289145429 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5421+0+0 (secure 0 0 0) 0x7f98c808a7d0 con 0x7f98d40b3b30 2024-05-20T22:10:16.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.693+0000 7f98da315640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98b806be30 0x7f98b806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:16.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.693+0000 7f98da315640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98b806be30 0x7f98b806e340 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f98d0034000 tx=0x7f98d002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:16.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.697+0000 7f98c6ffd640 1 -- 172.21.15.55:0/4289145429 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f98c8058bc0 con 0x7f98d40b3b30 2024-05-20T22:10:16.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.882+0000 7f98dc579640 1 -- 172.21.15.55:0/4289145429 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98d40d2ee0 con 0x7f98d40b3b30 2024-05-20T22:10:16.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.883+0000 7f98c6ffd640 1 -- 172.21.15.55:0/4289145429 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+13425 (secure 0 0 0) 0x7f98c805cec0 con 0x7f98d40b3b30 2024-05-20T22:10:16.884 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:16.884 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":47,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:10:15.539784+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6835/270300569":"2024-05-21T22:10:12.481886+0000","172.21.15.138:6834/270300569":"2024-05-21T22:10:12.481886+0000","172.21.15.138:6835/4137968836":"2024-05-21T22:09:29.217850+0000","172.21.15.138:6833/4137968836":"2024-05-21T22:09:29.217850+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:10:16.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 -- 172.21.15.55:0/4289145429 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98b806be30 msgr2=0x7f98b806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98b806be30 0x7f98b806e340 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f98d0034000 tx=0x7f98d002e000 comp rx=0 tx=0).stop 2024-05-20T22:10:16.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 -- 172.21.15.55:0/4289145429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 msgr2=0x7f98d40abf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:16.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 0x7f98d40abf00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c390 tx=0x7f98c800c850 comp rx=0 tx=0).stop 2024-05-20T22:10:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 -- 172.21.15.55:0/4289145429 shutdown_connections 2024-05-20T22:10:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98b806be30 0x7f98b806e340 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d415f100 0x7f98d40b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98d40b3b30 0x7f98d40abf00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 --2- 172.21.15.55:0/4289145429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d406c790 0x7f98d41f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 -- 172.21.15.55:0/4289145429 >> 172.21.15.55:0/4289145429 conn(0x7f98d40d51d0 msgr2=0x7f98d40b6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 -- 172.21.15.55:0/4289145429 shutdown_connections 2024-05-20T22:10:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:16.885+0000 7f98dc579640 1 -- 172.21.15.55:0/4289145429 wait complete. 2024-05-20T22:10:16.899 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T22:10:16.899 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:10:17.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.017+0000 7f3a7e437640 1 Processor -- start 2024-05-20T22:10:17.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.017+0000 7f3a7e437640 1 -- start start 2024-05-20T22:10:17.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.018+0000 7f3a7e437640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a7815e550 0x7f3a7815e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a7e437640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 0x7f3a7806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a7e437640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a78157870 con 0x7f3a78160a30 2024-05-20T22:10:17.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a7e437640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f3a7806c930 con 0x7f3a7815e550 2024-05-20T22:10:17.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a7e437640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3a78073310 con 0x7f3a7806bfe0 2024-05-20T22:10:17.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a77fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a7815e550 0x7f3a7815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a77fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a7815e550 0x7f3a7815e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44932/0 (socket says 172.21.15.55:44932) 2024-05-20T22:10:17.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a77fff640 1 -- 172.21.15.55:0/2928234039 learned_addr learned my addr 172.21.15.55:0/2928234039 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:17.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a777fe640 1 --2- 172.21.15.55:0/2928234039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 0x7f3a7806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a77fff640 1 -- 172.21.15.55:0/2928234039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 msgr2=0x7f3a7806c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:17.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a77fff640 1 --2- 172.21.15.55:0/2928234039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 0x7f3a7806c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a77fff640 1 -- 172.21.15.55:0/2928234039 >> v1:172.21.15.138:6789/0 conn(0x7f3a78160a30 legacy=0x7f3a78160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:17.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a77fff640 1 -- 172.21.15.55:0/2928234039 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a780735f0 con 0x7f3a7815e550 2024-05-20T22:10:17.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.019+0000 7f3a777fe640 1 --2- 172.21.15.55:0/2928234039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 0x7f3a7806c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:17.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.020+0000 7f3a77fff640 1 --2- 172.21.15.55:0/2928234039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 0x7f3a7815e960 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009870 tx=0x7f3a6802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd5e4253f95f134 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:17.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.020+0000 7f3a76ffd640 1 -- 172.21.15.55:0/2928234039 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68037070 con 0x7f3a7815e550 2024-05-20T22:10:17.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.020+0000 7f3a76ffd640 1 -- 172.21.15.55:0/2928234039 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a68032440 con 0x7f3a7815e550 2024-05-20T22:10:17.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.020+0000 7f3a76ffd640 1 -- 172.21.15.55:0/2928234039 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68032810 con 0x7f3a7815e550 2024-05-20T22:10:17.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.021+0000 7f3a7e437640 1 -- 172.21.15.55:0/2928234039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 msgr2=0x7f3a7815e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:17.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.021+0000 7f3a7e437640 1 --2- 172.21.15.55:0/2928234039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 0x7f3a7815e960 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009870 tx=0x7f3a6802a250 comp rx=0 tx=0).stop 2024-05-20T22:10:17.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.021+0000 7f3a7e437640 1 -- 172.21.15.55:0/2928234039 shutdown_connections 2024-05-20T22:10:17.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.021+0000 7f3a7e437640 1 --2- 172.21.15.55:0/2928234039 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 0x7f3a7806c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.021+0000 7f3a7e437640 1 --2- 172.21.15.55:0/2928234039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 0x7f3a7815e960 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.021+0000 7f3a7e437640 1 -- 172.21.15.55:0/2928234039 >> 172.21.15.55:0/2928234039 conn(0x7f3a780b3980 msgr2=0x7f3a780b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:17.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.021+0000 7f3a7e437640 1 -- 172.21.15.55:0/2928234039 shutdown_connections 2024-05-20T22:10:17.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.021+0000 7f3a7e437640 1 -- 172.21.15.55:0/2928234039 wait complete. 2024-05-20T22:10:17.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.023+0000 7f3a7e437640 1 Processor -- start 2024-05-20T22:10:17.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.023+0000 7f3a7e437640 1 -- start start 2024-05-20T22:10:17.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a7e437640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 0x7f3a780d0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a7e437640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 0x7f3a781f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a7e437640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a78160a30 0x7f3a781f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a7e437640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7806bac0 con 0x7f3a7815e550 2024-05-20T22:10:17.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a7e437640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7806b7e0 con 0x7f3a78160a30 2024-05-20T22:10:17.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a7e437640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3a78072690 con 0x7f3a7806bfe0 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a777fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 0x7f3a781f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a777fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 0x7f3a781f3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44948/0 (socket says 172.21.15.55:44948) 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a777fe640 1 -- 172.21.15.55:0/2506067595 learned_addr learned my addr 172.21.15.55:0/2506067595 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a7c9d4640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a78160a30 0x7f3a781f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a77fff640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 0x7f3a780d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a777fe640 1 -- 172.21.15.55:0/2506067595 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 msgr2=0x7f3a780d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.024+0000 7f3a777fe640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 0x7f3a780d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a777fe640 1 -- 172.21.15.55:0/2506067595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a78160a30 msgr2=0x7f3a781f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a777fe640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a78160a30 0x7f3a781f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a777fe640 1 -- 172.21.15.55:0/2506067595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a781f3bd0 con 0x7f3a7815e550 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a7c9d4640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a78160a30 0x7f3a781f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a77fff640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 0x7f3a780d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:17.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a777fe640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 0x7f3a781f3150 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000c7f0 tx=0x7f3a6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:17.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a74ff9640 1 -- 172.21.15.55:0/2506067595 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60015070 con 0x7f3a7815e550 2024-05-20T22:10:17.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a74ff9640 1 -- 172.21.15.55:0/2506067595 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a60007820 con 0x7f3a7815e550 2024-05-20T22:10:17.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a7e437640 1 -- 172.21.15.55:0/2506067595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a780ce410 con 0x7f3a7815e550 2024-05-20T22:10:17.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a74ff9640 1 -- 172.21.15.55:0/2506067595 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60007bd0 con 0x7f3a7815e550 2024-05-20T22:10:17.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.025+0000 7f3a7e437640 1 -- 172.21.15.55:0/2506067595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a780ce9d0 con 0x7f3a7815e550 2024-05-20T22:10:17.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.026+0000 7f3a7e437640 1 -- 172.21.15.55:0/2506067595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a780ce180 con 0x7f3a7815e550 2024-05-20T22:10:17.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.027+0000 7f3a74ff9640 1 -- 172.21.15.55:0/2506067595 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a60018070 con 0x7f3a7815e550 2024-05-20T22:10:17.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.027+0000 7f3a74ff9640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3a5806be60 0x7f3a5806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.027+0000 7f3a74ff9640 1 -- 172.21.15.55:0/2506067595 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5421+0+0 (secure 0 0 0) 0x7f3a6008ae40 con 0x7f3a7815e550 2024-05-20T22:10:17.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.027+0000 7f3a77fff640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3a5806be60 0x7f3a5806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.028+0000 7f3a77fff640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3a5806be60 0x7f3a5806e370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3a68034000 tx=0x7f3a6802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:17.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.032+0000 7f3a74ff9640 1 -- 172.21.15.55:0/2506067595 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3a60059910 con 0x7f3a7815e550 2024-05-20T22:10:17.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.218+0000 7f3a7e437640 1 -- 172.21.15.55:0/2506067595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a780cd080 con 0x7f3a7815e550 2024-05-20T22:10:17.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.218+0000 7f3a74ff9640 1 -- 172.21.15.55:0/2506067595 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7f3a60059210 con 0x7f3a7815e550 2024-05-20T22:10:17.219 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:10:17.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.220+0000 7f3a7e437640 1 -- 172.21.15.55:0/2506067595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3a5806be60 msgr2=0x7f3a5806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:17.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.220+0000 7f3a7e437640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3a5806be60 0x7f3a5806e370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3a68034000 tx=0x7f3a6802e000 comp rx=0 tx=0).stop 2024-05-20T22:10:17.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.220+0000 7f3a7e437640 1 -- 172.21.15.55:0/2506067595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 msgr2=0x7f3a781f3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:17.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.220+0000 7f3a7e437640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 0x7f3a781f3150 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000c7f0 tx=0x7f3a6000ccb0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.220+0000 7f3a7e437640 1 -- 172.21.15.55:0/2506067595 shutdown_connections 2024-05-20T22:10:17.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.220+0000 7f3a7e437640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f3a5806be60 0x7f3a5806e370 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.220+0000 7f3a7e437640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a78160a30 0x7f3a781f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.220+0000 7f3a7e437640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7815e550 0x7f3a781f3150 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.220+0000 7f3a7e437640 1 --2- 172.21.15.55:0/2506067595 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a7806bfe0 0x7f3a780d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.220+0000 7f3a7e437640 1 -- 172.21.15.55:0/2506067595 >> 172.21.15.55:0/2506067595 conn(0x7f3a780b3980 msgr2=0x7f3a780b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.221+0000 7f3a7e437640 1 -- 172.21.15.55:0/2506067595 shutdown_connections 2024-05-20T22:10:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.221+0000 7f3a7e437640 1 -- 172.21.15.55:0/2506067595 wait complete. 2024-05-20T22:10:17.233 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters' 2024-05-20T22:10:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.346+0000 7f387e00d640 1 Processor -- start 2024-05-20T22:10:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.346+0000 7f387e00d640 1 -- start start 2024-05-20T22:10:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f387e00d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f38780cb640 0x7f3878160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f387e00d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3878160fe0 0x7f387806bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f387e00d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38781613f0 con 0x7f38780af980 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f387e00d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f38780cbc60 con 0x7f3878160fe0 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f387e00d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f38780cbf40 con 0x7f38780cb640 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f3877fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3878160fe0 0x7f387806bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f38777fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f38780af980 0x7f38780cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56484/0 (socket says 172.21.15.55:56484) 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f38777fe640 1 -- 172.21.15.55:0/2818444786 learned_addr learned my addr 172.21.15.55:0/2818444786 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f3876ffd640 1 --2- 172.21.15.55:0/2818444786 >> v2:172.21.15.138:3300/0 conn(0x7f38780cb640 0x7f3878160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f3877fff640 1 -- 172.21.15.55:0/2818444786 >> v2:172.21.15.138:3300/0 conn(0x7f38780cb640 msgr2=0x7f3878160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f3877fff640 1 --2- 172.21.15.55:0/2818444786 >> v2:172.21.15.138:3300/0 conn(0x7f38780cb640 0x7f3878160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f3877fff640 1 -- 172.21.15.55:0/2818444786 >> v1:172.21.15.55:6789/0 conn(0x7f38780af980 legacy=0x7f38780cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f3877fff640 1 -- 172.21.15.55:0/2818444786 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780cc220 con 0x7f3878160fe0 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.348+0000 7f38767fc640 1 -- 172.21.15.55:0/2818444786 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3602877216 0 0) 0x7f3868001110 con 0x7f38780af980 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.349+0000 7f3877fff640 1 --2- 172.21.15.55:0/2818444786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3878160fe0 0x7f387806bac0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3878070580 tx=0x7f387002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3477a44263a43bd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.349+0000 7f38767fc640 1 -- 172.21.15.55:0/2818444786 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870007740 con 0x7f3878160fe0 2024-05-20T22:10:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.349+0000 7f38767fc640 1 -- 172.21.15.55:0/2818444786 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38700078e0 con 0x7f3878160fe0 2024-05-20T22:10:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.349+0000 7f38767fc640 1 -- 172.21.15.55:0/2818444786 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870007cb0 con 0x7f3878160fe0 2024-05-20T22:10:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.350+0000 7f387e00d640 1 -- 172.21.15.55:0/2818444786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3878160fe0 msgr2=0x7f387806bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.350+0000 7f387e00d640 1 --2- 172.21.15.55:0/2818444786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3878160fe0 0x7f387806bac0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3878070580 tx=0x7f387002a5f0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.350+0000 7f387e00d640 1 -- 172.21.15.55:0/2818444786 shutdown_connections 2024-05-20T22:10:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.350+0000 7f387e00d640 1 --2- 172.21.15.55:0/2818444786 >> v2:172.21.15.138:3300/0 conn(0x7f38780cb640 0x7f3878160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.350+0000 7f387e00d640 1 --2- 172.21.15.55:0/2818444786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3878160fe0 0x7f387806bac0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.350+0000 7f387e00d640 1 -- 172.21.15.55:0/2818444786 >> 172.21.15.55:0/2818444786 conn(0x7f38780cd4d0 msgr2=0x7f38780cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.350+0000 7f387e00d640 1 -- 172.21.15.55:0/2818444786 shutdown_connections 2024-05-20T22:10:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.350+0000 7f387e00d640 1 -- 172.21.15.55:0/2818444786 wait complete. 2024-05-20T22:10:17.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.352+0000 7f387e00d640 1 Processor -- start 2024-05-20T22:10:17.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.352+0000 7f387e00d640 1 -- start start 2024-05-20T22:10:17.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.353+0000 7f387e00d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38780af980 0x7f38781f9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.353+0000 7f387e00d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38780cb640 0x7f38780c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.353+0000 7f387e00d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878160fe0 0x7f387814bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.353+0000 7f387e00d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f387806c180 con 0x7f38780af980 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.353+0000 7f387e00d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f387806c000 con 0x7f3878160fe0 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.353+0000 7f387e00d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f387806c300 con 0x7f38780cb640 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.353+0000 7f38777fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38780af980 0x7f38781f9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.353+0000 7f38777fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38780af980 0x7f38781f9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44966/0 (socket says 172.21.15.55:44966) 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.353+0000 7f38777fe640 1 -- 172.21.15.55:0/2321020830 learned_addr learned my addr 172.21.15.55:0/2321020830 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f3876ffd640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38780cb640 0x7f38780c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f3877fff640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878160fe0 0x7f387814bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f38777fe640 1 -- 172.21.15.55:0/2321020830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38780cb640 msgr2=0x7f38780c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:17.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f38777fe640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38780cb640 0x7f38780c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f38777fe640 1 -- 172.21.15.55:0/2321020830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878160fe0 msgr2=0x7f387814bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f38777fe640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878160fe0 0x7f387814bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f38777fe640 1 -- 172.21.15.55:0/2321020830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780b3730 con 0x7f38780af980 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f3877fff640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878160fe0 0x7f387814bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f3876ffd640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38780cb640 0x7f38780c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f38777fe640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38780af980 0x7f38781f9a10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f386800c410 tx=0x7f386800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f385ffff640 1 -- 172.21.15.55:0/2321020830 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38680074e0 con 0x7f38780af980 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f385ffff640 1 -- 172.21.15.55:0/2321020830 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38680047e0 con 0x7f38780af980 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f385ffff640 1 -- 172.21.15.55:0/2321020830 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868004bd0 con 0x7f38780af980 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f387e00d640 1 -- 172.21.15.55:0/2321020830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38781f6b40 con 0x7f38780af980 2024-05-20T22:10:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.354+0000 7f387e00d640 1 -- 172.21.15.55:0/2321020830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38781f5f60 con 0x7f38780af980 2024-05-20T22:10:17.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.355+0000 7f387e00d640 1 -- 172.21.15.55:0/2321020830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3878069090 con 0x7f38780af980 2024-05-20T22:10:17.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.356+0000 7f385ffff640 1 -- 172.21.15.55:0/2321020830 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3868011070 con 0x7f38780af980 2024-05-20T22:10:17.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.356+0000 7f385ffff640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f386406be30 0x7f386406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:17.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.356+0000 7f385ffff640 1 -- 172.21.15.55:0/2321020830 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5421+0+0 (secure 0 0 0) 0x7f386808aaa0 con 0x7f38780af980 2024-05-20T22:10:17.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.356+0000 7f3876ffd640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f386406be30 0x7f386406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:17.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.357+0000 7f3876ffd640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f386406be30 0x7f386406e340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f386c006fd0 tx=0x7f386c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:17.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.361+0000 7f385ffff640 1 -- 172.21.15.55:0/2321020830 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f386808ce20 con 0x7f38780af980 2024-05-20T22:10:17.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:17.619+0000 7f387e00d640 1 -- 172.21.15.55:0/2321020830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]} v 0) -- 0x7f38780ad670 con 0x7f38780af980 2024-05-20T22:10:18.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.569+0000 7f385ffff640 1 -- 172.21.15.55:0/2321020830 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]}]=0 v146) ==== 146+0+0 (secure 0 0 0) 0x7f38781613f0 con 0x7f38780af980 2024-05-20T22:10:18.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 -- 172.21.15.55:0/2321020830 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f386406be30 msgr2=0x7f386406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:18.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f386406be30 0x7f386406e340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f386c006fd0 tx=0x7f386c008040 comp rx=0 tx=0).stop 2024-05-20T22:10:18.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 -- 172.21.15.55:0/2321020830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38780af980 msgr2=0x7f38781f9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:18.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38780af980 0x7f38781f9a10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f386800c410 tx=0x7f386800c8d0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 -- 172.21.15.55:0/2321020830 shutdown_connections 2024-05-20T22:10:18.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f386406be30 0x7f386406e340 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878160fe0 0x7f387814bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38780cb640 0x7f38780c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 --2- 172.21.15.55:0/2321020830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38780af980 0x7f38781f9a10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 -- 172.21.15.55:0/2321020830 >> 172.21.15.55:0/2321020830 conn(0x7f38780cd4d0 msgr2=0x7f387815b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:18.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 -- 172.21.15.55:0/2321020830 shutdown_connections 2024-05-20T22:10:18.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.572+0000 7f387e00d640 1 -- 172.21.15.55:0/2321020830 wait complete. 2024-05-20T22:10:18.587 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-20T22:10:18.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.711+0000 7fc8ddb4c640 1 Processor -- start 2024-05-20T22:10:18.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.712+0000 7fc8ddb4c640 1 -- start start 2024-05-20T22:10:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.713+0000 7fc8ddb4c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8d815f150 0x7fc8d80cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.713+0000 7fc8ddb4c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d806b500 0x7fc8d806b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.713+0000 7fc8ddb4c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8d80d3780 con 0x7fc8d80d32a0 2024-05-20T22:10:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.713+0000 7fc8ddb4c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc8d80d3a60 con 0x7fc8d815f150 2024-05-20T22:10:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.713+0000 7fc8ddb4c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d80d3d40 con 0x7fc8d806b500 2024-05-20T22:10:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.713+0000 7fc8d7fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d806b500 0x7fc8d806b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.713+0000 7fc8d6ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8d815f150 0x7fc8d80cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.713+0000 7fc8d77fe640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc8d80d32a0 0x7fc8d80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:39262/0 (socket says 172.21.15.55:39262) 2024-05-20T22:10:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.713+0000 7fc8d77fe640 1 -- 172.21.15.55:0/1472529306 learned_addr learned my addr 172.21.15.55:0/1472529306 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.714+0000 7fc8d6ffd640 1 -- 172.21.15.55:0/1472529306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d806b500 msgr2=0x7fc8d806b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.714+0000 7fc8d6ffd640 1 --2- 172.21.15.55:0/1472529306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d806b500 0x7fc8d806b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.714+0000 7fc8d6ffd640 1 -- 172.21.15.55:0/1472529306 >> v1:172.21.15.138:6789/0 conn(0x7fc8d80d32a0 legacy=0x7fc8d80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.714+0000 7fc8d6ffd640 1 -- 172.21.15.55:0/1472529306 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d80d4020 con 0x7fc8d815f150 2024-05-20T22:10:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.714+0000 7fc8d7fff640 1 --2- 172.21.15.55:0/1472529306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d806b500 0x7fc8d806b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.715+0000 7fc8d6ffd640 1 --2- 172.21.15.55:0/1472529306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d815f150 0x7fc8d80cfda0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0097a0 tx=0x7fc8cc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2bea4e1197d802a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.715+0000 7fc8d67fc640 1 -- 172.21.15.55:0/1472529306 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc037070 con 0x7fc8d815f150 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.715+0000 7fc8d67fc640 1 -- 172.21.15.55:0/1472529306 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8cc032440 con 0x7fc8d815f150 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.715+0000 7fc8d67fc640 1 -- 172.21.15.55:0/1472529306 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc032810 con 0x7fc8d815f150 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.715+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1472529306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d815f150 msgr2=0x7fc8d80cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.715+0000 7fc8ddb4c640 1 --2- 172.21.15.55:0/1472529306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d815f150 0x7fc8d80cfda0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0097a0 tx=0x7fc8cc02a180 comp rx=0 tx=0).stop 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.716+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1472529306 shutdown_connections 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.716+0000 7fc8ddb4c640 1 --2- 172.21.15.55:0/1472529306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d806b500 0x7fc8d806b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.716+0000 7fc8ddb4c640 1 --2- 172.21.15.55:0/1472529306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d815f150 0x7fc8d80cfda0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.716+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1472529306 >> 172.21.15.55:0/1472529306 conn(0x7fc8d80b6de0 msgr2=0x7fc8d80d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.716+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1472529306 shutdown_connections 2024-05-20T22:10:18.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.716+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1472529306 wait complete. 2024-05-20T22:10:18.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.717+0000 7fc8ddb4c640 1 Processor -- start 2024-05-20T22:10:18.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.718+0000 7fc8ddb4c640 1 -- start start 2024-05-20T22:10:18.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8ddb4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d806b500 0x7fc8d80c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:18.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8ddb4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d32a0 0x7fc8d81533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:18.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8ddb4c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d815f150 0x7fc8d80b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:18.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8ddb4c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d80d4820 con 0x7fc8d806b500 2024-05-20T22:10:18.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8ddb4c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d815f560 con 0x7fc8d80d32a0 2024-05-20T22:10:18.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8ddb4c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d80cd3f0 con 0x7fc8d815f150 2024-05-20T22:10:18.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d806b500 0x7fc8d80c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:18.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d806b500 0x7fc8d80c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55472/0 (socket says 172.21.15.55:55472) 2024-05-20T22:10:18.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d77fe640 1 -- 172.21.15.55:0/1277554014 learned_addr learned my addr 172.21.15.55:0/1277554014 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d7fff640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d815f150 0x7fc8d80b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d6ffd640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d32a0 0x7fc8d81533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d77fe640 1 -- 172.21.15.55:0/1277554014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d815f150 msgr2=0x7fc8d80b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d77fe640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d815f150 0x7fc8d80b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d77fe640 1 -- 172.21.15.55:0/1277554014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d32a0 msgr2=0x7fc8d81533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d77fe640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d32a0 0x7fc8d81533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d77fe640 1 -- 172.21.15.55:0/1277554014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d81609f0 con 0x7fc8d806b500 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d6ffd640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d32a0 0x7fc8d81533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.719+0000 7fc8d7fff640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d815f150 0x7fc8d80b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.720+0000 7fc8d77fe640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d806b500 0x7fc8d80c91e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c420 tx=0x7fc8c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.720+0000 7fc8bffff640 1 -- 172.21.15.55:0/1277554014 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8017070 con 0x7fc8d806b500 2024-05-20T22:10:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.720+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1277554014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d8160c90 con 0x7fc8d806b500 2024-05-20T22:10:18.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.720+0000 7fc8bffff640 1 -- 172.21.15.55:0/1277554014 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c8005ce0 con 0x7fc8d806b500 2024-05-20T22:10:18.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.720+0000 7fc8bffff640 1 -- 172.21.15.55:0/1277554014 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c80072f0 con 0x7fc8d806b500 2024-05-20T22:10:18.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.720+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1277554014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d8161250 con 0x7fc8d806b500 2024-05-20T22:10:18.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.721+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1277554014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d8069090 con 0x7fc8d806b500 2024-05-20T22:10:18.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.722+0000 7fc8bffff640 1 -- 172.21.15.55:0/1277554014 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8c801a070 con 0x7fc8d806b500 2024-05-20T22:10:18.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.722+0000 7fc8bffff640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc8c406be60 0x7fc8c406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:18.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.722+0000 7fc8d6ffd640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc8c406be60 0x7fc8c406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:18.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.722+0000 7fc8bffff640 1 -- 172.21.15.55:0/1277554014 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5421+0+0 (secure 0 0 0) 0x7fc8c808a190 con 0x7fc8d806b500 2024-05-20T22:10:18.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.723+0000 7fc8d6ffd640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc8c406be60 0x7fc8c406e370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc034000 tx=0x7fc8cc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:18.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.727+0000 7fc8bffff640 1 -- 172.21.15.55:0/1277554014 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc8c8058b50 con 0x7fc8d806b500 2024-05-20T22:10:18.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.908+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1277554014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc8d80d2e50 con 0x7fc8d806b500 2024-05-20T22:10:18.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.912+0000 7fc8bffff640 1 -- 172.21.15.55:0/1277554014 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fc8c8058450 con 0x7fc8d806b500 2024-05-20T22:10:18.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1277554014 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc8c406be60 msgr2=0x7fc8c406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:18.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc8c406be60 0x7fc8c406e370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc034000 tx=0x7fc8cc02e000 comp rx=0 tx=0).stop 2024-05-20T22:10:18.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1277554014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d806b500 msgr2=0x7fc8d80c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:18.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d806b500 0x7fc8d80c91e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c420 tx=0x7fc8c800c8e0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1277554014 shutdown_connections 2024-05-20T22:10:18.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc8c406be60 0x7fc8c406e370 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8d815f150 0x7fc8d80b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d32a0 0x7fc8d81533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 --2- 172.21.15.55:0/1277554014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8d806b500 0x7fc8d80c91e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:18.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1277554014 >> 172.21.15.55:0/1277554014 conn(0x7fc8d80b6de0 msgr2=0x7fc8d80b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:18.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1277554014 shutdown_connections 2024-05-20T22:10:18.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:18.914+0000 7fc8ddb4c640 1 -- 172.21.15.55:0/1277554014 wait complete. 2024-05-20T22:10:18.926 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T22:10:18.927 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T22:10:18.972 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T22:10:19.023 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T22:10:19.024 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:10:19.024 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:10:19.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3618 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T22:10:19.040 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:10:19.040 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:10:19.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=450 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T22:10:19.058 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:10:19.058 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:10:19.096 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3618 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T22:10:19.097 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:10:19.097 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:10:19.117 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=450 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T22:10:19.117 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-20T22:10:19.117 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:19.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.250+0000 7f658de06640 1 Processor -- start 2024-05-20T22:10:19.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.250+0000 7f658de06640 1 -- start start 2024-05-20T22:10:19.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.252+0000 7f658de06640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65880b3b30 0x7f65880b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.252+0000 7f658de06640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f658806c790 0x7f65880cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.252+0000 7f658de06640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65880d3dd0 con 0x7f658815f100 2024-05-20T22:10:19.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.252+0000 7f658de06640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f65880d40b0 con 0x7f658806c790 2024-05-20T22:10:19.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.252+0000 7f658de06640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f65880d4390 con 0x7f65880b3b30 2024-05-20T22:10:19.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.252+0000 7f65877fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65880b3b30 0x7f65880b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.252+0000 7f6587fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f658806c790 0x7f65880cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.252+0000 7f65877fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65880b3b30 0x7f65880b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:50678/0 (socket says 172.21.15.55:50678) 2024-05-20T22:10:19.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.252+0000 7f65877fe640 1 -- 172.21.15.55:0/341877798 learned_addr learned my addr 172.21.15.55:0/341877798 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:19.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.253+0000 7f6587fff640 1 -- 172.21.15.55:0/341877798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65880b3b30 msgr2=0x7f65880b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:19.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.253+0000 7f6587fff640 1 --2- 172.21.15.55:0/341877798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65880b3b30 0x7f65880b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.253+0000 7f6587fff640 1 -- 172.21.15.55:0/341877798 >> v1:172.21.15.138:6789/0 conn(0x7f658815f100 legacy=0x7f658806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:19.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.253+0000 7f6587fff640 1 -- 172.21.15.55:0/341877798 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658806b5a0 con 0x7f658806c790 2024-05-20T22:10:19.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.253+0000 7f65877fe640 1 --2- 172.21.15.55:0/341877798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65880b3b30 0x7f65880b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:19.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.253+0000 7f65867fc640 1 -- 172.21.15.55:0/341877798 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2409258786 0 0) 0x7f65880d3dd0 con 0x7f658815f100 2024-05-20T22:10:19.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.253+0000 7f6587fff640 1 --2- 172.21.15.55:0/341877798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658806c790 0x7f65880cf740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6580009d90 tx=0x7f658002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9acf5570d8a9f001 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:19.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f65867fc640 1 -- 172.21.15.55:0/341877798 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580007740 con 0x7f658806c790 2024-05-20T22:10:19.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f65867fc640 1 -- 172.21.15.55:0/341877798 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65800078e0 con 0x7f658806c790 2024-05-20T22:10:19.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f65867fc640 1 -- 172.21.15.55:0/341877798 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580007cb0 con 0x7f658806c790 2024-05-20T22:10:19.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f658de06640 1 -- 172.21.15.55:0/341877798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658806c790 msgr2=0x7f65880cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:19.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f658de06640 1 --2- 172.21.15.55:0/341877798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658806c790 0x7f65880cf740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6580009d90 tx=0x7f658002a660 comp rx=0 tx=0).stop 2024-05-20T22:10:19.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f658de06640 1 -- 172.21.15.55:0/341877798 shutdown_connections 2024-05-20T22:10:19.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f658de06640 1 --2- 172.21.15.55:0/341877798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658806c790 0x7f65880cf740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f658de06640 1 --2- 172.21.15.55:0/341877798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65880b3b30 0x7f65880b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f658de06640 1 -- 172.21.15.55:0/341877798 >> 172.21.15.55:0/341877798 conn(0x7f65880d51d0 msgr2=0x7f65880b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:19.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f658de06640 1 -- 172.21.15.55:0/341877798 shutdown_connections 2024-05-20T22:10:19.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.254+0000 7f658de06640 1 -- 172.21.15.55:0/341877798 wait complete. 2024-05-20T22:10:19.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.256+0000 7f658de06640 1 Processor -- start 2024-05-20T22:10:19.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.256+0000 7f658de06640 1 -- start start 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f658de06640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f658806c790 0x7f65881531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f658de06640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65880b3b30 0x7f65881f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f658de06640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f658815f100 0x7f65881f9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f658de06640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f658806cba0 con 0x7f658815f100 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f658de06640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f658806bb60 con 0x7f65880b3b30 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f658de06640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f658806cd20 con 0x7f658806c790 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f6587fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f658815f100 0x7f65881f9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f6587fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f658815f100 0x7f65881f9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55486/0 (socket says 172.21.15.55:55486) 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f6587fff640 1 -- 172.21.15.55:0/1903205885 learned_addr learned my addr 172.21.15.55:0/1903205885 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f65877fe640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f658806c790 0x7f65881531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f6586ffd640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65880b3b30 0x7f65881f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f6587fff640 1 -- 172.21.15.55:0/1903205885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f658806c790 msgr2=0x7f65881531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f6587fff640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f658806c790 0x7f65881531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f6587fff640 1 -- 172.21.15.55:0/1903205885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65880b3b30 msgr2=0x7f65881f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:19.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f6587fff640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65880b3b30 0x7f65881f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.258+0000 7f6587fff640 1 -- 172.21.15.55:0/1903205885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65881614c0 con 0x7f658815f100 2024-05-20T22:10:19.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.259+0000 7f6587fff640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f658815f100 0x7f65881f9310 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6580009eb0 tx=0x7f6580004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:19.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.259+0000 7f656ffff640 1 -- 172.21.15.55:0/1903205885 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658003f070 con 0x7f658815f100 2024-05-20T22:10:19.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.259+0000 7f656ffff640 1 -- 172.21.15.55:0/1903205885 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6580031bc0 con 0x7f658815f100 2024-05-20T22:10:19.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.259+0000 7f658de06640 1 -- 172.21.15.55:0/1903205885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65881617a0 con 0x7f658815f100 2024-05-20T22:10:19.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.259+0000 7f656ffff640 1 -- 172.21.15.55:0/1903205885 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580009450 con 0x7f658815f100 2024-05-20T22:10:19.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.259+0000 7f658de06640 1 -- 172.21.15.55:0/1903205885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6588161d60 con 0x7f658815f100 2024-05-20T22:10:19.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.260+0000 7f658de06640 1 -- 172.21.15.55:0/1903205885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6588069090 con 0x7f658815f100 2024-05-20T22:10:19.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.260+0000 7f656ffff640 1 -- 172.21.15.55:0/1903205885 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f658003b410 con 0x7f658815f100 2024-05-20T22:10:19.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.261+0000 7f656ffff640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f657406be60 0x7f657406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.261+0000 7f656ffff640 1 -- 172.21.15.55:0/1903205885 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5421+0+0 (secure 0 0 0) 0x7f65800ac350 con 0x7f658815f100 2024-05-20T22:10:19.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.261+0000 7f65877fe640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f657406be60 0x7f657406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.262+0000 7f65877fe640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f657406be60 0x7f657406e370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6578006cc0 tx=0x7f6578009240 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:19.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.266+0000 7f656ffff640 1 -- 172.21.15.55:0/1903205885 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f658007a7d0 con 0x7f658815f100 2024-05-20T22:10:19.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.487+0000 7f658de06640 1 -- 172.21.15.55:0/1903205885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65880d2ee0 con 0x7f658815f100 2024-05-20T22:10:19.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.488+0000 7f656ffff640 1 -- 172.21.15.55:0/1903205885 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3877 (secure 0 0 0) 0x7f658007ead0 con 0x7f658815f100 2024-05-20T22:10:19.489 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:19.489 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3873372908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3873372908},{"type":"v1","addr":"172.21.15.55:6836","nonce":3873372908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3417305427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":3417305427},{"type":"v1","addr":"172.21.15.55:6837","nonce":3417305427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":35081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4211704780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4211704780},{"type":"v1","addr":"172.21.15.138:6833","nonce":4211704780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":35189,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3480486673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3480486673},{"type":"v1","addr":"172.21.15.138:6835","nonce":3480486673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-05-20T22:10:19.489 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 28 2024-05-20T22:10:19.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 -- 172.21.15.55:0/1903205885 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f657406be60 msgr2=0x7f657406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:19.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f657406be60 0x7f657406e370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6578006cc0 tx=0x7f6578009240 comp rx=0 tx=0).stop 2024-05-20T22:10:19.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 -- 172.21.15.55:0/1903205885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f658815f100 msgr2=0x7f65881f9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:19.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f658815f100 0x7f65881f9310 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6580009eb0 tx=0x7f6580004a40 comp rx=0 tx=0).stop 2024-05-20T22:10:19.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 -- 172.21.15.55:0/1903205885 shutdown_connections 2024-05-20T22:10:19.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f657406be60 0x7f657406e370 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f658815f100 0x7f65881f9310 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65880b3b30 0x7f65881f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 --2- 172.21.15.55:0/1903205885 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f658806c790 0x7f65881531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 -- 172.21.15.55:0/1903205885 >> 172.21.15.55:0/1903205885 conn(0x7f65880d51d0 msgr2=0x7f6588162ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:19.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 -- 172.21.15.55:0/1903205885 shutdown_connections 2024-05-20T22:10:19.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.490+0000 7f658de06640 1 -- 172.21.15.55:0/1903205885 wait complete. 2024-05-20T22:10:19.503 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T22:10:19.503 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T22:10:19.503 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T22:10:19.503 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:10:19.503 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T22:10:19.503 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T22:10:19.504 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T22:10:19.504 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:10:19.504 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T22:10:19.504 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T22:10:19.504 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T22:10:19.504 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:10:19.504 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T22:10:19.505 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T22:10:19.505 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T22:10:19.505 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:10:19.516 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T22:10:19.516 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T22:10:19.517 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T22:10:19.517 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T22:10:19.519 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T22:10:19.519 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T22:10:19.520 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T22:10:19.520 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T22:10:19.559 INFO:tasks.ceph.mds.a:Started 2024-05-20T22:10:19.560 DEBUG:teuthology.parallel:result is None 2024-05-20T22:10:19.560 INFO:tasks.ceph.mds.c:Started 2024-05-20T22:10:19.560 DEBUG:teuthology.parallel:result is None 2024-05-20T22:10:19.562 INFO:tasks.ceph.mds.d:Started 2024-05-20T22:10:19.563 DEBUG:teuthology.parallel:result is None 2024-05-20T22:10:19.563 INFO:tasks.ceph.mds.b:Started 2024-05-20T22:10:19.563 DEBUG:teuthology.parallel:result is None 2024-05-20T22:10:19.564 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-20T22:10:19.652 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-20T22:10:19.653 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-20T22:10:19.663 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-05-20T22:10:19.663 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-05-20T22:10:19.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.686+0000 7fcedf275640 1 Processor -- start 2024-05-20T22:10:19.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.687+0000 7fcedf275640 1 -- start start 2024-05-20T22:10:19.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedf275640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fced80c4c60 0x7fced80c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedf275640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced80b2ca0 0x7fced80b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedf275640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced80b35f0 con 0x7fced815d220 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedf275640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fced80c55b0 con 0x7fced80c4c60 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedf275640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fced806b500 con 0x7fced80b2ca0 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedc810640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fced80c4c60 0x7fced80c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedc810640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fced80c4c60 0x7fced80c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55518/0 (socket says 172.21.15.55:55518) 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedc810640 1 -- 172.21.15.55:0/3186676374 learned_addr learned my addr 172.21.15.55:0/3186676374 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedd812640 1 --2- 172.21.15.55:0/3186676374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced80b2ca0 0x7fced80b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedc810640 1 -- 172.21.15.55:0/3186676374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced80b2ca0 msgr2=0x7fced80b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedc810640 1 --2- 172.21.15.55:0/3186676374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced80b2ca0 0x7fced80b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedc810640 1 -- 172.21.15.55:0/3186676374 >> v1:172.21.15.138:6789/0 conn(0x7fced815d220 legacy=0x7fced806c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.688+0000 7fcedc810640 1 -- 172.21.15.55:0/3186676374 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced806b7e0 con 0x7fced80c4c60 2024-05-20T22:10:19.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.689+0000 7fcedd812640 1 --2- 172.21.15.55:0/3186676374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced80b2ca0 0x7fced80b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.689+0000 7fcedc810640 1 --2- 172.21.15.55:0/3186676374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 0x7fced80c5070 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fced80d0fb0 tx=0x7fcec802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fd2bb6328d5f7d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.689+0000 7fcecffff640 1 -- 172.21.15.55:0/3186676374 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8037070 con 0x7fced80c4c60 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.689+0000 7fcecffff640 1 -- 172.21.15.55:0/3186676374 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec8032440 con 0x7fced80c4c60 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.689+0000 7fcecffff640 1 -- 172.21.15.55:0/3186676374 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8032810 con 0x7fced80c4c60 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.690+0000 7fcedf275640 1 -- 172.21.15.55:0/3186676374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 msgr2=0x7fced80c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.690+0000 7fcedf275640 1 --2- 172.21.15.55:0/3186676374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 0x7fced80c5070 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fced80d0fb0 tx=0x7fcec802a270 comp rx=0 tx=0).stop 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.690+0000 7fcedf275640 1 -- 172.21.15.55:0/3186676374 shutdown_connections 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.690+0000 7fcedf275640 1 --2- 172.21.15.55:0/3186676374 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced80b2ca0 0x7fced80b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.690+0000 7fcedf275640 1 --2- 172.21.15.55:0/3186676374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 0x7fced80c5070 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.690+0000 7fcedf275640 1 -- 172.21.15.55:0/3186676374 >> 172.21.15.55:0/3186676374 conn(0x7fced80b62c0 msgr2=0x7fced809a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.690+0000 7fcedf275640 1 -- 172.21.15.55:0/3186676374 shutdown_connections 2024-05-20T22:10:19.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.690+0000 7fcedf275640 1 -- 172.21.15.55:0/3186676374 wait complete. 2024-05-20T22:10:19.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.692+0000 7fcedf275640 1 Processor -- start 2024-05-20T22:10:19.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.692+0000 7fcedf275640 1 -- start start 2024-05-20T22:10:19.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.693+0000 7fcedf275640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced80b2ca0 0x7fced80ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.693+0000 7fcedf275640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 0x7fced80ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.693+0000 7fcedf275640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced815d220 0x7fced80cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.693+0000 7fcedf275640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fced806c080 con 0x7fced80c4c60 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.693+0000 7fcedf275640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fced806bda0 con 0x7fced80b2ca0 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.693+0000 7fcedf275640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fced806ccb0 con 0x7fced815d220 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.693+0000 7fcedc810640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 0x7fced80ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedc810640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 0x7fced80ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55526/0 (socket says 172.21.15.55:55526) 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedc810640 1 -- 172.21.15.55:0/2186853360 learned_addr learned my addr 172.21.15.55:0/2186853360 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedd812640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced815d220 0x7fced80cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedd011640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced80b2ca0 0x7fced80ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedc810640 1 -- 172.21.15.55:0/2186853360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced815d220 msgr2=0x7fced80cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedc810640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced815d220 0x7fced80cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedc810640 1 -- 172.21.15.55:0/2186853360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced80b2ca0 msgr2=0x7fced80ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:19.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedc810640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced80b2ca0 0x7fced80ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:19.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedc810640 1 -- 172.21.15.55:0/2186853360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced815d850 con 0x7fced80c4c60 2024-05-20T22:10:19.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedc810640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 0x7fced80ca910 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcec802a2b0 tx=0x7fcec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:19.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcecdffb640 1 -- 172.21.15.55:0/2186853360 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8041070 con 0x7fced80c4c60 2024-05-20T22:10:19.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcecdffb640 1 -- 172.21.15.55:0/2186853360 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec803c870 con 0x7fced80c4c60 2024-05-20T22:10:19.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcedf275640 1 -- 172.21.15.55:0/2186853360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced8160580 con 0x7fced80c4c60 2024-05-20T22:10:19.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.694+0000 7fcecdffb640 1 -- 172.21.15.55:0/2186853360 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8037040 con 0x7fced80c4c60 2024-05-20T22:10:19.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.695+0000 7fcedf275640 1 -- 172.21.15.55:0/2186853360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced815f7c0 con 0x7fced80c4c60 2024-05-20T22:10:19.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.695+0000 7fcedf275640 1 -- 172.21.15.55:0/2186853360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced815f250 con 0x7fced80c4c60 2024-05-20T22:10:19.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.696+0000 7fcecdffb640 1 -- 172.21.15.55:0/2186853360 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcec8044070 con 0x7fced80c4c60 2024-05-20T22:10:19.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.697+0000 7fcecdffb640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcebc06be30 0x7fcebc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:19.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.697+0000 7fcecdffb640 1 -- 172.21.15.55:0/2186853360 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5421+0+0 (secure 0 0 0) 0x7fcec80ab9f0 con 0x7fced80c4c60 2024-05-20T22:10:19.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.697+0000 7fcedd011640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcebc06be30 0x7fcebc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:19.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.697+0000 7fcedd011640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcebc06be30 0x7fcebc06e340 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fced4006fd0 tx=0x7fced4009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:19.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.701+0000 7fcecdffb640 1 -- 172.21.15.55:0/2186853360 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fced80b35f0 con 0x7fced80c4c60 2024-05-20T22:10:19.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:19.897+0000 7fcedf275640 1 -- 172.21.15.55:0/2186853360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fced8069090 con 0x7fced80c4c60 2024-05-20T22:10:20.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.586+0000 7fcecdffb640 1 -- 172.21.15.55:0/2186853360 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) ==== 97+0+0 (secure 0 0 0) 0x7fcec807a020 con 0x7fced80c4c60 2024-05-20T22:10:20.587 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2024-05-20T22:10:20.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.589+0000 7fcedf275640 1 -- 172.21.15.55:0/2186853360 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcebc06be30 msgr2=0x7fcebc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.589+0000 7fcedf275640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcebc06be30 0x7fcebc06e340 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fced4006fd0 tx=0x7fced4009060 comp rx=0 tx=0).stop 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.589+0000 7fcedf275640 1 -- 172.21.15.55:0/2186853360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 msgr2=0x7fced80ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.589+0000 7fcedf275640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 0x7fced80ca910 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcec802a2b0 tx=0x7fcec80046c0 comp rx=0 tx=0).stop 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.589+0000 7fcedf275640 1 -- 172.21.15.55:0/2186853360 shutdown_connections 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.589+0000 7fcedf275640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fcebc06be30 0x7fcebc06e340 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.589+0000 7fcedf275640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fced815d220 0x7fced80cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.589+0000 7fcedf275640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fced80c4c60 0x7fced80ca910 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.589+0000 7fcedf275640 1 --2- 172.21.15.55:0/2186853360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced80b2ca0 0x7fced80ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.590+0000 7fcedf275640 1 -- 172.21.15.55:0/2186853360 >> 172.21.15.55:0/2186853360 conn(0x7fced80b62c0 msgr2=0x7fced8070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.590+0000 7fcedf275640 1 -- 172.21.15.55:0/2186853360 shutdown_connections 2024-05-20T22:10:20.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.590+0000 7fcedf275640 1 -- 172.21.15.55:0/2186853360 wait complete. 2024-05-20T22:10:20.606 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-20T22:10:20.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.727+0000 7f726ee56640 1 Processor -- start 2024-05-20T22:10:20.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.727+0000 7f726ee56640 1 -- start start 2024-05-20T22:10:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.728+0000 7f726ee56640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f726815f100 0x7f72680b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.728+0000 7f726ee56640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f726806c790 0x7f72680cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.728+0000 7f726ee56640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72680d3dd0 con 0x7f72680b3b30 2024-05-20T22:10:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.728+0000 7f726ee56640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f72680d40b0 con 0x7f726815f100 2024-05-20T22:10:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.728+0000 7f726ee56640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f72680d4390 con 0x7f726806c790 2024-05-20T22:10:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.728+0000 7f725ffff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f726815f100 0x7f72680b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.729+0000 7f726cbf2640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f72680b3b30 0x7f726806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35408/0 (socket says 172.21.15.55:35408) 2024-05-20T22:10:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.729+0000 7f726cbf2640 1 -- 172.21.15.55:0/2924177408 learned_addr learned my addr 172.21.15.55:0/2924177408 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.729+0000 7f726d3f3640 1 --2- 172.21.15.55:0/2924177408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f726806c790 0x7f72680cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.729+0000 7f725f7fe640 1 -- 172.21.15.55:0/2924177408 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158840536 0 0) 0x7f72680d3dd0 con 0x7f72680b3b30 2024-05-20T22:10:20.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.729+0000 7f725f7fe640 1 -- 172.21.15.55:0/2924177408 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7250003520 con 0x7f72680b3b30 2024-05-20T22:10:20.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.729+0000 7f725ffff640 1 -- 172.21.15.55:0/2924177408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f726806c790 msgr2=0x7f72680cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:20.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.729+0000 7f725ffff640 1 --2- 172.21.15.55:0/2924177408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f726806c790 0x7f72680cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:20.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.729+0000 7f725ffff640 1 -- 172.21.15.55:0/2924177408 >> v1:172.21.15.55:6789/0 conn(0x7f72680b3b30 legacy=0x7f726806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:20.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.729+0000 7f725ffff640 1 -- 172.21.15.55:0/2924177408 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726806b5a0 con 0x7f726815f100 2024-05-20T22:10:20.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.730+0000 7f725ffff640 1 --2- 172.21.15.55:0/2924177408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 0x7f72680b7fe0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f72580097a0 tx=0x7f725802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e4b74dddeb4540 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:20.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.730+0000 7f725f7fe640 1 -- 172.21.15.55:0/2924177408 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258037070 con 0x7f726815f100 2024-05-20T22:10:20.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.730+0000 7f725f7fe640 1 -- 172.21.15.55:0/2924177408 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7258032440 con 0x7f726815f100 2024-05-20T22:10:20.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.730+0000 7f725f7fe640 1 -- 172.21.15.55:0/2924177408 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258032810 con 0x7f726815f100 2024-05-20T22:10:20.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.731+0000 7f726ee56640 1 -- 172.21.15.55:0/2924177408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 msgr2=0x7f72680b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:20.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.731+0000 7f726ee56640 1 --2- 172.21.15.55:0/2924177408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 0x7f72680b7fe0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f72580097a0 tx=0x7f725802a180 comp rx=0 tx=0).stop 2024-05-20T22:10:20.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.731+0000 7f726ee56640 1 -- 172.21.15.55:0/2924177408 shutdown_connections 2024-05-20T22:10:20.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.731+0000 7f726ee56640 1 --2- 172.21.15.55:0/2924177408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f726806c790 0x7f72680cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:20.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.731+0000 7f726ee56640 1 --2- 172.21.15.55:0/2924177408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 0x7f72680b7fe0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:20.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.731+0000 7f726ee56640 1 -- 172.21.15.55:0/2924177408 >> 172.21.15.55:0/2924177408 conn(0x7f72680d51d0 msgr2=0x7f72680b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:20.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.731+0000 7f726ee56640 1 -- 172.21.15.55:0/2924177408 shutdown_connections 2024-05-20T22:10:20.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.731+0000 7f726ee56640 1 -- 172.21.15.55:0/2924177408 wait complete. 2024-05-20T22:10:20.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.733+0000 7f726ee56640 1 Processor -- start 2024-05-20T22:10:20.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.733+0000 7f726ee56640 1 -- start start 2024-05-20T22:10:20.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f726ee56640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f726806c790 0x7f72680c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:20.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f726ee56640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72680b3b30 0x7f72681575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:20.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f726ee56640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 0x7f72680b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:20.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f726ee56640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f726806cba0 con 0x7f726815f100 2024-05-20T22:10:20.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f726ee56640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f726806bb60 con 0x7f726806c790 2024-05-20T22:10:20.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f726ee56640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f726806cd20 con 0x7f72680b3b30 2024-05-20T22:10:20.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f726d3f3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 0x7f72680b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:20.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f726d3f3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 0x7f72680b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55546/0 (socket says 172.21.15.55:55546) 2024-05-20T22:10:20.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f726d3f3640 1 -- 172.21.15.55:0/4185021766 learned_addr learned my addr 172.21.15.55:0/4185021766 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f725ffff640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72680b3b30 0x7f72681575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.734+0000 7f726cbf2640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f726806c790 0x7f72680c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f726d3f3640 1 -- 172.21.15.55:0/4185021766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72680b3b30 msgr2=0x7f72681575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f726d3f3640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72680b3b30 0x7f72681575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f726d3f3640 1 -- 172.21.15.55:0/4185021766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f726806c790 msgr2=0x7f72680c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f726d3f3640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f726806c790 0x7f72680c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f726d3f3640 1 -- 172.21.15.55:0/4185021766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72681595f0 con 0x7f726815f100 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f726d3f3640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 0x7f72680b53e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7260007e90 tx=0x7f726000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f725d7fa640 1 -- 172.21.15.55:0/4185021766 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7260014420 con 0x7f726815f100 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f725d7fa640 1 -- 172.21.15.55:0/4185021766 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7260019070 con 0x7f726815f100 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f726ee56640 1 -- 172.21.15.55:0/4185021766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72681598d0 con 0x7f726815f100 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f725d7fa640 1 -- 172.21.15.55:0/4185021766 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7260014800 con 0x7f726815f100 2024-05-20T22:10:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.735+0000 7f726ee56640 1 -- 172.21.15.55:0/4185021766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7268159e90 con 0x7f726815f100 2024-05-20T22:10:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.736+0000 7f726ee56640 1 -- 172.21.15.55:0/4185021766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72681494e0 con 0x7f726815f100 2024-05-20T22:10:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.737+0000 7f725d7fa640 1 -- 172.21.15.55:0/4185021766 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72600149a0 con 0x7f726815f100 2024-05-20T22:10:20.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.737+0000 7f725d7fa640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f725406be30 0x7f725406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:20.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.737+0000 7f725d7fa640 1 -- 172.21.15.55:0/4185021766 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5249+0+0 (secure 0 0 0) 0x7f726008b370 con 0x7f726815f100 2024-05-20T22:10:20.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.737+0000 7f726cbf2640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f725406be30 0x7f725406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:20.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.738+0000 7f726cbf2640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f725406be30 0x7f725406e340 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7264005880 tx=0x7f7264009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:20.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:20.742+0000 7f725d7fa640 1 -- 172.21.15.55:0/4185021766 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7260059890 con 0x7f726815f100 2024-05-20T22:10:21.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.002+0000 7f726ee56640 1 -- 172.21.15.55:0/4185021766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7268069090 con 0x7f726815f100 2024-05-20T22:10:21.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.002+0000 7f725d7fa640 1 -- 172.21.15.55:0/4185021766 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f726005db90 con 0x7f726815f100 2024-05-20T22:10:21.003 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:21.003 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmnF7NJRAAIHaudq8A1NdLVw4llE1GKA==", 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2024-05-20T22:10:21.004 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmNDXWMhAAQU8KHijh4cVysa71xLvMjg==", 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmKe+YLRAAagJbIVjyJ+R1oZEBhZyGiQ==", 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:10:21.005 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtm6mtqOhAA6mTeQoWYP+yUvHt/szLnlw==", 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.006 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB8yUtm5SdYLhAAleTxIMKS9IsQjv4FusrryA==", 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB/yUtm7zGqLhAAU1uR214d/bSdRlcl5frlYA==", 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.007 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCCyUtmwBfXLxAAwRbdPpoNxTIoCSDsNdZOKQ==", 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.008 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCFyUtmTcaaMBAAQCJ6epS4s2FPsSvRBFdcLg==", 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCLyUtmNsNGNRAAQA8pn/2GQ4FPNxu3zsS9JQ==", 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.009 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCOyUtmdmFLNBAAUH+yBiv2QSbfBqob0d6X8Q==", 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2024-05-20T22:10:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCRyUtmVZGkMhAAmI118TqDPXtUvsH8We4hyg==", 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCUyUtmNmlRMRAAUL0mlbzjAXGdx7oOZeKQ/w==", 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.011 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB2yUtm592RAxAAiaDYBHgkpVfsQHm+jLOy3g==", 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB2yUtmNXZWCxAArgL/Kf6oP/vHiIhC/4f/kg==", 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:10:21.012 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB2yUtmDwN6BxAAuS4b7g8QxESuhrW9d39wMQ==", 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.013 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB2yUtmIsoxDxAAdHDRYPZ5HeYfqye4wIR2wQ==", 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmTdmWBxAA2yyudY+ao2TWVP52VQjobg==", 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.014 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-20T22:10:21.015 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-20T22:10:21.015 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-20T22:10:21.015 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.015 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.015 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.015 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.015 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2024-05-20T22:10:21.015 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmwQ/lIRAAuSUOiqQA/GU0trg5dUB8Sg==", 2024-05-20T22:10:21.015 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.015 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-20T22:10:21.016 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-20T22:10:21.016 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.016 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.017 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:21.017 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:21.017 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2024-05-20T22:10:21.017 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1yUtmXhIUHRAAzFxrH0wPo95yWm4KoEFLvw==", 2024-05-20T22:10:21.017 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-20T22:10:21.018 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-20T22:10:21.018 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-20T22:10:21.018 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-20T22:10:21.019 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.019 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:21.019 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-05-20T22:10:21.019 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:10:21.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 -- 172.21.15.55:0/4185021766 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f725406be30 msgr2=0x7f725406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f725406be30 0x7f725406e340 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7264005880 tx=0x7f7264009040 comp rx=0 tx=0).stop 2024-05-20T22:10:21.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 -- 172.21.15.55:0/4185021766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 msgr2=0x7f72680b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 0x7f72680b53e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7260007e90 tx=0x7f726000e4b0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 -- 172.21.15.55:0/4185021766 shutdown_connections 2024-05-20T22:10:21.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f725406be30 0x7f725406e340 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726815f100 0x7f72680b53e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72680b3b30 0x7f72681575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 --2- 172.21.15.55:0/4185021766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f726806c790 0x7f72680c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 -- 172.21.15.55:0/4185021766 >> 172.21.15.55:0/4185021766 conn(0x7f72680d51d0 msgr2=0x7f72680b9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:21.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 -- 172.21.15.55:0/4185021766 shutdown_connections 2024-05-20T22:10:21.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.005+0000 7f726ee56640 1 -- 172.21.15.55:0/4185021766 wait complete. 2024-05-20T22:10:21.025 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:10:21.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.139+0000 7f61d3796640 1 Processor -- start 2024-05-20T22:10:21.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.139+0000 7f61d3796640 1 -- start start 2024-05-20T22:10:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d3796640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f61cc06ffc0 0x7f61cc0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d3796640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc06c4a0 0x7f61cc0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d3796640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61cc0d03f0 con 0x7f61cc1450a0 2024-05-20T22:10:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d3796640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f61cc15fc00 con 0x7f61cc06ffc0 2024-05-20T22:10:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d3796640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc15f2d0 con 0x7f61cc06c4a0 2024-05-20T22:10:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d1d33640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc06c4a0 0x7f61cc0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d1532640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f61cc06ffc0 0x7f61cc0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d0d31640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f61cc1450a0 0x7f61cc06bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:39302/0 (socket says 172.21.15.55:39302) 2024-05-20T22:10:21.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d0d31640 1 -- 172.21.15.55:0/743467062 learned_addr learned my addr 172.21.15.55:0/743467062 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:21.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d1d33640 1 -- 172.21.15.55:0/743467062 >> v2:172.21.15.138:3300/0 conn(0x7f61cc06ffc0 msgr2=0x7f61cc0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:21.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d1d33640 1 --2- 172.21.15.55:0/743467062 >> v2:172.21.15.138:3300/0 conn(0x7f61cc06ffc0 0x7f61cc0cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d1d33640 1 -- 172.21.15.55:0/743467062 >> v1:172.21.15.138:6789/0 conn(0x7f61cc1450a0 legacy=0x7f61cc06bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d1d33640 1 -- 172.21.15.55:0/743467062 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc15f5b0 con 0x7f61cc06c4a0 2024-05-20T22:10:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.141+0000 7f61d1532640 1 --2- 172.21.15.55:0/743467062 >> v2:172.21.15.138:3300/0 conn(0x7f61cc06ffc0 0x7f61cc0cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.142+0000 7f61d1d33640 1 --2- 172.21.15.55:0/743467062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc06c4a0 0x7f61cc0ce230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009d90 tx=0x7f61c402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79e9d39ee4e34e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.142+0000 7f61c3fff640 1 -- 172.21.15.55:0/743467062 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4007740 con 0x7f61cc06c4a0 2024-05-20T22:10:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.142+0000 7f61c3fff640 1 -- 172.21.15.55:0/743467062 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61c40078e0 con 0x7f61cc06c4a0 2024-05-20T22:10:21.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.142+0000 7f61c3fff640 1 -- 172.21.15.55:0/743467062 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4007cb0 con 0x7f61cc06c4a0 2024-05-20T22:10:21.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.143+0000 7f61d3796640 1 -- 172.21.15.55:0/743467062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc06c4a0 msgr2=0x7f61cc0ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.143+0000 7f61d3796640 1 --2- 172.21.15.55:0/743467062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc06c4a0 0x7f61cc0ce230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009d90 tx=0x7f61c402a6f0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.143+0000 7f61d3796640 1 -- 172.21.15.55:0/743467062 shutdown_connections 2024-05-20T22:10:21.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.143+0000 7f61d3796640 1 --2- 172.21.15.55:0/743467062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc06c4a0 0x7f61cc0ce230 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.143+0000 7f61d3796640 1 --2- 172.21.15.55:0/743467062 >> v2:172.21.15.138:3300/0 conn(0x7f61cc06ffc0 0x7f61cc0cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.143+0000 7f61d3796640 1 -- 172.21.15.55:0/743467062 >> 172.21.15.55:0/743467062 conn(0x7f61cc0cd4d0 msgr2=0x7f61cc0cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:21.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.143+0000 7f61d3796640 1 -- 172.21.15.55:0/743467062 shutdown_connections 2024-05-20T22:10:21.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.143+0000 7f61d3796640 1 -- 172.21.15.55:0/743467062 wait complete. 2024-05-20T22:10:21.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.145+0000 7f61d3796640 1 Processor -- start 2024-05-20T22:10:21.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.145+0000 7f61d3796640 1 -- start start 2024-05-20T22:10:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.146+0000 7f61d3796640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61cc06ffc0 0x7f61cc15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.146+0000 7f61d3796640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61cc1450a0 0x7f61cc15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.146+0000 7f61d3796640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc15a820 0x7f61cc1f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.146+0000 7f61d3796640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc160090 con 0x7f61cc1450a0 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.146+0000 7f61d3796640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc1454b0 con 0x7f61cc06ffc0 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.146+0000 7f61d3796640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc0ccd90 con 0x7f61cc15a820 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.146+0000 7f61d0d31640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61cc1450a0 0x7f61cc15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.146+0000 7f61d0d31640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61cc1450a0 0x7f61cc15c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55558/0 (socket says 172.21.15.55:55558) 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.146+0000 7f61d0d31640 1 -- 172.21.15.55:0/2937438782 learned_addr learned my addr 172.21.15.55:0/2937438782 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61d1d33640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc15a820 0x7f61cc1f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61d1532640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61cc06ffc0 0x7f61cc15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61d0d31640 1 -- 172.21.15.55:0/2937438782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc15a820 msgr2=0x7f61cc1f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61d0d31640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc15a820 0x7f61cc1f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61d0d31640 1 -- 172.21.15.55:0/2937438782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61cc06ffc0 msgr2=0x7f61cc15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61d0d31640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61cc06ffc0 0x7f61cc15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61d0d31640 1 -- 172.21.15.55:0/2937438782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc15dcb0 con 0x7f61cc1450a0 2024-05-20T22:10:21.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61d0d31640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61cc1450a0 0x7f61cc15c810 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00c3c0 tx=0x7f61bc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:21.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61c1ffb640 1 -- 172.21.15.55:0/2937438782 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc017070 con 0x7f61cc1450a0 2024-05-20T22:10:21.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61c1ffb640 1 -- 172.21.15.55:0/2937438782 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61bc005ce0 con 0x7f61cc1450a0 2024-05-20T22:10:21.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61d3796640 1 -- 172.21.15.55:0/2937438782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc15df90 con 0x7f61cc1450a0 2024-05-20T22:10:21.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61c1ffb640 1 -- 172.21.15.55:0/2937438782 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0072f0 con 0x7f61cc1450a0 2024-05-20T22:10:21.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.147+0000 7f61d3796640 1 -- 172.21.15.55:0/2937438782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc15c010 con 0x7f61cc1450a0 2024-05-20T22:10:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.149+0000 7f61d3796640 1 -- 172.21.15.55:0/2937438782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61cc069090 con 0x7f61cc1450a0 2024-05-20T22:10:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.149+0000 7f61c1ffb640 1 -- 172.21.15.55:0/2937438782 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61bc01a070 con 0x7f61cc1450a0 2024-05-20T22:10:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.150+0000 7f61c1ffb640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f61b006be60 0x7f61b006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.150+0000 7f61c1ffb640 1 -- 172.21.15.55:0/2937438782 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5249+0+0 (secure 0 0 0) 0x7f61bc08a110 con 0x7f61cc1450a0 2024-05-20T22:10:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.150+0000 7f61d1532640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f61b006be60 0x7f61b006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.150+0000 7f61d1532640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f61b006be60 0x7f61b006e370 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009a30 tx=0x7f61c8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:21.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.154+0000 7f61c1ffb640 1 -- 172.21.15.55:0/2937438782 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f61bc058c00 con 0x7f61cc1450a0 2024-05-20T22:10:21.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.338+0000 7f61d3796640 1 -- 172.21.15.55:0/2937438782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61cc15da40 con 0x7f61cc1450a0 2024-05-20T22:10:21.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.339+0000 7f61c1ffb640 1 -- 172.21.15.55:0/2937438782 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+13164 (secure 0 0 0) 0x7f61bc058500 con 0x7f61cc1450a0 2024-05-20T22:10:21.340 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:21.340 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":51,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:10:20.598948+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:10:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 -- 172.21.15.55:0/2937438782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f61b006be60 msgr2=0x7f61b006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f61b006be60 0x7f61b006e370 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009a30 tx=0x7f61c8008040 comp rx=0 tx=0).stop 2024-05-20T22:10:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 -- 172.21.15.55:0/2937438782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61cc1450a0 msgr2=0x7f61cc15c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61cc1450a0 0x7f61cc15c810 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00c3c0 tx=0x7f61bc00c7c0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 -- 172.21.15.55:0/2937438782 shutdown_connections 2024-05-20T22:10:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f61b006be60 0x7f61b006e370 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61cc15a820 0x7f61cc1f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61cc1450a0 0x7f61cc15c810 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 --2- 172.21.15.55:0/2937438782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61cc06ffc0 0x7f61cc15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 -- 172.21.15.55:0/2937438782 >> 172.21.15.55:0/2937438782 conn(0x7f61cc0cd4d0 msgr2=0x7f61cc06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:21.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 -- 172.21.15.55:0/2937438782 shutdown_connections 2024-05-20T22:10:21.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.342+0000 7f61d3796640 1 -- 172.21.15.55:0/2937438782 wait complete. 2024-05-20T22:10:21.354 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T22:10:21.355 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:10:21.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.472+0000 7fb53b709640 1 Processor -- start 2024-05-20T22:10:21.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.472+0000 7fb53b709640 1 -- start start 2024-05-20T22:10:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.473+0000 7fb53b709640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb5340d32a0 0x7fb53415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.473+0000 7fb53b709640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb5340d01b0 0x7fb53406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb53b709640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5340d3780 con 0x7fb5340cf690 2024-05-20T22:10:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb53b709640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fb5340d3a60 con 0x7fb5340d01b0 2024-05-20T22:10:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb53b709640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb5340d3d40 con 0x7fb5340d32a0 2024-05-20T22:10:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb539ca6640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb5340d01b0 0x7fb53406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb538ca4640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb5340cf690 0x7fb5340cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35410/0 (socket says 172.21.15.55:35410) 2024-05-20T22:10:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb538ca4640 1 -- 172.21.15.55:0/3109327500 learned_addr learned my addr 172.21.15.55:0/3109327500 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb5394a5640 1 --2- 172.21.15.55:0/3109327500 >> v2:172.21.15.138:3300/0 conn(0x7fb5340d32a0 0x7fb53415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb539ca6640 1 -- 172.21.15.55:0/3109327500 >> v2:172.21.15.138:3300/0 conn(0x7fb5340d32a0 msgr2=0x7fb53415f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb539ca6640 1 --2- 172.21.15.55:0/3109327500 >> v2:172.21.15.138:3300/0 conn(0x7fb5340d32a0 0x7fb53415f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb539ca6640 1 -- 172.21.15.55:0/3109327500 >> v1:172.21.15.55:6789/0 conn(0x7fb5340cf690 legacy=0x7fb5340cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb533fff640 1 -- 172.21.15.55:0/3109327500 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477324411 0 0) 0x7fb5340d3780 con 0x7fb5340cf690 2024-05-20T22:10:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.474+0000 7fb539ca6640 1 -- 172.21.15.55:0/3109327500 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5340d4020 con 0x7fb5340d01b0 2024-05-20T22:10:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.475+0000 7fb539ca6640 1 --2- 172.21.15.55:0/3109327500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 0x7fb53406b500 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb5340d36b0 tx=0x7fb52c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce5bea74afaed57 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.475+0000 7fb533fff640 1 -- 172.21.15.55:0/3109327500 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c037070 con 0x7fb5340d01b0 2024-05-20T22:10:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.475+0000 7fb533fff640 1 -- 172.21.15.55:0/3109327500 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52c032440 con 0x7fb5340d01b0 2024-05-20T22:10:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.475+0000 7fb533fff640 1 -- 172.21.15.55:0/3109327500 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c032810 con 0x7fb5340d01b0 2024-05-20T22:10:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.476+0000 7fb53b709640 1 -- 172.21.15.55:0/3109327500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 msgr2=0x7fb53406b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.476+0000 7fb53b709640 1 --2- 172.21.15.55:0/3109327500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 0x7fb53406b500 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb5340d36b0 tx=0x7fb52c02a1a0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.476+0000 7fb53b709640 1 -- 172.21.15.55:0/3109327500 shutdown_connections 2024-05-20T22:10:21.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.476+0000 7fb53b709640 1 --2- 172.21.15.55:0/3109327500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 0x7fb53406b500 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.476+0000 7fb53b709640 1 --2- 172.21.15.55:0/3109327500 >> v2:172.21.15.138:3300/0 conn(0x7fb5340d32a0 0x7fb53415f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.476+0000 7fb53b709640 1 -- 172.21.15.55:0/3109327500 >> 172.21.15.55:0/3109327500 conn(0x7fb5340b6de0 msgr2=0x7fb5340d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:21.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.476+0000 7fb53b709640 1 -- 172.21.15.55:0/3109327500 shutdown_connections 2024-05-20T22:10:21.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.476+0000 7fb53b709640 1 -- 172.21.15.55:0/3109327500 wait complete. 2024-05-20T22:10:21.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.478+0000 7fb53b709640 1 Processor -- start 2024-05-20T22:10:21.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.478+0000 7fb53b709640 1 -- start start 2024-05-20T22:10:21.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb53b709640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5340cf690 0x7fb534158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb53b709640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 0x7fb5341f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb53b709640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5340d32a0 0x7fb5340cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb53b709640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb5340d4820 con 0x7fb5340d01b0 2024-05-20T22:10:21.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb53b709640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5340b4350 con 0x7fb5340d32a0 2024-05-20T22:10:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb53b709640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb5341f6140 con 0x7fb5340cf690 2024-05-20T22:10:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb538ca4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 0x7fb5341f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb538ca4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 0x7fb5341f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55576/0 (socket says 172.21.15.55:55576) 2024-05-20T22:10:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb538ca4640 1 -- 172.21.15.55:0/4013437921 learned_addr learned my addr 172.21.15.55:0/4013437921 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb5394a5640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5340cf690 0x7fb534158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb539ca6640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5340d32a0 0x7fb5340cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb538ca4640 1 -- 172.21.15.55:0/4013437921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5340cf690 msgr2=0x7fb534158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb538ca4640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5340cf690 0x7fb534158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb538ca4640 1 -- 172.21.15.55:0/4013437921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5340d32a0 msgr2=0x7fb5340cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb538ca4640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5340d32a0 0x7fb5340cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb538ca4640 1 -- 172.21.15.55:0/4013437921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5340ce000 con 0x7fb5340d01b0 2024-05-20T22:10:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb5394a5640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5340cf690 0x7fb534158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.479+0000 7fb539ca6640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5340d32a0 0x7fb5340cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.480+0000 7fb538ca4640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 0x7fb5341f5c00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb52800c430 tx=0x7fb52800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.480+0000 7fb531ffb640 1 -- 172.21.15.55:0/4013437921 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5280074e0 con 0x7fb5340d01b0 2024-05-20T22:10:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.480+0000 7fb531ffb640 1 -- 172.21.15.55:0/4013437921 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5280047e0 con 0x7fb5340d01b0 2024-05-20T22:10:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.480+0000 7fb53b709640 1 -- 172.21.15.55:0/4013437921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5340ce2e0 con 0x7fb5340d01b0 2024-05-20T22:10:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.480+0000 7fb531ffb640 1 -- 172.21.15.55:0/4013437921 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528004bd0 con 0x7fb5340d01b0 2024-05-20T22:10:21.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.480+0000 7fb53b709640 1 -- 172.21.15.55:0/4013437921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53415a650 con 0x7fb5340d01b0 2024-05-20T22:10:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.481+0000 7fb53b709640 1 -- 172.21.15.55:0/4013437921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb53415a190 con 0x7fb5340d01b0 2024-05-20T22:10:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.482+0000 7fb531ffb640 1 -- 172.21.15.55:0/4013437921 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb528011070 con 0x7fb5340d01b0 2024-05-20T22:10:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.482+0000 7fb531ffb640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb52006be60 0x7fb52006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.482+0000 7fb531ffb640 1 -- 172.21.15.55:0/4013437921 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5249+0+0 (secure 0 0 0) 0x7fb52808a940 con 0x7fb5340d01b0 2024-05-20T22:10:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.482+0000 7fb5394a5640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb52006be60 0x7fb52006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.483+0000 7fb5394a5640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb52006be60 0x7fb52006e370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb524009f50 tx=0x7fb52400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:21.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.487+0000 7fb531ffb640 1 -- 172.21.15.55:0/4013437921 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb53415a190 con 0x7fb5340d01b0 2024-05-20T22:10:21.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.674+0000 7fb53b709640 1 -- 172.21.15.55:0/4013437921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb534069090 con 0x7fb5340d01b0 2024-05-20T22:10:21.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.675+0000 7fb531ffb640 1 -- 172.21.15.55:0/4013437921 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7fb52800e280 con 0x7fb5340d01b0 2024-05-20T22:10:21.676 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:10:21.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 -- 172.21.15.55:0/4013437921 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb52006be60 msgr2=0x7fb52006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb52006be60 0x7fb52006e370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb524009f50 tx=0x7fb52400b040 comp rx=0 tx=0).stop 2024-05-20T22:10:21.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 -- 172.21.15.55:0/4013437921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 msgr2=0x7fb5341f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 0x7fb5341f5c00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb52800c430 tx=0x7fb52800c8f0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 -- 172.21.15.55:0/4013437921 shutdown_connections 2024-05-20T22:10:21.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb52006be60 0x7fb52006e370 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5340d32a0 0x7fb5340cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5340d01b0 0x7fb5341f5c00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 --2- 172.21.15.55:0/4013437921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5340cf690 0x7fb534158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 -- 172.21.15.55:0/4013437921 >> 172.21.15.55:0/4013437921 conn(0x7fb5340b6de0 msgr2=0x7fb53406c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:21.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 -- 172.21.15.55:0/4013437921 shutdown_connections 2024-05-20T22:10:21.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.677+0000 7fb53b709640 1 -- 172.21.15.55:0/4013437921 wait complete. 2024-05-20T22:10:21.690 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T22:10:21.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.804+0000 7f7701b09640 1 Processor -- start 2024-05-20T22:10:21.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.804+0000 7f7701b09640 1 -- start start 2024-05-20T22:10:21.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.805+0000 7f7701b09640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f76fc0d2d20 0x7f76fc0ba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.805+0000 7f7701b09640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76fc0d3f00 0x7f76fc06c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.805+0000 7f7701b09640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76fc06b5a0 con 0x7f76fc06cb80 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.805+0000 7f7701b09640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f76fc06c1b0 con 0x7f76fc0d3f00 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.805+0000 7f7701b09640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f76fc0baaa0 con 0x7f76fc0d2d20 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.805+0000 7f76f2ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76fc0d3f00 0x7f76fc06c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.805+0000 7f76fbfff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f76fc06cb80 0x7f76fc0cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35412/0 (socket says 172.21.15.55:35412) 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.805+0000 7f76fbfff640 1 -- 172.21.15.55:0/3163993645 learned_addr learned my addr 172.21.15.55:0/3163993645 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.805+0000 7f76fb7fe640 1 --2- 172.21.15.55:0/3163993645 >> v2:172.21.15.138:3300/0 conn(0x7f76fc0d2d20 0x7f76fc0ba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.806+0000 7f76f2ffd640 1 -- 172.21.15.55:0/3163993645 >> v2:172.21.15.138:3300/0 conn(0x7f76fc0d2d20 msgr2=0x7f76fc0ba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.806+0000 7f76f2ffd640 1 --2- 172.21.15.55:0/3163993645 >> v2:172.21.15.138:3300/0 conn(0x7f76fc0d2d20 0x7f76fc0ba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.806+0000 7f76f2ffd640 1 -- 172.21.15.55:0/3163993645 >> v1:172.21.15.55:6789/0 conn(0x7f76fc06cb80 legacy=0x7f76fc0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.806+0000 7f76faffd640 1 -- 172.21.15.55:0/3163993645 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103008719 0 0) 0x7f76fc06b5a0 con 0x7f76fc06cb80 2024-05-20T22:10:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.806+0000 7f76f2ffd640 1 -- 172.21.15.55:0/3163993645 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc06b880 con 0x7f76fc0d3f00 2024-05-20T22:10:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.806+0000 7f76f2ffd640 1 --2- 172.21.15.55:0/3163993645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc0d3f00 0x7f76fc06c640 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f76fc06c330 tx=0x7f76e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cadf43a6e824f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f76faffd640 1 -- 172.21.15.55:0/3163993645 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4037070 con 0x7f76fc0d3f00 2024-05-20T22:10:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f76faffd640 1 -- 172.21.15.55:0/3163993645 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e4032440 con 0x7f76fc0d3f00 2024-05-20T22:10:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f76faffd640 1 -- 172.21.15.55:0/3163993645 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4032810 con 0x7f76fc0d3f00 2024-05-20T22:10:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f7701b09640 1 -- 172.21.15.55:0/3163993645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc0d3f00 msgr2=0x7f76fc06c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f7701b09640 1 --2- 172.21.15.55:0/3163993645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc0d3f00 0x7f76fc06c640 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f76fc06c330 tx=0x7f76e402a1a0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f7701b09640 1 -- 172.21.15.55:0/3163993645 shutdown_connections 2024-05-20T22:10:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f7701b09640 1 --2- 172.21.15.55:0/3163993645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc0d3f00 0x7f76fc06c640 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f7701b09640 1 --2- 172.21.15.55:0/3163993645 >> v2:172.21.15.138:3300/0 conn(0x7f76fc0d2d20 0x7f76fc0ba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f7701b09640 1 -- 172.21.15.55:0/3163993645 >> 172.21.15.55:0/3163993645 conn(0x7f76fc0d1cb0 msgr2=0x7f76fc0d4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f7701b09640 1 -- 172.21.15.55:0/3163993645 shutdown_connections 2024-05-20T22:10:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.807+0000 7f7701b09640 1 -- 172.21.15.55:0/3163993645 wait complete. 2024-05-20T22:10:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.809+0000 7f7701b09640 1 Processor -- start 2024-05-20T22:10:21.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.809+0000 7f7701b09640 1 -- start start 2024-05-20T22:10:21.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f7701b09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc06cb80 0x7f76fc152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f7701b09640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76fc0d2d20 0x7f76fc0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f7701b09640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc0d3f00 0x7f76fc160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f7701b09640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc0d4310 con 0x7f76fc06cb80 2024-05-20T22:10:21.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f7701b09640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc06be40 con 0x7f76fc0d3f00 2024-05-20T22:10:21.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f7701b09640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f76fc0d4490 con 0x7f76fc0d2d20 2024-05-20T22:10:21.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc06cb80 0x7f76fc152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc06cb80 0x7f76fc152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55596/0 (socket says 172.21.15.55:55596) 2024-05-20T22:10:21.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fb7fe640 1 -- 172.21.15.55:0/2714473721 learned_addr learned my addr 172.21.15.55:0/2714473721 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:21.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fbfff640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc0d3f00 0x7f76fc160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76f2ffd640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76fc0d2d20 0x7f76fc0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fb7fe640 1 -- 172.21.15.55:0/2714473721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76fc0d2d20 msgr2=0x7f76fc0c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fb7fe640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76fc0d2d20 0x7f76fc0c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fb7fe640 1 -- 172.21.15.55:0/2714473721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc0d3f00 msgr2=0x7f76fc160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:21.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fb7fe640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc0d3f00 0x7f76fc160160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:21.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fb7fe640 1 -- 172.21.15.55:0/2714473721 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc154ad0 con 0x7f76fc06cb80 2024-05-20T22:10:21.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fbfff640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc0d3f00 0x7f76fc160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:21.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76fb7fe640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc06cb80 0x7f76fc152ce0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00d600 tx=0x7f76ec00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:21.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76f8ff9640 1 -- 172.21.15.55:0/2714473721 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004240 con 0x7f76fc06cb80 2024-05-20T22:10:21.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76f8ff9640 1 -- 172.21.15.55:0/2714473721 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76ec0043e0 con 0x7f76fc06cb80 2024-05-20T22:10:21.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f7701b09640 1 -- 172.21.15.55:0/2714473721 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76fc151c40 con 0x7f76fc06cb80 2024-05-20T22:10:21.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f76f8ff9640 1 -- 172.21.15.55:0/2714473721 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec003830 con 0x7f76fc06cb80 2024-05-20T22:10:21.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.811+0000 7f7701b09640 1 -- 172.21.15.55:0/2714473721 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76fc152200 con 0x7f76fc06cb80 2024-05-20T22:10:21.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.812+0000 7f7701b09640 1 -- 172.21.15.55:0/2714473721 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76fc069090 con 0x7f76fc06cb80 2024-05-20T22:10:21.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.813+0000 7f76f8ff9640 1 -- 172.21.15.55:0/2714473721 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76ec0039d0 con 0x7f76fc06cb80 2024-05-20T22:10:21.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.813+0000 7f76f8ff9640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76dc06be30 0x7f76dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:21.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.814+0000 7f76f8ff9640 1 -- 172.21.15.55:0/2714473721 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5249+0+0 (secure 0 0 0) 0x7f76ec08b050 con 0x7f76fc06cb80 2024-05-20T22:10:21.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.814+0000 7f76f2ffd640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76dc06be30 0x7f76dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:21.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.814+0000 7f76f2ffd640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76dc06be30 0x7f76dc06e340 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f76e4034000 tx=0x7f76e402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:21.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:21.818+0000 7f76f8ff9640 1 -- 172.21.15.55:0/2714473721 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f76ec01c360 con 0x7f76fc06cb80 2024-05-20T22:10:22.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.004+0000 7f7701b09640 1 -- 172.21.15.55:0/2714473721 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f76fc0b8120 con 0x7f76fc06cb80 2024-05-20T22:10:22.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.004+0000 7f76f8ff9640 1 -- 172.21.15.55:0/2714473721 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) ==== 84+0+65 (secure 0 0 0) 0x7f76ec059a90 con 0x7f76fc06cb80 2024-05-20T22:10:22.005 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:22.005 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-05-20T22:10:22.005 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-20T22:10:22.005 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-05-20T22:10:22.006 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-05-20T22:10:22.006 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:22.006 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-05-20T22:10:22.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.006+0000 7f7701b09640 1 -- 172.21.15.55:0/2714473721 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76dc06be30 msgr2=0x7f76dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.006+0000 7f7701b09640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76dc06be30 0x7f76dc06e340 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f76e4034000 tx=0x7f76e402e000 comp rx=0 tx=0).stop 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.006+0000 7f7701b09640 1 -- 172.21.15.55:0/2714473721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc06cb80 msgr2=0x7f76fc152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.006+0000 7f7701b09640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc06cb80 0x7f76fc152ce0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00d600 tx=0x7f76ec00dac0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.006+0000 7f7701b09640 1 -- 172.21.15.55:0/2714473721 shutdown_connections 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.006+0000 7f7701b09640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f76dc06be30 0x7f76dc06e340 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.006+0000 7f7701b09640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc0d3f00 0x7f76fc160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.006+0000 7f7701b09640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76fc0d2d20 0x7f76fc0c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.006+0000 7f7701b09640 1 --2- 172.21.15.55:0/2714473721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76fc06cb80 0x7f76fc152ce0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.006+0000 7f7701b09640 1 -- 172.21.15.55:0/2714473721 >> 172.21.15.55:0/2714473721 conn(0x7f76fc0d1cb0 msgr2=0x7f76fc15db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.007+0000 7f7701b09640 1 -- 172.21.15.55:0/2714473721 shutdown_connections 2024-05-20T22:10:22.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.007+0000 7f7701b09640 1 -- 172.21.15.55:0/2714473721 wait complete. 2024-05-20T22:10:22.019 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T22:10:22.020 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T22:10:22.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.138+0000 7f4b11f38640 1 Processor -- start 2024-05-20T22:10:22.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.138+0000 7f4b11f38640 1 -- start start 2024-05-20T22:10:22.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.139+0000 7f4b11f38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0b2470 0x7f4b0c0b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.139+0000 7f4b11f38640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b0c0cfad0 0x7f4b0c0d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.139+0000 7f4b11f38640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b0c0b37b0 con 0x7f4b0c06d0b0 2024-05-20T22:10:22.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.139+0000 7f4b11f38640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0c0b2880 con 0x7f4b0c0cfad0 2024-05-20T22:10:22.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.139+0000 7f4b11f38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0c06d8b0 con 0x7f4b0c0b2470 2024-05-20T22:10:22.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.140+0000 7f4b0bfff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b0c0cfad0 0x7f4b0c0d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.140+0000 7f4b0affd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4b0c06d0b0 0x7f4b0c0cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35426/0 (socket says 172.21.15.55:35426) 2024-05-20T22:10:22.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.140+0000 7f4b0affd640 1 -- 172.21.15.55:0/2470769701 learned_addr learned my addr 172.21.15.55:0/2470769701 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:22.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.140+0000 7f4b0b7fe640 1 --2- 172.21.15.55:0/2470769701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0b2470 0x7f4b0c0b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.140+0000 7f4b0bfff640 1 -- 172.21.15.55:0/2470769701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0b2470 msgr2=0x7f4b0c0b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:22.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.140+0000 7f4b0bfff640 1 --2- 172.21.15.55:0/2470769701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0b2470 0x7f4b0c0b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.140+0000 7f4b0bfff640 1 -- 172.21.15.55:0/2470769701 >> v1:172.21.15.55:6789/0 conn(0x7f4b0c06d0b0 legacy=0x7f4b0c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.140+0000 7f4b0a7fc640 1 -- 172.21.15.55:0/2470769701 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1441357032 0 0) 0x7f4b00001110 con 0x7f4b0c06d0b0 2024-05-20T22:10:22.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.140+0000 7f4b0bfff640 1 -- 172.21.15.55:0/2470769701 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c06db90 con 0x7f4b0c0cfad0 2024-05-20T22:10:22.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.141+0000 7f4b0bfff640 1 --2- 172.21.15.55:0/2470769701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0cfad0 0x7f4b0c0d0b60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009d90 tx=0x7f4b0402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4d81f62dea296b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:22.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.141+0000 7f4b0a7fc640 1 -- 172.21.15.55:0/2470769701 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04007740 con 0x7f4b0c0cfad0 2024-05-20T22:10:22.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.141+0000 7f4b0a7fc640 1 -- 172.21.15.55:0/2470769701 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b040078e0 con 0x7f4b0c0cfad0 2024-05-20T22:10:22.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.141+0000 7f4b0a7fc640 1 -- 172.21.15.55:0/2470769701 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04007cb0 con 0x7f4b0c0cfad0 2024-05-20T22:10:22.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.141+0000 7f4b11f38640 1 -- 172.21.15.55:0/2470769701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0cfad0 msgr2=0x7f4b0c0d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.141+0000 7f4b11f38640 1 --2- 172.21.15.55:0/2470769701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0cfad0 0x7f4b0c0d0b60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009d90 tx=0x7f4b0402a660 comp rx=0 tx=0).stop 2024-05-20T22:10:22.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.142+0000 7f4b11f38640 1 -- 172.21.15.55:0/2470769701 shutdown_connections 2024-05-20T22:10:22.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.142+0000 7f4b11f38640 1 --2- 172.21.15.55:0/2470769701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0cfad0 0x7f4b0c0d0b60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.142+0000 7f4b11f38640 1 --2- 172.21.15.55:0/2470769701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c0b2470 0x7f4b0c0b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.142+0000 7f4b11f38640 1 -- 172.21.15.55:0/2470769701 >> 172.21.15.55:0/2470769701 conn(0x7f4b0c09be10 msgr2=0x7f4b0c0d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:22.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.142+0000 7f4b11f38640 1 -- 172.21.15.55:0/2470769701 shutdown_connections 2024-05-20T22:10:22.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.142+0000 7f4b11f38640 1 -- 172.21.15.55:0/2470769701 wait complete. 2024-05-20T22:10:22.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.143+0000 7f4b11f38640 1 Processor -- start 2024-05-20T22:10:22.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.144+0000 7f4b11f38640 1 -- start start 2024-05-20T22:10:22.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b11f38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c06d0b0 0x7f4b0c15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b11f38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0b2470 0x7f4b0c09c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b11f38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c09ce30 0x7f4b0c09f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b11f38640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c06d4c0 con 0x7f4b0c0b2470 2024-05-20T22:10:22.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b11f38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c0b3a30 con 0x7f4b0c06d0b0 2024-05-20T22:10:22.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b11f38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0c0b3f50 con 0x7f4b0c09ce30 2024-05-20T22:10:22.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0b2470 0x7f4b0c09c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0b2470 0x7f4b0c09c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55624/0 (socket says 172.21.15.55:55624) 2024-05-20T22:10:22.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0affd640 1 -- 172.21.15.55:0/988269788 learned_addr learned my addr 172.21.15.55:0/988269788 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:22.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0bfff640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c09ce30 0x7f4b0c09f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0b7fe640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c06d0b0 0x7f4b0c15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0affd640 1 -- 172.21.15.55:0/988269788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c09ce30 msgr2=0x7f4b0c09f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0affd640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c09ce30 0x7f4b0c09f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0affd640 1 -- 172.21.15.55:0/988269788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c06d0b0 msgr2=0x7f4b0c15f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0affd640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c06d0b0 0x7f4b0c15f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0affd640 1 -- 172.21.15.55:0/988269788 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c09d640 con 0x7f4b0c0b2470 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.145+0000 7f4b0affd640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0b2470 0x7f4b0c09c8f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000c410 tx=0x7f4b0000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.146+0000 7f4af3fff640 1 -- 172.21.15.55:0/988269788 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00017070 con 0x7f4b0c0b2470 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.146+0000 7f4af3fff640 1 -- 172.21.15.55:0/988269788 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b00005ce0 con 0x7f4b0c0b2470 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.146+0000 7f4b11f38640 1 -- 172.21.15.55:0/988269788 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0c09d920 con 0x7f4b0c0b2470 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.146+0000 7f4af3fff640 1 -- 172.21.15.55:0/988269788 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000072f0 con 0x7f4b0c0b2470 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.146+0000 7f4b11f38640 1 -- 172.21.15.55:0/988269788 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0c09dee0 con 0x7f4b0c0b2470 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.147+0000 7f4b11f38640 1 -- 172.21.15.55:0/988269788 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0c069090 con 0x7f4b0c0b2470 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.147+0000 7f4af3fff640 1 -- 172.21.15.55:0/988269788 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b0001a070 con 0x7f4b0c0b2470 2024-05-20T22:10:22.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.148+0000 7f4af3fff640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4af806be60 0x7f4af806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.148+0000 7f4af3fff640 1 -- 172.21.15.55:0/988269788 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5249+0+0 (secure 0 0 0) 0x7f4b0008a100 con 0x7f4b0c0b2470 2024-05-20T22:10:22.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.148+0000 7f4b0b7fe640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4af806be60 0x7f4af806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.148+0000 7f4b0b7fe640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4af806be60 0x7f4af806e370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c073ae0 tx=0x7f4afc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:22.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.153+0000 7f4af3fff640 1 -- 172.21.15.55:0/988269788 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4b0c0b37b0 con 0x7f4b0c0b2470 2024-05-20T22:10:22.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.337+0000 7f4b11f38640 1 -- 172.21.15.55:0/988269788 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4b0c0d35a0 con 0x7f4b0c0b2470 2024-05-20T22:10:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.594+0000 7f4af3fff640 1 -- 172.21.15.55:0/988269788 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) ==== 138+0+0 (secure 0 0 0) 0x7f4b00058be0 con 0x7f4b0c0b2470 2024-05-20T22:10:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-05-20T22:10:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 -- 172.21.15.55:0/988269788 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4af806be60 msgr2=0x7f4af806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4af806be60 0x7f4af806e370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c073ae0 tx=0x7f4afc008040 comp rx=0 tx=0).stop 2024-05-20T22:10:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 -- 172.21.15.55:0/988269788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0b2470 msgr2=0x7f4b0c09c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0b2470 0x7f4b0c09c8f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000c410 tx=0x7f4b0000c8d0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 -- 172.21.15.55:0/988269788 shutdown_connections 2024-05-20T22:10:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4af806be60 0x7f4af806e370 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b0c09ce30 0x7f4b0c09f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b0c0b2470 0x7f4b0c09c8f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 --2- 172.21.15.55:0/988269788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b0c06d0b0 0x7f4b0c15f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 -- 172.21.15.55:0/988269788 >> 172.21.15.55:0/988269788 conn(0x7f4b0c09be10 msgr2=0x7f4b0c071310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 -- 172.21.15.55:0/988269788 shutdown_connections 2024-05-20T22:10:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.597+0000 7f4b11f38640 1 -- 172.21.15.55:0/988269788 wait complete. 2024-05-20T22:10:22.615 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T22:10:22.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.738+0000 7fb411a92640 1 Processor -- start 2024-05-20T22:10:22.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.738+0000 7fb411a92640 1 -- start start 2024-05-20T22:10:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.739+0000 7fb411a92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c167ff0 0x7fb40c16a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.739+0000 7fb411a92640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40c0d04b0 con 0x7fb40c15f150 2024-05-20T22:10:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.739+0000 7fb411a92640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40c0d3780 con 0x7fb40c0d32a0 2024-05-20T22:10:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.739+0000 7fb411a92640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c06c510 con 0x7fb40c167ff0 2024-05-20T22:10:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.739+0000 7fb40affd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb40c15f150 0x7fb40c0cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35440/0 (socket says 172.21.15.55:35440) 2024-05-20T22:10:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.739+0000 7fb40affd640 1 -- 172.21.15.55:0/89642259 learned_addr learned my addr 172.21.15.55:0/89642259 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.739+0000 7fb40bfff640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c167ff0 0x7fb40c16a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.739+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1398685044 0 0) 0x7fb40c0d04b0 con 0x7fb40c15f150 2024-05-20T22:10:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.739+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3f4003520 con 0x7fb40c15f150 2024-05-20T22:10:22.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 568113642 0 0) 0x7fb40c0d3780 con 0x7fb40c0d32a0 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb40c0d04b0 con 0x7fb40c0d32a0 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3820937907 0 0) 0x7fb3f4003520 con 0x7fb40c15f150 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c167ff0 msgr2=0x7fb40c16a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c167ff0 0x7fb40c16a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 >> v1:172.21.15.138:6789/0 conn(0x7fb40c0d32a0 legacy=0x7fb40c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c0d3a60 con 0x7fb40c15f150 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fb400001670 con 0x7fb40c15f150 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 >> v1:172.21.15.55:6789/0 conn(0x7fb40c15f150 legacy=0x7fb40c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c167ff0 0x7fb3f4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3f4004120 0x7fb3f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3f4004d40 0x7fb3f4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c06c510 con 0x7fb3f4004120 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f400bb80 con 0x7fb3f4004d40 2024-05-20T22:10:22.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f400bd80 con 0x7fb40c167ff0 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40affd640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3f4004120 0x7fb3f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40b7fe640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c167ff0 0x7fb3f4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40bfff640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3f4004d40 0x7fb3f4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40affd640 1 -- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c167ff0 msgr2=0x7fb3f4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40affd640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c167ff0 0x7fb3f4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40affd640 1 -- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3f4004d40 msgr2=0x7fb3f4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40affd640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3f4004d40 0x7fb3f4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40affd640 1 -- 172.21.15.55:0/89642259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f4003520 con 0x7fb3f4004120 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40bfff640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3f4004d40 0x7fb3f4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.740+0000 7fb40b7fe640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c167ff0 0x7fb3f4003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.741+0000 7fb40affd640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3f4004120 0x7fb3f4004800 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb400007fc0 tx=0x7fb3f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d64fb36ebf41dc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.741+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400007be0 con 0x7fb3f4004120 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.741+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb400005af0 con 0x7fb3f4004120 2024-05-20T22:10:22.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.741+0000 7fb40a7fc640 1 -- 172.21.15.55:0/89642259 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400007070 con 0x7fb3f4004120 2024-05-20T22:10:22.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.742+0000 7fb411a92640 1 -- 172.21.15.55:0/89642259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3f4004120 msgr2=0x7fb3f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.742+0000 7fb411a92640 1 --2- 172.21.15.55:0/89642259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3f4004120 0x7fb3f4004800 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb400007fc0 tx=0x7fb3f4005830 comp rx=0 tx=0).stop 2024-05-20T22:10:22.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.742+0000 7fb40bfff640 1 -- 172.21.15.55:0/89642259 reap_dead start 2024-05-20T22:10:22.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.742+0000 7fb411a92640 1 -- 172.21.15.55:0/89642259 shutdown_connections 2024-05-20T22:10:22.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.742+0000 7fb411a92640 1 -- 172.21.15.55:0/89642259 >> 172.21.15.55:0/89642259 conn(0x7fb40c0b6de0 msgr2=0x7fb40c0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:22.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.742+0000 7fb411a92640 1 -- 172.21.15.55:0/89642259 shutdown_connections 2024-05-20T22:10:22.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.742+0000 7fb411a92640 1 -- 172.21.15.55:0/89642259 wait complete. 2024-05-20T22:10:22.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.744+0000 7fb411a92640 1 Processor -- start 2024-05-20T22:10:22.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.744+0000 7fb411a92640 1 -- start start 2024-05-20T22:10:22.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.745+0000 7fb411a92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f4004120 0x7fb40c1f6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.745+0000 7fb411a92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0ce420 0x7fb40c0b9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.745+0000 7fb411a92640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb40c152670 0x7fb40c0b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.745+0000 7fb411a92640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c0d4300 con 0x7fb40c152670 2024-05-20T22:10:22.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.745+0000 7fb411a92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c0d4020 con 0x7fb40c0ce420 2024-05-20T22:10:22.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.745+0000 7fb411a92640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c15f560 con 0x7fb3f4004120 2024-05-20T22:10:22.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.745+0000 7fb40bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb40c152670 0x7fb40c0b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.745+0000 7fb40bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb40c152670 0x7fb40c0b9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55636/0 (socket says 172.21.15.55:55636) 2024-05-20T22:10:22.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.745+0000 7fb40bfff640 1 -- 172.21.15.55:0/3136085666 learned_addr learned my addr 172.21.15.55:0/3136085666 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.745+0000 7fb40b7fe640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f4004120 0x7fb40c1f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb40affd640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0ce420 0x7fb40c0b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb40bfff640 1 -- 172.21.15.55:0/3136085666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f4004120 msgr2=0x7fb40c1f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb40bfff640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f4004120 0x7fb40c1f6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb40bfff640 1 -- 172.21.15.55:0/3136085666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0ce420 msgr2=0x7fb40c0b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb40bfff640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0ce420 0x7fb40c0b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb40bfff640 1 -- 172.21.15.55:0/3136085666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c0b85a0 con 0x7fb40c152670 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb40bfff640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb40c152670 0x7fb40c0b9b80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0cf2a0 tx=0x7fb404010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb3f3fff640 1 -- 172.21.15.55:0/3136085666 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40401a030 con 0x7fb40c152670 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb3f3fff640 1 -- 172.21.15.55:0/3136085666 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb404010d80 con 0x7fb40c152670 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb411a92640 1 -- 172.21.15.55:0/3136085666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c0b8880 con 0x7fb40c152670 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb3f3fff640 1 -- 172.21.15.55:0/3136085666 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404003bd0 con 0x7fb40c152670 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.746+0000 7fb411a92640 1 -- 172.21.15.55:0/3136085666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c0b8e40 con 0x7fb40c152670 2024-05-20T22:10:22.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.747+0000 7fb411a92640 1 -- 172.21.15.55:0/3136085666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c069090 con 0x7fb40c152670 2024-05-20T22:10:22.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.748+0000 7fb3f3fff640 1 -- 172.21.15.55:0/3136085666 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb404005d10 con 0x7fb40c152670 2024-05-20T22:10:22.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.748+0000 7fb3f3fff640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3f806be30 0x7fb3f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:22.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.748+0000 7fb3f3fff640 1 -- 172.21.15.55:0/3136085666 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5619+0+0 (secure 0 0 0) 0x7fb40408b3b0 con 0x7fb40c152670 2024-05-20T22:10:22.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.748+0000 7fb40b7fe640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3f806be30 0x7fb3f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:22.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.749+0000 7fb40b7fe640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3f806be30 0x7fb3f806e340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc007fd0 tx=0x7fb3fc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:22.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.753+0000 7fb3f3fff640 1 -- 172.21.15.55:0/3136085666 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb404025020 con 0x7fb40c152670 2024-05-20T22:10:22.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:22.939+0000 7fb411a92640 1 -- 172.21.15.55:0/3136085666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb40c0d2e50 con 0x7fb40c152670 2024-05-20T22:10:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.598+0000 7fb3f3fff640 1 -- 172.21.15.55:0/3136085666 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) ==== 170+0+0 (secure 0 0 0) 0x7fb404059e40 con 0x7fb40c152670 2024-05-20T22:10:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-05-20T22:10:23.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 -- 172.21.15.55:0/3136085666 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3f806be30 msgr2=0x7fb3f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:23.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3f806be30 0x7fb3f806e340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc007fd0 tx=0x7fb3fc009040 comp rx=0 tx=0).stop 2024-05-20T22:10:23.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 -- 172.21.15.55:0/3136085666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb40c152670 msgr2=0x7fb40c0b9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:23.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb40c152670 0x7fb40c0b9b80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0cf2a0 tx=0x7fb404010590 comp rx=0 tx=0).stop 2024-05-20T22:10:23.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 -- 172.21.15.55:0/3136085666 shutdown_connections 2024-05-20T22:10:23.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb3f806be30 0x7fb3f806e340 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:23.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb40c152670 0x7fb40c0b9b80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:23.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0ce420 0x7fb40c0b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:23.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 --2- 172.21.15.55:0/3136085666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3f4004120 0x7fb40c1f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:23.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 -- 172.21.15.55:0/3136085666 >> 172.21.15.55:0/3136085666 conn(0x7fb40c0b6de0 msgr2=0x7fb40c0d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:23.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.601+0000 7fb411a92640 1 -- 172.21.15.55:0/3136085666 shutdown_connections 2024-05-20T22:10:23.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.602+0000 7fb411a92640 1 -- 172.21.15.55:0/3136085666 wait complete. 2024-05-20T22:10:23.632 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T22:10:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.755+0000 7f52d9779640 1 Processor -- start 2024-05-20T22:10:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.755+0000 7f52d9779640 1 -- start start 2024-05-20T22:10:23.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.756+0000 7f52d9779640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d415f080 0x7f52d406c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:23.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.756+0000 7f52d9779640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f52d406c9a0 0x7f52d40d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:23.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.756+0000 7f52d9779640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52d406b6a0 con 0x7f52d40cf680 2024-05-20T22:10:23.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.756+0000 7f52d9779640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f52d406b980 con 0x7f52d406c9a0 2024-05-20T22:10:23.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.756+0000 7f52d9779640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f52d406bc60 con 0x7f52d415f080 2024-05-20T22:10:23.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.756+0000 7f52d27fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d415f080 0x7f52d406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:23.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.756+0000 7f52d37fe640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f52d406c9a0 0x7f52d40d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:23.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.756+0000 7f52d27fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d415f080 0x7f52d406c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:50844/0 (socket says 172.21.15.55:50844) 2024-05-20T22:10:23.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.756+0000 7f52d27fc640 1 -- 172.21.15.55:0/3935903470 learned_addr learned my addr 172.21.15.55:0/3935903470 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:23.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.756+0000 7f52d2ffd640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f52d40cf680 0x7f52d40b7fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:39306/0 (socket says 172.21.15.55:39306) 2024-05-20T22:10:23.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.757+0000 7f52d27fc640 1 -- 172.21.15.55:0/3935903470 >> v2:172.21.15.138:3300/0 conn(0x7f52d406c9a0 msgr2=0x7f52d40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:23.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.757+0000 7f52d27fc640 1 --2- 172.21.15.55:0/3935903470 >> v2:172.21.15.138:3300/0 conn(0x7f52d406c9a0 0x7f52d40d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:23.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.757+0000 7f52d27fc640 1 -- 172.21.15.55:0/3935903470 >> v1:172.21.15.138:6789/0 conn(0x7f52d40cf680 legacy=0x7f52d40b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:23.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.757+0000 7f52d27fc640 1 -- 172.21.15.55:0/3935903470 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d406bf40 con 0x7f52d415f080 2024-05-20T22:10:23.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.757+0000 7f52d1ffb640 1 -- 172.21.15.55:0/3935903470 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345566327 0 0) 0x7f52d406b6a0 con 0x7f52d40cf680 2024-05-20T22:10:23.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.757+0000 7f52d37fe640 1 --2- 172.21.15.55:0/3935903470 >> v2:172.21.15.138:3300/0 conn(0x7f52d406c9a0 0x7f52d40d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:10:23.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.758+0000 7f52d27fc640 1 --2- 172.21.15.55:0/3935903470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d415f080 0x7f52d406c460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009870 tx=0x7f52c802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e997b720a89452a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:23.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.758+0000 7f52d1ffb640 1 -- 172.21.15.55:0/3935903470 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8037070 con 0x7f52d415f080 2024-05-20T22:10:23.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.758+0000 7f52d1ffb640 1 -- 172.21.15.55:0/3935903470 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c8032440 con 0x7f52d415f080 2024-05-20T22:10:23.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.758+0000 7f52d1ffb640 1 -- 172.21.15.55:0/3935903470 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8032810 con 0x7f52d415f080 2024-05-20T22:10:23.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.759+0000 7f52d9779640 1 -- 172.21.15.55:0/3935903470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d415f080 msgr2=0x7f52d406c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:23.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.759+0000 7f52d9779640 1 --2- 172.21.15.55:0/3935903470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d415f080 0x7f52d406c460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009870 tx=0x7f52c802a1c0 comp rx=0 tx=0).stop 2024-05-20T22:10:23.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.759+0000 7f52d9779640 1 -- 172.21.15.55:0/3935903470 shutdown_connections 2024-05-20T22:10:23.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.759+0000 7f52d9779640 1 --2- 172.21.15.55:0/3935903470 >> v2:172.21.15.138:3300/0 conn(0x7f52d406c9a0 0x7f52d40d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:23.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.759+0000 7f52d9779640 1 --2- 172.21.15.55:0/3935903470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d415f080 0x7f52d406c460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:23.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.759+0000 7f52d9779640 1 -- 172.21.15.55:0/3935903470 >> 172.21.15.55:0/3935903470 conn(0x7f52d40d5450 msgr2=0x7f52d40b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:23.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.759+0000 7f52d9779640 1 -- 172.21.15.55:0/3935903470 shutdown_connections 2024-05-20T22:10:23.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.759+0000 7f52d9779640 1 -- 172.21.15.55:0/3935903470 wait complete. 2024-05-20T22:10:23.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.761+0000 7f52d9779640 1 Processor -- start 2024-05-20T22:10:23.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.761+0000 7f52d9779640 1 -- start start 2024-05-20T22:10:23.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d9779640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d406c9a0 0x7f52d41f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:23.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d9779640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52d40cf680 0x7f52d41f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:23.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d9779640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d415f080 0x7f52d40b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d9779640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f52d406cf30 con 0x7f52d40cf680 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d9779640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52d406cdb0 con 0x7f52d415f080 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d9779640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f52d40b4120 con 0x7f52d406c9a0 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52d40cf680 0x7f52d41f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52d40cf680 0x7f52d41f87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55650/0 (socket says 172.21.15.55:55650) 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d27fc640 1 -- 172.21.15.55:0/2767380672 learned_addr learned my addr 172.21.15.55:0/2767380672 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d2ffd640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d406c9a0 0x7f52d41f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d37fe640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d415f080 0x7f52d40b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d27fc640 1 -- 172.21.15.55:0/2767380672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d406c9a0 msgr2=0x7f52d41f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d27fc640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d406c9a0 0x7f52d41f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d27fc640 1 -- 172.21.15.55:0/2767380672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d415f080 msgr2=0x7f52d40b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d27fc640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d415f080 0x7f52d40b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.762+0000 7f52d27fc640 1 -- 172.21.15.55:0/2767380672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d41f94e0 con 0x7f52d40cf680 2024-05-20T22:10:23.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.763+0000 7f52d27fc640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52d40cf680 0x7f52d41f87b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f52c8000c00 tx=0x7f52c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:23.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.763+0000 7f52bb7fe640 1 -- 172.21.15.55:0/2767380672 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8037040 con 0x7f52d40cf680 2024-05-20T22:10:23.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.763+0000 7f52bb7fe640 1 -- 172.21.15.55:0/2767380672 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c8042070 con 0x7f52d40cf680 2024-05-20T22:10:23.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.763+0000 7f52d9779640 1 -- 172.21.15.55:0/2767380672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d414fea0 con 0x7f52d40cf680 2024-05-20T22:10:23.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.763+0000 7f52bb7fe640 1 -- 172.21.15.55:0/2767380672 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8030880 con 0x7f52d40cf680 2024-05-20T22:10:23.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.763+0000 7f52d9779640 1 -- 172.21.15.55:0/2767380672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d4151a00 con 0x7f52d40cf680 2024-05-20T22:10:23.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.764+0000 7f52d9779640 1 -- 172.21.15.55:0/2767380672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d41f9250 con 0x7f52d40cf680 2024-05-20T22:10:23.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.765+0000 7f52bb7fe640 1 -- 172.21.15.55:0/2767380672 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52c802ad10 con 0x7f52d40cf680 2024-05-20T22:10:23.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.765+0000 7f52bb7fe640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f52c006be30 0x7f52c006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:23.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.765+0000 7f52d2ffd640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f52c006be30 0x7f52c006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:23.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.765+0000 7f52bb7fe640 1 -- 172.21.15.55:0/2767380672 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6001+0+0 (secure 0 0 0) 0x7f52c80ab5e0 con 0x7f52d40cf680 2024-05-20T22:10:23.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.766+0000 7f52d2ffd640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f52c006be30 0x7f52c006e340 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f52c4007ed0 tx=0x7f52c4007e60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:23.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.770+0000 7f52bb7fe640 1 -- 172.21.15.55:0/2767380672 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f52c8079e60 con 0x7f52d40cf680 2024-05-20T22:10:23.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:23.993+0000 7f52d9779640 1 -- 172.21.15.55:0/2767380672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f52d4069090 con 0x7f52d40cf680 2024-05-20T22:10:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.630+0000 7f52bb7fe640 1 -- 172.21.15.55:0/2767380672 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v31) ==== 172+0+0 (secure 0 0 0) 0x7f52c8079760 con 0x7f52d40cf680 2024-05-20T22:10:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 7 and data pool 8 2024-05-20T22:10:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.642+0000 7f52d9779640 1 -- 172.21.15.55:0/2767380672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f52c006be30 msgr2=0x7f52c006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.642+0000 7f52d9779640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f52c006be30 0x7f52c006e340 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f52c4007ed0 tx=0x7f52c4007e60 comp rx=0 tx=0).stop 2024-05-20T22:10:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.642+0000 7f52d9779640 1 -- 172.21.15.55:0/2767380672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52d40cf680 msgr2=0x7f52d41f87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.642+0000 7f52d9779640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52d40cf680 0x7f52d41f87b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f52c8000c00 tx=0x7f52c80046c0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.644+0000 7f52d9779640 1 -- 172.21.15.55:0/2767380672 shutdown_connections 2024-05-20T22:10:24.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.644+0000 7f52d9779640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f52c006be30 0x7f52c006e340 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.644+0000 7f52d9779640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d415f080 0x7f52d40b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.644+0000 7f52d9779640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52d40cf680 0x7f52d41f87b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.644+0000 7f52d9779640 1 --2- 172.21.15.55:0/2767380672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52d406c9a0 0x7f52d41f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.644+0000 7f52d9779640 1 -- 172.21.15.55:0/2767380672 >> 172.21.15.55:0/2767380672 conn(0x7f52d40d5450 msgr2=0x7f52d40ba2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.646+0000 7f52d9779640 1 -- 172.21.15.55:0/2767380672 shutdown_connections 2024-05-20T22:10:24.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.647+0000 7f52d9779640 1 -- 172.21.15.55:0/2767380672 wait complete. 2024-05-20T22:10:24.668 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:10:24.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.786+0000 7f05a5b36640 1 Processor -- start 2024-05-20T22:10:24.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.786+0000 7f05a5b36640 1 -- start start 2024-05-20T22:10:24.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f05a5b36640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f05a015f100 0x7f05a00b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:24.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f05a5b36640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f05a006c790 0x7f05a00cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:24.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f05a5b36640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a00d3dd0 con 0x7f05a00b3b30 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f05a5b36640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f05a00d40b0 con 0x7f05a015f100 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f05a5b36640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f05a00d4390 con 0x7f05a006c790 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f059effd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f05a015f100 0x7f05a00b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f059f7fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f05a00b3b30 0x7f05a006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35448/0 (socket says 172.21.15.55:35448) 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f059f7fe640 1 -- 172.21.15.55:0/3844844556 learned_addr learned my addr 172.21.15.55:0/3844844556 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f059ffff640 1 --2- 172.21.15.55:0/3844844556 >> v2:172.21.15.138:3300/0 conn(0x7f05a006c790 0x7f05a00cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f059effd640 1 -- 172.21.15.55:0/3844844556 >> v2:172.21.15.138:3300/0 conn(0x7f05a006c790 msgr2=0x7f05a00cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f059effd640 1 --2- 172.21.15.55:0/3844844556 >> v2:172.21.15.138:3300/0 conn(0x7f05a006c790 0x7f05a00cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f059effd640 1 -- 172.21.15.55:0/3844844556 >> v1:172.21.15.55:6789/0 conn(0x7f05a00b3b30 legacy=0x7f05a006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f059effd640 1 -- 172.21.15.55:0/3844844556 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a006b5a0 con 0x7f05a015f100 2024-05-20T22:10:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f059e7fc640 1 -- 172.21.15.55:0/3844844556 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1155138480 0 0) 0x7f05a00d3dd0 con 0x7f05a00b3b30 2024-05-20T22:10:24.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.788+0000 7f059ffff640 1 --2- 172.21.15.55:0/3844844556 >> v2:172.21.15.138:3300/0 conn(0x7f05a006c790 0x7f05a00cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.789+0000 7f059effd640 1 --2- 172.21.15.55:0/3844844556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a015f100 0x7f05a00b7fe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f05a00d12f0 tx=0x7f059402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecda785442f4288 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.789+0000 7f059e7fc640 1 -- 172.21.15.55:0/3844844556 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594037070 con 0x7f05a015f100 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.789+0000 7f059e7fc640 1 -- 172.21.15.55:0/3844844556 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0594032440 con 0x7f05a015f100 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.789+0000 7f059e7fc640 1 -- 172.21.15.55:0/3844844556 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594032810 con 0x7f05a015f100 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.790+0000 7f05a5b36640 1 -- 172.21.15.55:0/3844844556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a015f100 msgr2=0x7f05a00b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.790+0000 7f05a5b36640 1 --2- 172.21.15.55:0/3844844556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a015f100 0x7f05a00b7fe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f05a00d12f0 tx=0x7f059402a1a0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.790+0000 7f05a5b36640 1 -- 172.21.15.55:0/3844844556 shutdown_connections 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.790+0000 7f05a5b36640 1 --2- 172.21.15.55:0/3844844556 >> v2:172.21.15.138:3300/0 conn(0x7f05a006c790 0x7f05a00cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.790+0000 7f05a5b36640 1 --2- 172.21.15.55:0/3844844556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a015f100 0x7f05a00b7fe0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.790+0000 7f05a5b36640 1 -- 172.21.15.55:0/3844844556 >> 172.21.15.55:0/3844844556 conn(0x7f05a00d51d0 msgr2=0x7f05a00b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.790+0000 7f05a5b36640 1 -- 172.21.15.55:0/3844844556 shutdown_connections 2024-05-20T22:10:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.790+0000 7f05a5b36640 1 -- 172.21.15.55:0/3844844556 wait complete. 2024-05-20T22:10:24.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.792+0000 7f05a5b36640 1 Processor -- start 2024-05-20T22:10:24.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.792+0000 7f05a5b36640 1 -- start start 2024-05-20T22:10:24.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.793+0000 7f05a5b36640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a006c790 0x7f05a01f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:24.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.793+0000 7f05a5b36640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00b3b30 0x7f05a015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.793+0000 7f05a5b36640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05a015f100 0x7f05a01531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.793+0000 7f05a5b36640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f05a006cba0 con 0x7f05a006c790 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.793+0000 7f05a5b36640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05a006bb60 con 0x7f05a015f100 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.793+0000 7f05a5b36640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05a006cd20 con 0x7f05a00b3b30 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a006c790 0x7f05a01f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a006c790 0x7f05a01f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55660/0 (socket says 172.21.15.55:55660) 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059f7fe640 1 -- 172.21.15.55:0/1081855423 learned_addr learned my addr 172.21.15.55:0/1081855423 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059ffff640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05a015f100 0x7f05a01531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059effd640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00b3b30 0x7f05a015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059f7fe640 1 -- 172.21.15.55:0/1081855423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00b3b30 msgr2=0x7f05a015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059f7fe640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00b3b30 0x7f05a015a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059f7fe640 1 -- 172.21.15.55:0/1081855423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05a015f100 msgr2=0x7f05a01531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059f7fe640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05a015f100 0x7f05a01531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059f7fe640 1 -- 172.21.15.55:0/1081855423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a01536e0 con 0x7f05a006c790 2024-05-20T22:10:24.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f059f7fe640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a006c790 0x7f05a01f8e50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f059000c440 tx=0x7f059000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:24.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f0587fff640 1 -- 172.21.15.55:0/1081855423 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900074e0 con 0x7f05a006c790 2024-05-20T22:10:24.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f0587fff640 1 -- 172.21.15.55:0/1081855423 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05900047e0 con 0x7f05a006c790 2024-05-20T22:10:24.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f05a5b36640 1 -- 172.21.15.55:0/1081855423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a01f96a0 con 0x7f05a006c790 2024-05-20T22:10:24.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f0587fff640 1 -- 172.21.15.55:0/1081855423 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590004bd0 con 0x7f05a006c790 2024-05-20T22:10:24.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.794+0000 7f05a5b36640 1 -- 172.21.15.55:0/1081855423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a01f9c60 con 0x7f05a006c790 2024-05-20T22:10:24.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.795+0000 7f05a5b36640 1 -- 172.21.15.55:0/1081855423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a00ba8a0 con 0x7f05a006c790 2024-05-20T22:10:24.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.796+0000 7f0587fff640 1 -- 172.21.15.55:0/1081855423 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0590011070 con 0x7f05a006c790 2024-05-20T22:10:24.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.796+0000 7f0587fff640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058c06be30 0x7f058c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:24.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.797+0000 7f059effd640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058c06be30 0x7f058c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:24.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.797+0000 7f0587fff640 1 -- 172.21.15.55:0/1081855423 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f059008aca0 con 0x7f05a006c790 2024-05-20T22:10:24.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.797+0000 7f059effd640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058c06be30 0x7f058c06e340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0594034000 tx=0x7f059402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:24.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.801+0000 7f0587fff640 1 -- 172.21.15.55:0/1081855423 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f05a00ba8a0 con 0x7f05a006c790 2024-05-20T22:10:24.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.985+0000 7f05a5b36640 1 -- 172.21.15.55:0/1081855423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05a0069090 con 0x7f05a006c790 2024-05-20T22:10:24.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.986+0000 7f0587fff640 1 -- 172.21.15.55:0/1081855423 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16955 (secure 0 0 0) 0x7f059000e280 con 0x7f05a006c790 2024-05-20T22:10:24.987 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:24.987 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":54,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:10:23.994806+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:10:22.369451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T22:10:22.970959+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:10:24.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 -- 172.21.15.55:0/1081855423 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058c06be30 msgr2=0x7f058c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:24.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058c06be30 0x7f058c06e340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0594034000 tx=0x7f059402e000 comp rx=0 tx=0).stop 2024-05-20T22:10:24.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 -- 172.21.15.55:0/1081855423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a006c790 msgr2=0x7f05a01f8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:24.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a006c790 0x7f05a01f8e50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f059000c440 tx=0x7f059000c900 comp rx=0 tx=0).stop 2024-05-20T22:10:24.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 -- 172.21.15.55:0/1081855423 shutdown_connections 2024-05-20T22:10:24.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f058c06be30 0x7f058c06e340 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05a015f100 0x7f05a01531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05a00b3b30 0x7f05a015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 --2- 172.21.15.55:0/1081855423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05a006c790 0x7f05a01f8e50 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:24.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 -- 172.21.15.55:0/1081855423 >> 172.21.15.55:0/1081855423 conn(0x7f05a00d51d0 msgr2=0x7f05a015e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:24.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 -- 172.21.15.55:0/1081855423 shutdown_connections 2024-05-20T22:10:24.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:24.988+0000 7f05a5b36640 1 -- 172.21.15.55:0/1081855423 wait complete. 2024-05-20T22:10:25.001 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:10:25.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.114+0000 7f11789ea640 1 Processor -- start 2024-05-20T22:10:25.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.114+0000 7f11789ea640 1 -- start start 2024-05-20T22:10:25.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f11789ea640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f11740d2d20 0x7f11740ba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f11789ea640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f11740cf870 0x7f11740cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f11789ea640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11740d01c0 con 0x7f11740d3f00 2024-05-20T22:10:25.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f11789ea640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f117406c1b0 con 0x7f11740d2d20 2024-05-20T22:10:25.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f11789ea640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f11740baaa0 con 0x7f11740cf870 2024-05-20T22:10:25.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f117259c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f11740d2d20 0x7f11740ba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f1171d9b640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f11740d3f00 0x7f117406c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35460/0 (socket says 172.21.15.55:35460) 2024-05-20T22:10:25.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f1171d9b640 1 -- 172.21.15.55:0/2117952656 learned_addr learned my addr 172.21.15.55:0/2117952656 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:25.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f1172d9d640 1 --2- 172.21.15.55:0/2117952656 >> v2:172.21.15.138:3300/0 conn(0x7f11740cf870 0x7f11740cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f117259c640 1 -- 172.21.15.55:0/2117952656 >> v2:172.21.15.138:3300/0 conn(0x7f11740cf870 msgr2=0x7f11740cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:25.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f117259c640 1 --2- 172.21.15.55:0/2117952656 >> v2:172.21.15.138:3300/0 conn(0x7f11740cf870 0x7f11740cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f117259c640 1 -- 172.21.15.55:0/2117952656 >> v1:172.21.15.55:6789/0 conn(0x7f11740d3f00 legacy=0x7f117406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:25.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f117259c640 1 -- 172.21.15.55:0/2117952656 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117406b5a0 con 0x7f11740d2d20 2024-05-20T22:10:25.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.116+0000 7f117159a640 1 -- 172.21.15.55:0/2117952656 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 625195082 0 0) 0x7f1168001110 con 0x7f11740d3f00 2024-05-20T22:10:25.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.117+0000 7f117259c640 1 --2- 172.21.15.55:0/2117952656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 0x7f11740ba560 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f11740d0dd0 tx=0x7f116402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fab802dca2cf636 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:25.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.117+0000 7f117159a640 1 -- 172.21.15.55:0/2117952656 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164037070 con 0x7f11740d2d20 2024-05-20T22:10:25.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.117+0000 7f117159a640 1 -- 172.21.15.55:0/2117952656 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1164032440 con 0x7f11740d2d20 2024-05-20T22:10:25.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.117+0000 7f117159a640 1 -- 172.21.15.55:0/2117952656 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116402ae60 con 0x7f11740d2d20 2024-05-20T22:10:25.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.118+0000 7f11789ea640 1 -- 172.21.15.55:0/2117952656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 msgr2=0x7f11740ba560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:25.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.118+0000 7f11789ea640 1 --2- 172.21.15.55:0/2117952656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 0x7f11740ba560 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f11740d0dd0 tx=0x7f116402a110 comp rx=0 tx=0).stop 2024-05-20T22:10:25.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.118+0000 7f11789ea640 1 -- 172.21.15.55:0/2117952656 shutdown_connections 2024-05-20T22:10:25.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.118+0000 7f11789ea640 1 --2- 172.21.15.55:0/2117952656 >> v2:172.21.15.138:3300/0 conn(0x7f11740cf870 0x7f11740cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.118+0000 7f11789ea640 1 --2- 172.21.15.55:0/2117952656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 0x7f11740ba560 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.118+0000 7f11789ea640 1 -- 172.21.15.55:0/2117952656 >> 172.21.15.55:0/2117952656 conn(0x7f11740d1cb0 msgr2=0x7f11740d4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:25.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.118+0000 7f11789ea640 1 -- 172.21.15.55:0/2117952656 shutdown_connections 2024-05-20T22:10:25.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.118+0000 7f11789ea640 1 -- 172.21.15.55:0/2117952656 wait complete. 2024-05-20T22:10:25.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.120+0000 7f11789ea640 1 Processor -- start 2024-05-20T22:10:25.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.120+0000 7f11789ea640 1 -- start start 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.121+0000 7f11789ea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11740cf870 0x7f11740ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.121+0000 7f11789ea640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 0x7f11740cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.121+0000 7f11789ea640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11740d3f00 0x7f117414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.121+0000 7f11789ea640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f117406be40 con 0x7f11740d2d20 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.121+0000 7f11789ea640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f117406bb60 con 0x7f11740cf870 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.121+0000 7f11789ea640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f117406cd50 con 0x7f11740d3f00 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.121+0000 7f1171d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 0x7f11740cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.121+0000 7f1171d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 0x7f11740cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55678/0 (socket says 172.21.15.55:55678) 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.121+0000 7f1171d9b640 1 -- 172.21.15.55:0/1951170691 learned_addr learned my addr 172.21.15.55:0/1951170691 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f1172d9d640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11740d3f00 0x7f117414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f117259c640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11740cf870 0x7f11740ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f1171d9b640 1 -- 172.21.15.55:0/1951170691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11740d3f00 msgr2=0x7f117414d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f1171d9b640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11740d3f00 0x7f117414d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f1171d9b640 1 -- 172.21.15.55:0/1951170691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11740cf870 msgr2=0x7f11740ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f1171d9b640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11740cf870 0x7f11740ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f1171d9b640 1 -- 172.21.15.55:0/1951170691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11740cc8c0 con 0x7f11740d2d20 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f117259c640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11740cf870 0x7f11740ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f1171d9b640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 0x7f11740cc190 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f116800c3e0 tx=0x7f116800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f1162ffd640 1 -- 172.21.15.55:0/1951170691 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168017070 con 0x7f11740d2d20 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f1162ffd640 1 -- 172.21.15.55:0/1951170691 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1168005ce0 con 0x7f11740d2d20 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f11789ea640 1 -- 172.21.15.55:0/1951170691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11740ccba0 con 0x7f11740d2d20 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f1162ffd640 1 -- 172.21.15.55:0/1951170691 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680072f0 con 0x7f11740d2d20 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.122+0000 7f11789ea640 1 -- 172.21.15.55:0/1951170691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11740cd160 con 0x7f11740d2d20 2024-05-20T22:10:25.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.123+0000 7f11789ea640 1 -- 172.21.15.55:0/1951170691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1174069090 con 0x7f11740d2d20 2024-05-20T22:10:25.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.124+0000 7f1162ffd640 1 -- 172.21.15.55:0/1951170691 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f116801a070 con 0x7f11740d2d20 2024-05-20T22:10:25.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.124+0000 7f1162ffd640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f115406be30 0x7f115406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.124+0000 7f1162ffd640 1 -- 172.21.15.55:0/1951170691 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f116808a400 con 0x7f11740d2d20 2024-05-20T22:10:25.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.124+0000 7f117259c640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f115406be30 0x7f115406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.125+0000 7f117259c640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f115406be30 0x7f115406e340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1164034000 tx=0x7f116402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:25.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.129+0000 7f1162ffd640 1 -- 172.21.15.55:0/1951170691 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f11740d01c0 con 0x7f11740d2d20 2024-05-20T22:10:25.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.317+0000 7f11789ea640 1 -- 172.21.15.55:0/1951170691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11740d3740 con 0x7f11740d2d20 2024-05-20T22:10:25.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.318+0000 7f1162ffd640 1 -- 172.21.15.55:0/1951170691 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16955 (secure 0 0 0) 0x7f1168058c10 con 0x7f11740d2d20 2024-05-20T22:10:25.320 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:25.320 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":54,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:10:23.994806+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:10:22.369451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T22:10:22.970959+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:10:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 -- 172.21.15.55:0/1951170691 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f115406be30 msgr2=0x7f115406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f115406be30 0x7f115406e340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1164034000 tx=0x7f116402e000 comp rx=0 tx=0).stop 2024-05-20T22:10:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 -- 172.21.15.55:0/1951170691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 msgr2=0x7f11740cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 0x7f11740cc190 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f116800c3e0 tx=0x7f116800c8a0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 -- 172.21.15.55:0/1951170691 shutdown_connections 2024-05-20T22:10:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f115406be30 0x7f115406e340 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11740d3f00 0x7f117414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11740d2d20 0x7f11740cc190 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 --2- 172.21.15.55:0/1951170691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11740cf870 0x7f11740ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 -- 172.21.15.55:0/1951170691 >> 172.21.15.55:0/1951170691 conn(0x7f11740d1cb0 msgr2=0x7f117415ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 -- 172.21.15.55:0/1951170691 shutdown_connections 2024-05-20T22:10:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.321+0000 7f11789ea640 1 -- 172.21.15.55:0/1951170691 wait complete. 2024-05-20T22:10:25.334 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T22:10:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.451+0000 7feba6fea640 1 Processor -- start 2024-05-20T22:10:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.451+0000 7feba6fea640 1 -- start start 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feba6fea640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba00b29e0 0x7feba015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feba6fea640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feba00cf7f0 0x7feba00d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feba6fea640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba008c690 con 0x7feba008bcb0 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feba6fea640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feba00b2df0 con 0x7feba00cf7f0 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feba6fea640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feba008c970 con 0x7feba00b29e0 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feb9ffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7feba008bcb0 0x7feba00cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35470/0 (socket says 172.21.15.55:35470) 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feb9ffff640 1 -- 172.21.15.55:0/2037220841 learned_addr learned my addr 172.21.15.55:0/2037220841 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feba4d86640 1 --2- 172.21.15.55:0/2037220841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba00b29e0 0x7feba015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feba5587640 1 --2- 172.21.15.55:0/2037220841 >> v2:172.21.15.138:3300/0 conn(0x7feba00cf7f0 0x7feba00d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feb9f7fe640 1 -- 172.21.15.55:0/2037220841 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4071505190 0 0) 0x7feba008c690 con 0x7feba008bcb0 2024-05-20T22:10:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.452+0000 7feb9f7fe640 1 -- 172.21.15.55:0/2037220841 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb90003520 con 0x7feba008bcb0 2024-05-20T22:10:25.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.453+0000 7feba5587640 1 -- 172.21.15.55:0/2037220841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba00b29e0 msgr2=0x7feba015f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:25.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.453+0000 7feba5587640 1 --2- 172.21.15.55:0/2037220841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba00b29e0 0x7feba015f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.453+0000 7feba5587640 1 -- 172.21.15.55:0/2037220841 >> v1:172.21.15.55:6789/0 conn(0x7feba008bcb0 legacy=0x7feba00cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:25.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.453+0000 7feb9f7fe640 1 -- 172.21.15.55:0/2037220841 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3625135519 0 0) 0x7feb90003520 con 0x7feba008bcb0 2024-05-20T22:10:25.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.453+0000 7feba5587640 1 -- 172.21.15.55:0/2037220841 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba00d0de0 con 0x7feba00cf7f0 2024-05-20T22:10:25.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.454+0000 7feba5587640 1 --2- 172.21.15.55:0/2037220841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00cf7f0 0x7feba00d15e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feb98009d90 tx=0x7feb9802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89955c3f33633fe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:25.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.454+0000 7feb9f7fe640 1 -- 172.21.15.55:0/2037220841 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c001670 con 0x7feba00cf7f0 2024-05-20T22:10:25.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.454+0000 7feb9f7fe640 1 -- 172.21.15.55:0/2037220841 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb98007540 con 0x7feba00cf7f0 2024-05-20T22:10:25.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.454+0000 7feb9f7fe640 1 -- 172.21.15.55:0/2037220841 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98007910 con 0x7feba00cf7f0 2024-05-20T22:10:25.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.454+0000 7feba6fea640 1 -- 172.21.15.55:0/2037220841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00cf7f0 msgr2=0x7feba00d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:25.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.454+0000 7feba6fea640 1 --2- 172.21.15.55:0/2037220841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00cf7f0 0x7feba00d15e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feb98009d90 tx=0x7feb9802a5f0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.454+0000 7feba6fea640 1 -- 172.21.15.55:0/2037220841 shutdown_connections 2024-05-20T22:10:25.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.454+0000 7feba6fea640 1 --2- 172.21.15.55:0/2037220841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00cf7f0 0x7feba00d15e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.454+0000 7feba6fea640 1 --2- 172.21.15.55:0/2037220841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba00b29e0 0x7feba015f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.455+0000 7feba6fea640 1 -- 172.21.15.55:0/2037220841 >> 172.21.15.55:0/2037220841 conn(0x7feba00d2860 msgr2=0x7feba00d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:25.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.455+0000 7feba6fea640 1 -- 172.21.15.55:0/2037220841 shutdown_connections 2024-05-20T22:10:25.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.455+0000 7feba6fea640 1 -- 172.21.15.55:0/2037220841 wait complete. 2024-05-20T22:10:25.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.456+0000 7feba6fea640 1 Processor -- start 2024-05-20T22:10:25.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.456+0000 7feba6fea640 1 -- start start 2024-05-20T22:10:25.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba6fea640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feba008bcb0 0x7feba006ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba6fea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00b29e0 0x7feba006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba6fea640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba015f830 0x7feba0161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba6fea640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7feba008cc50 con 0x7feba008bcb0 2024-05-20T22:10:25.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba6fea640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feba015f620 con 0x7feba00b29e0 2024-05-20T22:10:25.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba6fea640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feba00b80e0 con 0x7feba015f830 2024-05-20T22:10:25.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba4d86640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feba008bcb0 0x7feba006ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba4d86640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feba008bcb0 0x7feba006ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55692/0 (socket says 172.21.15.55:55692) 2024-05-20T22:10:25.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba4d86640 1 -- 172.21.15.55:0/934221773 learned_addr learned my addr 172.21.15.55:0/934221773 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:25.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba5587640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba015f830 0x7feba0161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feb9ffff640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00b29e0 0x7feba006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba4d86640 1 -- 172.21.15.55:0/934221773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba015f830 msgr2=0x7feba0161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba4d86640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba015f830 0x7feba0161c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba4d86640 1 -- 172.21.15.55:0/934221773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00b29e0 msgr2=0x7feba006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba4d86640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00b29e0 0x7feba006d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba4d86640 1 -- 172.21.15.55:0/934221773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba015ff80 con 0x7feba008bcb0 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba5587640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba015f830 0x7feba0161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feb9ffff640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00b29e0 0x7feba006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.458+0000 7feba4d86640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feba008bcb0 0x7feba006ce80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feb9400e8a0 tx=0x7feb9400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.459+0000 7feb9d7fa640 1 -- 172.21.15.55:0/934221773 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94017020 con 0x7feba008bcb0 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.459+0000 7feb9d7fa640 1 -- 172.21.15.55:0/934221773 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94004240 con 0x7feba008bcb0 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.459+0000 7feba6fea640 1 -- 172.21.15.55:0/934221773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba00b5750 con 0x7feba008bcb0 2024-05-20T22:10:25.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.459+0000 7feb9d7fa640 1 -- 172.21.15.55:0/934221773 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9400c730 con 0x7feba008bcb0 2024-05-20T22:10:25.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.459+0000 7feba6fea640 1 -- 172.21.15.55:0/934221773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba00b5d10 con 0x7feba008bcb0 2024-05-20T22:10:25.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.460+0000 7feba6fea640 1 -- 172.21.15.55:0/934221773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba0069090 con 0x7feba008bcb0 2024-05-20T22:10:25.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.460+0000 7feb9d7fa640 1 -- 172.21.15.55:0/934221773 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb9401a020 con 0x7feba008bcb0 2024-05-20T22:10:25.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.461+0000 7feb9d7fa640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feb8406be30 0x7feb8406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:25.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.461+0000 7feb9d7fa640 1 -- 172.21.15.55:0/934221773 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7feb9408afd0 con 0x7feba008bcb0 2024-05-20T22:10:25.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.461+0000 7feb9ffff640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feb8406be30 0x7feb8406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:25.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.461+0000 7feb9ffff640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feb8406be30 0x7feb8406e340 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feba0154f90 tx=0x7feb8c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:25.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.465+0000 7feb9d7fa640 1 -- 172.21.15.55:0/934221773 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7feb94021030 con 0x7feba008bcb0 2024-05-20T22:10:25.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:25.691+0000 7feba6fea640 1 -- 172.21.15.55:0/934221773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feba00b27c0 con 0x7feba008bcb0 2024-05-20T22:10:26.595 INFO:tasks.ceph.mds.a.smithi055.stderr:2024-05-20T22:10:26.594+0000 7f57c4d7f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T22:10:26.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.635+0000 7feb9d7fa640 1 -- 172.21.15.55:0/934221773 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7feb94059880 con 0x7feba008bcb0 2024-05-20T22:10:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.638+0000 7feba6fea640 1 -- 172.21.15.55:0/934221773 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feb8406be30 msgr2=0x7feb8406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.638+0000 7feba6fea640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feb8406be30 0x7feb8406e340 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feba0154f90 tx=0x7feb8c009040 comp rx=0 tx=0).stop 2024-05-20T22:10:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.638+0000 7feba6fea640 1 -- 172.21.15.55:0/934221773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feba008bcb0 msgr2=0x7feba006ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:26.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.638+0000 7feba6fea640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feba008bcb0 0x7feba006ce80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feb9400e8a0 tx=0x7feb9400ed60 comp rx=0 tx=0).stop 2024-05-20T22:10:26.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.638+0000 7feba6fea640 1 -- 172.21.15.55:0/934221773 shutdown_connections 2024-05-20T22:10:26.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.638+0000 7feba6fea640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7feb8406be30 0x7feb8406e340 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:26.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.638+0000 7feba6fea640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba015f830 0x7feba0161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:26.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.638+0000 7feba6fea640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00b29e0 0x7feba006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:26.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.638+0000 7feba6fea640 1 --2- 172.21.15.55:0/934221773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feba008bcb0 0x7feba006ce80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:26.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.638+0000 7feba6fea640 1 -- 172.21.15.55:0/934221773 >> 172.21.15.55:0/934221773 conn(0x7feba00d2860 msgr2=0x7feba00706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:26.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.639+0000 7feba6fea640 1 -- 172.21.15.55:0/934221773 shutdown_connections 2024-05-20T22:10:26.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.639+0000 7feba6fea640 1 -- 172.21.15.55:0/934221773 wait complete. 2024-05-20T22:10:26.654 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-20T22:10:26.654 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T22:10:26.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.778+0000 7f85a34b8640 1 Processor -- start 2024-05-20T22:10:26.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.778+0000 7f85a34b8640 1 -- start start 2024-05-20T22:10:26.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a34b8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f859c0b2470 0x7f859c0b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:26.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a34b8640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f859c0cfad0 0x7f859c0d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:26.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a34b8640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f859c0b37b0 con 0x7f859c06d0b0 2024-05-20T22:10:26.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a34b8640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f859c0b2880 con 0x7f859c0cfad0 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a34b8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f859c06d8b0 con 0x7f859c0b2470 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a1a55640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f859c0cfad0 0x7f859c0d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a0a53640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f859c06d0b0 0x7f859c0cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35472/0 (socket says 172.21.15.55:35472) 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a0a53640 1 -- 172.21.15.55:0/1844551405 learned_addr learned my addr 172.21.15.55:0/1844551405 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a1254640 1 --2- 172.21.15.55:0/1844551405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f859c0b2470 0x7f859c0b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f8593fff640 1 -- 172.21.15.55:0/1844551405 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2135161688 0 0) 0x7f858c001110 con 0x7f859c06d0b0 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f8593fff640 1 -- 172.21.15.55:0/1844551405 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8588003520 con 0x7f859c06d0b0 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a1a55640 1 -- 172.21.15.55:0/1844551405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f859c0b2470 msgr2=0x7f859c0b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a1a55640 1 --2- 172.21.15.55:0/1844551405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f859c0b2470 0x7f859c0b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a1a55640 1 -- 172.21.15.55:0/1844551405 >> v1:172.21.15.55:6789/0 conn(0x7f859c06d0b0 legacy=0x7f859c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.780+0000 7f85a1a55640 1 -- 172.21.15.55:0/1844551405 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c06db90 con 0x7f859c0cfad0 2024-05-20T22:10:26.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.781+0000 7f85a1a55640 1 --2- 172.21.15.55:0/1844551405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c0cfad0 0x7f859c0d0b60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8594009d90 tx=0x7f859402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911903313ddfa130 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.781+0000 7f8593fff640 1 -- 172.21.15.55:0/1844551405 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594007740 con 0x7f859c0cfad0 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.781+0000 7f8593fff640 1 -- 172.21.15.55:0/1844551405 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85940078e0 con 0x7f859c0cfad0 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.781+0000 7f8593fff640 1 -- 172.21.15.55:0/1844551405 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594007cb0 con 0x7f859c0cfad0 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.782+0000 7f85a34b8640 1 -- 172.21.15.55:0/1844551405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c0cfad0 msgr2=0x7f859c0d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.782+0000 7f85a34b8640 1 --2- 172.21.15.55:0/1844551405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c0cfad0 0x7f859c0d0b60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8594009d90 tx=0x7f859402a660 comp rx=0 tx=0).stop 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.782+0000 7f85a34b8640 1 -- 172.21.15.55:0/1844551405 shutdown_connections 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.782+0000 7f85a34b8640 1 --2- 172.21.15.55:0/1844551405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c0cfad0 0x7f859c0d0b60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.782+0000 7f85a34b8640 1 --2- 172.21.15.55:0/1844551405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f859c0b2470 0x7f859c0b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.782+0000 7f85a34b8640 1 -- 172.21.15.55:0/1844551405 >> 172.21.15.55:0/1844551405 conn(0x7f859c09be10 msgr2=0x7f859c0d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.782+0000 7f85a34b8640 1 -- 172.21.15.55:0/1844551405 shutdown_connections 2024-05-20T22:10:26.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.782+0000 7f85a34b8640 1 -- 172.21.15.55:0/1844551405 wait complete. 2024-05-20T22:10:26.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.784+0000 7f85a34b8640 1 Processor -- start 2024-05-20T22:10:26.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.784+0000 7f85a34b8640 1 -- start start 2024-05-20T22:10:26.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.785+0000 7f85a34b8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c06d0b0 0x7f859c09fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:26.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.785+0000 7f85a34b8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f859c0b2470 0x7f859c0b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:26.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.785+0000 7f85a34b8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f859c0a0530 0x7f859c0b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:26.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.785+0000 7f85a34b8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f859c06d4c0 con 0x7f859c06d0b0 2024-05-20T22:10:26.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.785+0000 7f85a34b8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f859c0b3a30 con 0x7f859c0a0530 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.785+0000 7f85a34b8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f859c0b3f50 con 0x7f859c0b2470 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.785+0000 7f85a1254640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c06d0b0 0x7f859c09fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.785+0000 7f85a1254640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c06d0b0 0x7f859c09fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55706/0 (socket says 172.21.15.55:55706) 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.785+0000 7f85a1254640 1 -- 172.21.15.55:0/3105574228 learned_addr learned my addr 172.21.15.55:0/3105574228 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f85a0a53640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f859c0b2470 0x7f859c0b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f85a1a55640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f859c0a0530 0x7f859c0b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f85a1254640 1 -- 172.21.15.55:0/3105574228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f859c0b2470 msgr2=0x7f859c0b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f85a1254640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f859c0b2470 0x7f859c0b67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f85a1254640 1 -- 172.21.15.55:0/3105574228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f859c0a0530 msgr2=0x7f859c0b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f85a1254640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f859c0a0530 0x7f859c0b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f85a1254640 1 -- 172.21.15.55:0/3105574228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c155170 con 0x7f859c06d0b0 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f85a1254640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c06d0b0 0x7f859c09fff0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f859800e840 tx=0x7f859800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f8591ffb640 1 -- 172.21.15.55:0/3105574228 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598017070 con 0x7f859c06d0b0 2024-05-20T22:10:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f8591ffb640 1 -- 172.21.15.55:0/3105574228 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859800c940 con 0x7f859c06d0b0 2024-05-20T22:10:26.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f85a34b8640 1 -- 172.21.15.55:0/3105574228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859c151300 con 0x7f859c06d0b0 2024-05-20T22:10:26.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f8591ffb640 1 -- 172.21.15.55:0/3105574228 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859800cd10 con 0x7f859c06d0b0 2024-05-20T22:10:26.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.786+0000 7f85a34b8640 1 -- 172.21.15.55:0/3105574228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859c1518c0 con 0x7f859c06d0b0 2024-05-20T22:10:26.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.788+0000 7f85a34b8640 1 -- 172.21.15.55:0/3105574228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859c069090 con 0x7f859c06d0b0 2024-05-20T22:10:26.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.788+0000 7f8591ffb640 1 -- 172.21.15.55:0/3105574228 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f859801a070 con 0x7f859c06d0b0 2024-05-20T22:10:26.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.789+0000 7f8591ffb640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f858006be60 0x7f858006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:26.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.789+0000 7f8591ffb640 1 -- 172.21.15.55:0/3105574228 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f859808adf0 con 0x7f859c06d0b0 2024-05-20T22:10:26.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.789+0000 7f85a0a53640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f858006be60 0x7f858006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:26.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.789+0000 7f85a0a53640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f858006be60 0x7f858006e370 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f859c0cfe50 tx=0x7f858c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:26.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:26.793+0000 7f8591ffb640 1 -- 172.21.15.55:0/3105574228 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8598059610 con 0x7f859c06d0b0 2024-05-20T22:10:27.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.015+0000 7f85a34b8640 1 -- 172.21.15.55:0/3105574228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f859c0d35a0 con 0x7f859c06d0b0 2024-05-20T22:10:27.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.640+0000 7f8591ffb640 1 -- 172.21.15.55:0/3105574228 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f8598058f10 con 0x7f859c06d0b0 2024-05-20T22:10:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 -- 172.21.15.55:0/3105574228 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f858006be60 msgr2=0x7f858006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f858006be60 0x7f858006e370 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f859c0cfe50 tx=0x7f858c009040 comp rx=0 tx=0).stop 2024-05-20T22:10:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 -- 172.21.15.55:0/3105574228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c06d0b0 msgr2=0x7f859c09fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c06d0b0 0x7f859c09fff0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f859800e840 tx=0x7f859800ed00 comp rx=0 tx=0).stop 2024-05-20T22:10:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 -- 172.21.15.55:0/3105574228 shutdown_connections 2024-05-20T22:10:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f858006be60 0x7f858006e370 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f859c0a0530 0x7f859c0b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f859c0b2470 0x7f859c0b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 --2- 172.21.15.55:0/3105574228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f859c06d0b0 0x7f859c09fff0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:27.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 -- 172.21.15.55:0/3105574228 >> 172.21.15.55:0/3105574228 conn(0x7f859c09be10 msgr2=0x7f859c08bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:27.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 -- 172.21.15.55:0/3105574228 shutdown_connections 2024-05-20T22:10:27.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.643+0000 7f85a34b8640 1 -- 172.21.15.55:0/3105574228 wait complete. 2024-05-20T22:10:27.658 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T22:10:27.658 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:27.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.780+0000 7f12ecc51640 1 Processor -- start 2024-05-20T22:10:27.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.780+0000 7f12ecc51640 1 -- start start 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.781+0000 7f12ecc51640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e80b3b30 0x7f12e80b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.781+0000 7f12ecc51640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f12e815f100 0x7f12e806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.781+0000 7f12ecc51640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e80d3dd0 con 0x7f12e806c5c0 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.781+0000 7f12ecc51640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f12e80d40b0 con 0x7f12e815f100 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.781+0000 7f12ecc51640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f12e80d4390 con 0x7f12e80b3b30 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.782+0000 7f12e5d9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f12e815f100 0x7f12e806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.782+0000 7f12e6d9d640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f12e806c5c0 0x7f12e8162b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35476/0 (socket says 172.21.15.55:35476) 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.782+0000 7f12e6d9d640 1 -- 172.21.15.55:0/3316457548 learned_addr learned my addr 172.21.15.55:0/3316457548 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.782+0000 7f12e659c640 1 --2- 172.21.15.55:0/3316457548 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e80b3b30 0x7f12e80b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.782+0000 7f12e5d9b640 1 -- 172.21.15.55:0/3316457548 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e80b3b30 msgr2=0x7f12e80b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.782+0000 7f12e5d9b640 1 --2- 172.21.15.55:0/3316457548 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e80b3b30 0x7f12e80b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.782+0000 7f12e5d9b640 1 -- 172.21.15.55:0/3316457548 >> v1:172.21.15.55:6789/0 conn(0x7f12e806c5c0 legacy=0x7f12e8162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.782+0000 7f12e5d9b640 1 -- 172.21.15.55:0/3316457548 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e806b5a0 con 0x7f12e815f100 2024-05-20T22:10:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.782+0000 7f12e559a640 1 -- 172.21.15.55:0/3316457548 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345827052 0 0) 0x7f12e80d3dd0 con 0x7f12e806c5c0 2024-05-20T22:10:27.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.783+0000 7f12e5d9b640 1 --2- 172.21.15.55:0/3316457548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e815f100 0x7f12e806c080 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0097a0 tx=0x7f12dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a3c537bb436275 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:27.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.783+0000 7f12e559a640 1 -- 172.21.15.55:0/3316457548 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc037070 con 0x7f12e815f100 2024-05-20T22:10:27.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.783+0000 7f12e559a640 1 -- 172.21.15.55:0/3316457548 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12dc032440 con 0x7f12e815f100 2024-05-20T22:10:27.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.783+0000 7f12e559a640 1 -- 172.21.15.55:0/3316457548 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc032810 con 0x7f12e815f100 2024-05-20T22:10:27.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.784+0000 7f12ecc51640 1 -- 172.21.15.55:0/3316457548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e815f100 msgr2=0x7f12e806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:27.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.784+0000 7f12ecc51640 1 --2- 172.21.15.55:0/3316457548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e815f100 0x7f12e806c080 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0097a0 tx=0x7f12dc02a180 comp rx=0 tx=0).stop 2024-05-20T22:10:27.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.784+0000 7f12ecc51640 1 -- 172.21.15.55:0/3316457548 shutdown_connections 2024-05-20T22:10:27.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.784+0000 7f12ecc51640 1 --2- 172.21.15.55:0/3316457548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e815f100 0x7f12e806c080 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:27.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.784+0000 7f12ecc51640 1 --2- 172.21.15.55:0/3316457548 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e80b3b30 0x7f12e80b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:27.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.784+0000 7f12ecc51640 1 -- 172.21.15.55:0/3316457548 >> 172.21.15.55:0/3316457548 conn(0x7f12e80d51d0 msgr2=0x7f12e80b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:27.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.784+0000 7f12ecc51640 1 -- 172.21.15.55:0/3316457548 shutdown_connections 2024-05-20T22:10:27.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.784+0000 7f12ecc51640 1 -- 172.21.15.55:0/3316457548 wait complete. 2024-05-20T22:10:27.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.786+0000 7f12ecc51640 1 Processor -- start 2024-05-20T22:10:27.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.786+0000 7f12ecc51640 1 -- start start 2024-05-20T22:10:27.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12ecc51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12e806c5c0 0x7f12e8160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:27.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12ecc51640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e80b3b30 0x7f12e8143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:27.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12ecc51640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e815f100 0x7f12e8159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:27.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12ecc51640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f12e806c9d0 con 0x7f12e80b3b30 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12ecc51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12e806bb60 con 0x7f12e806c5c0 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12ecc51640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f12e806cb50 con 0x7f12e815f100 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12e5d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e80b3b30 0x7f12e8143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12e5d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e80b3b30 0x7f12e8143cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55724/0 (socket says 172.21.15.55:55724) 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12e5d9b640 1 -- 172.21.15.55:0/1650577502 learned_addr learned my addr 172.21.15.55:0/1650577502 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12e6d9d640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e815f100 0x7f12e8159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.787+0000 7f12e659c640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12e806c5c0 0x7f12e8160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12e5d9b640 1 -- 172.21.15.55:0/1650577502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e815f100 msgr2=0x7f12e8159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12e5d9b640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e815f100 0x7f12e8159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12e5d9b640 1 -- 172.21.15.55:0/1650577502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12e806c5c0 msgr2=0x7f12e8160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12e5d9b640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12e806c5c0 0x7f12e8160fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12e5d9b640 1 -- 172.21.15.55:0/1650577502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e8144210 con 0x7f12e80b3b30 2024-05-20T22:10:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12e659c640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12e806c5c0 0x7f12e8160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:27.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12e6d9d640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e815f100 0x7f12e8159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:27.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12e5d9b640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e80b3b30 0x7f12e8143cd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0098c0 tx=0x7f12dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:27.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12d6ffd640 1 -- 172.21.15.55:0/1650577502 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc037040 con 0x7f12e80b3b30 2024-05-20T22:10:27.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12d6ffd640 1 -- 172.21.15.55:0/1650577502 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12dc030770 con 0x7f12e80b3b30 2024-05-20T22:10:27.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12ecc51640 1 -- 172.21.15.55:0/1650577502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e81588c0 con 0x7f12e80b3b30 2024-05-20T22:10:27.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12d6ffd640 1 -- 172.21.15.55:0/1650577502 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc030b40 con 0x7f12e80b3b30 2024-05-20T22:10:27.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.788+0000 7f12ecc51640 1 -- 172.21.15.55:0/1650577502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e8159040 con 0x7f12e80b3b30 2024-05-20T22:10:27.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.789+0000 7f12ecc51640 1 -- 172.21.15.55:0/1650577502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e8069090 con 0x7f12e80b3b30 2024-05-20T22:10:27.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.790+0000 7f12d6ffd640 1 -- 172.21.15.55:0/1650577502 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12dc044020 con 0x7f12e80b3b30 2024-05-20T22:10:27.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.790+0000 7f12d6ffd640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12c806be30 0x7f12c806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:27.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.790+0000 7f12d6ffd640 1 -- 172.21.15.55:0/1650577502 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f12dc0ab380 con 0x7f12e80b3b30 2024-05-20T22:10:27.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.790+0000 7f12e659c640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12c806be30 0x7f12c806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:27.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.791+0000 7f12e659c640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12c806be30 0x7f12c806e340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12d8006fd0 tx=0x7f12d8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:27.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:27.795+0000 7f12d6ffd640 1 -- 172.21.15.55:0/1650577502 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f12dc032e20 con 0x7f12e80b3b30 2024-05-20T22:10:28.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.023+0000 7f12ecc51640 1 -- 172.21.15.55:0/1650577502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12e80d2ee0 con 0x7f12e80b3b30 2024-05-20T22:10:28.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.023+0000 7f12d6ffd640 1 -- 172.21.15.55:0/1650577502 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5192 (secure 0 0 0) 0x7f12dc0313a0 con 0x7f12e80b3b30 2024-05-20T22:10:28.025 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:28.025 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":35,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2682361422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2682361422},{"type":"v1","addr":"172.21.15.138:6834","nonce":2682361422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1792037486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1792037486},{"type":"v1","addr":"172.21.15.138:6835","nonce":1792037486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3650250553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3650250553},{"type":"v1","addr":"172.21.15.55:6836","nonce":3650250553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:27.016303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":35231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35231":{"gid":35231,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6837/2445634740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":2445634740},{"type":"v1","addr":"172.21.15.55:6837","nonce":2445634740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":35231,"qdb_cluster":[35231]},"id":3}]} 2024-05-20T22:10:28.025 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 35 2024-05-20T22:10:28.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 -- 172.21.15.55:0/1650577502 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12c806be30 msgr2=0x7f12c806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12c806be30 0x7f12c806e340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12d8006fd0 tx=0x7f12d8008040 comp rx=0 tx=0).stop 2024-05-20T22:10:28.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 -- 172.21.15.55:0/1650577502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e80b3b30 msgr2=0x7f12e8143cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e80b3b30 0x7f12e8143cd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0098c0 tx=0x7f12dc0046c0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 -- 172.21.15.55:0/1650577502 shutdown_connections 2024-05-20T22:10:28.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f12c806be30 0x7f12c806e340 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12e815f100 0x7f12e8159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12e80b3b30 0x7f12e8143cd0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 --2- 172.21.15.55:0/1650577502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12e806c5c0 0x7f12e8160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 -- 172.21.15.55:0/1650577502 >> 172.21.15.55:0/1650577502 conn(0x7f12e80d51d0 msgr2=0x7f12e815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 -- 172.21.15.55:0/1650577502 shutdown_connections 2024-05-20T22:10:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.026+0000 7f12ecc51640 1 -- 172.21.15.55:0/1650577502 wait complete. 2024-05-20T22:10:28.039 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:10:28.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.152+0000 7fc045026640 1 Processor -- start 2024-05-20T22:10:28.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.152+0000 7fc045026640 1 -- start start 2024-05-20T22:10:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc045026640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc04006d8f0 0x7fc04006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc045026640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc040163510 0x7fc040163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc045026640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04006e370 con 0x7fc040163e60 2024-05-20T22:10:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc045026640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fc0400d2000 con 0x7fc04006d8f0 2024-05-20T22:10:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc045026640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc0400b7e20 con 0x7fc040163510 2024-05-20T22:10:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc03ed9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc04006d8f0 0x7fc04006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc03f59e640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc040163e60 0x7fc0400b2bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35490/0 (socket says 172.21.15.55:35490) 2024-05-20T22:10:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc03f59e640 1 -- 172.21.15.55:0/13010187 learned_addr learned my addr 172.21.15.55:0/13010187 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc03e59c640 1 --2- 172.21.15.55:0/13010187 >> v2:172.21.15.138:3300/0 conn(0x7fc040163510 0x7fc040163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc03ed9d640 1 -- 172.21.15.55:0/13010187 >> v2:172.21.15.138:3300/0 conn(0x7fc040163510 msgr2=0x7fc040163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc03ed9d640 1 --2- 172.21.15.55:0/13010187 >> v2:172.21.15.138:3300/0 conn(0x7fc040163510 0x7fc040163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc03ed9d640 1 -- 172.21.15.55:0/13010187 >> v1:172.21.15.55:6789/0 conn(0x7fc040163e60 legacy=0x7fc0400b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc03ed9d640 1 -- 172.21.15.55:0/13010187 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc040164270 con 0x7fc04006d8f0 2024-05-20T22:10:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.154+0000 7fc03dd9b640 1 -- 172.21.15.55:0/13010187 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3440609931 0 0) 0x7fc04006e370 con 0x7fc040163e60 2024-05-20T22:10:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.155+0000 7fc03ed9d640 1 --2- 172.21.15.55:0/13010187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc04006d8f0 0x7fc04006dd00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc0400cf270 tx=0x7fc03002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b809015bcf1d70 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.155+0000 7fc03dd9b640 1 -- 172.21.15.55:0/13010187 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030037070 con 0x7fc04006d8f0 2024-05-20T22:10:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.155+0000 7fc03dd9b640 1 -- 172.21.15.55:0/13010187 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc030032440 con 0x7fc04006d8f0 2024-05-20T22:10:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.155+0000 7fc03dd9b640 1 -- 172.21.15.55:0/13010187 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03002ae60 con 0x7fc04006d8f0 2024-05-20T22:10:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.156+0000 7fc045026640 1 -- 172.21.15.55:0/13010187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc04006d8f0 msgr2=0x7fc04006dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.156+0000 7fc045026640 1 --2- 172.21.15.55:0/13010187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc04006d8f0 0x7fc04006dd00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc0400cf270 tx=0x7fc03002a110 comp rx=0 tx=0).stop 2024-05-20T22:10:28.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.156+0000 7fc045026640 1 -- 172.21.15.55:0/13010187 shutdown_connections 2024-05-20T22:10:28.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.156+0000 7fc045026640 1 --2- 172.21.15.55:0/13010187 >> v2:172.21.15.138:3300/0 conn(0x7fc040163510 0x7fc040163920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.156+0000 7fc045026640 1 --2- 172.21.15.55:0/13010187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc04006d8f0 0x7fc04006dd00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.156+0000 7fc045026640 1 -- 172.21.15.55:0/13010187 >> 172.21.15.55:0/13010187 conn(0x7fc0400b4340 msgr2=0x7fc0400b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:28.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.156+0000 7fc045026640 1 -- 172.21.15.55:0/13010187 shutdown_connections 2024-05-20T22:10:28.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.156+0000 7fc045026640 1 -- 172.21.15.55:0/13010187 wait complete. 2024-05-20T22:10:28.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.158+0000 7fc045026640 1 Processor -- start 2024-05-20T22:10:28.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.158+0000 7fc045026640 1 -- start start 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.159+0000 7fc045026640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04006d8f0 0x7fc040153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.159+0000 7fc045026640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc040163510 0x7fc0400b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.159+0000 7fc045026640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc040163e60 0x7fc0400b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.159+0000 7fc045026640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc04009c1e0 con 0x7fc040163510 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.159+0000 7fc045026640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc04009c060 con 0x7fc04006d8f0 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.159+0000 7fc045026640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc04006d1c0 con 0x7fc040163e60 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc040163510 0x7fc0400b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc040163510 0x7fc0400b66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55740/0 (socket says 172.21.15.55:55740) 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03e59c640 1 -- 172.21.15.55:0/4212242202 learned_addr learned my addr 172.21.15.55:0/4212242202 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03ed9d640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04006d8f0 0x7fc040153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03f59e640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc040163e60 0x7fc0400b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03e59c640 1 -- 172.21.15.55:0/4212242202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc040163e60 msgr2=0x7fc0400b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03e59c640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc040163e60 0x7fc0400b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03e59c640 1 -- 172.21.15.55:0/4212242202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04006d8f0 msgr2=0x7fc040153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03e59c640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04006d8f0 0x7fc040153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03e59c640 1 -- 172.21.15.55:0/4212242202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0400caf10 con 0x7fc040163510 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03ed9d640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04006d8f0 0x7fc040153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc03e59c640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc040163510 0x7fc0400b66f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc03400d5a0 tx=0x7fc03400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc0277fe640 1 -- 172.21.15.55:0/4212242202 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034019070 con 0x7fc040163510 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc045026640 1 -- 172.21.15.55:0/4212242202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0400cb1f0 con 0x7fc040163510 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc0277fe640 1 -- 172.21.15.55:0/4212242202 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc034004240 con 0x7fc040163510 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc0277fe640 1 -- 172.21.15.55:0/4212242202 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034003610 con 0x7fc040163510 2024-05-20T22:10:28.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.160+0000 7fc045026640 1 -- 172.21.15.55:0/4212242202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0400cb7b0 con 0x7fc040163510 2024-05-20T22:10:28.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.162+0000 7fc045026640 1 -- 172.21.15.55:0/4212242202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc040069090 con 0x7fc040163510 2024-05-20T22:10:28.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.162+0000 7fc0277fe640 1 -- 172.21.15.55:0/4212242202 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc03401c070 con 0x7fc040163510 2024-05-20T22:10:28.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.162+0000 7fc0277fe640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc02c06be60 0x7fc02c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.162+0000 7fc0277fe640 1 -- 172.21.15.55:0/4212242202 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7fc03408a9a0 con 0x7fc040163510 2024-05-20T22:10:28.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.162+0000 7fc03ed9d640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc02c06be60 0x7fc02c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.163+0000 7fc03ed9d640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc02c06be60 0x7fc02c06e370 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc030034000 tx=0x7fc03002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:28.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.167+0000 7fc0277fe640 1 -- 172.21.15.55:0/4212242202 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc0340589c0 con 0x7fc040163510 2024-05-20T22:10:28.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.354+0000 7fc045026640 1 -- 172.21.15.55:0/4212242202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc04009c770 con 0x7fc040163510 2024-05-20T22:10:28.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.355+0000 7fc0277fe640 1 -- 172.21.15.55:0/4212242202 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16942 (secure 0 0 0) 0x7fc03405ccc0 con 0x7fc040163510 2024-05-20T22:10:28.356 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:28.357 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":55,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:10:25.613446+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:10:22.369451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T22:10:22.970959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:10:28.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 -- 172.21.15.55:0/4212242202 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc02c06be60 msgr2=0x7fc02c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc02c06be60 0x7fc02c06e370 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc030034000 tx=0x7fc03002e000 comp rx=0 tx=0).stop 2024-05-20T22:10:28.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 -- 172.21.15.55:0/4212242202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc040163510 msgr2=0x7fc0400b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc040163510 0x7fc0400b66f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc03400d5a0 tx=0x7fc03400da60 comp rx=0 tx=0).stop 2024-05-20T22:10:28.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 -- 172.21.15.55:0/4212242202 shutdown_connections 2024-05-20T22:10:28.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc02c06be60 0x7fc02c06e370 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc040163e60 0x7fc0400b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc040163510 0x7fc0400b66f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 --2- 172.21.15.55:0/4212242202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04006d8f0 0x7fc040153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 -- 172.21.15.55:0/4212242202 >> 172.21.15.55:0/4212242202 conn(0x7fc0400b4340 msgr2=0x7fc0400b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:28.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 -- 172.21.15.55:0/4212242202 shutdown_connections 2024-05-20T22:10:28.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.358+0000 7fc045026640 1 -- 172.21.15.55:0/4212242202 wait complete. 2024-05-20T22:10:28.372 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T22:10:28.372 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:10:28.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.485+0000 7fa9138b8640 1 Processor -- start 2024-05-20T22:10:28.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.486+0000 7fa9138b8640 1 -- start start 2024-05-20T22:10:28.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa9138b8640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa90c06d8f0 0x7fa90c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa9138b8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa90c163510 0x7fa90c163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa9138b8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa90c06e370 con 0x7fa90c163e60 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa9138b8640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa90c0d2000 con 0x7fa90c06d8f0 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa9138b8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa90c0b7e20 con 0x7fa90c163510 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa911654640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa90c06d8f0 0x7fa90c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa911654640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa90c06d8f0 0x7fa90c06dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55754/0 (socket says 172.21.15.55:55754) 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa911654640 1 -- 172.21.15.55:0/241673141 learned_addr learned my addr 172.21.15.55:0/241673141 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa910e53640 1 --2- 172.21.15.55:0/241673141 >> v2:172.21.15.138:3300/0 conn(0x7fa90c163510 0x7fa90c163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa911654640 1 -- 172.21.15.55:0/241673141 >> v2:172.21.15.138:3300/0 conn(0x7fa90c163510 msgr2=0x7fa90c163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa911654640 1 --2- 172.21.15.55:0/241673141 >> v2:172.21.15.138:3300/0 conn(0x7fa90c163510 0x7fa90c163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa911654640 1 -- 172.21.15.55:0/241673141 >> v1:172.21.15.138:6789/0 conn(0x7fa90c163e60 legacy=0x7fa90c0b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.487+0000 7fa911654640 1 -- 172.21.15.55:0/241673141 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c164270 con 0x7fa90c06d8f0 2024-05-20T22:10:28.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.488+0000 7fa911654640 1 --2- 172.21.15.55:0/241673141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 0x7fa90c06dd00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0cf270 tx=0x7fa90802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f8c60eb5440558 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.488+0000 7fa903fff640 1 -- 172.21.15.55:0/241673141 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908004030 con 0x7fa90c06d8f0 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.488+0000 7fa903fff640 1 -- 172.21.15.55:0/241673141 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9080041d0 con 0x7fa90c06d8f0 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.488+0000 7fa903fff640 1 -- 172.21.15.55:0/241673141 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908003610 con 0x7fa90c06d8f0 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.489+0000 7fa9138b8640 1 -- 172.21.15.55:0/241673141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 msgr2=0x7fa90c06dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.489+0000 7fa9138b8640 1 --2- 172.21.15.55:0/241673141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 0x7fa90c06dd00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0cf270 tx=0x7fa90802a1c0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.489+0000 7fa9138b8640 1 -- 172.21.15.55:0/241673141 shutdown_connections 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.489+0000 7fa9138b8640 1 --2- 172.21.15.55:0/241673141 >> v2:172.21.15.138:3300/0 conn(0x7fa90c163510 0x7fa90c163920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.489+0000 7fa9138b8640 1 --2- 172.21.15.55:0/241673141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 0x7fa90c06dd00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.489+0000 7fa9138b8640 1 -- 172.21.15.55:0/241673141 >> 172.21.15.55:0/241673141 conn(0x7fa90c0b4340 msgr2=0x7fa90c0b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.489+0000 7fa9138b8640 1 -- 172.21.15.55:0/241673141 shutdown_connections 2024-05-20T22:10:28.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.489+0000 7fa9138b8640 1 -- 172.21.15.55:0/241673141 wait complete. 2024-05-20T22:10:28.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.491+0000 7fa9138b8640 1 Processor -- start 2024-05-20T22:10:28.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.491+0000 7fa9138b8640 1 -- start start 2024-05-20T22:10:28.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.492+0000 7fa9138b8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 0x7fa90c0cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.492+0000 7fa9138b8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa90c163510 0x7fa90c09e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.492+0000 7fa9138b8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa90c163e60 0x7fa90c0b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.492+0000 7fa9138b8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c09c1e0 con 0x7fa90c06d8f0 2024-05-20T22:10:28.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.492+0000 7fa9138b8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c09c060 con 0x7fa90c163510 2024-05-20T22:10:28.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.492+0000 7fa9138b8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa90c06d1c0 con 0x7fa90c163e60 2024-05-20T22:10:28.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.492+0000 7fa911654640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 0x7fa90c0cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.492+0000 7fa911654640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 0x7fa90c0cc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55766/0 (socket says 172.21.15.55:55766) 2024-05-20T22:10:28.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.492+0000 7fa911654640 1 -- 172.21.15.55:0/1751456265 learned_addr learned my addr 172.21.15.55:0/1751456265 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa910e53640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa90c163510 0x7fa90c09e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa911654640 1 -- 172.21.15.55:0/1751456265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa90c163e60 msgr2=0x7fa90c0b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa911654640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa90c163e60 0x7fa90c0b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa911654640 1 -- 172.21.15.55:0/1751456265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa90c163510 msgr2=0x7fa90c09e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa911654640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa90c163510 0x7fa90c09e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa911654640 1 -- 172.21.15.55:0/1751456265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c09f1d0 con 0x7fa90c06d8f0 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa910e53640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa90c163510 0x7fa90c09e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa911654640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 0x7fa90c0cc2b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa90802a200 tx=0x7fa9080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa901ffb640 1 -- 172.21.15.55:0/1751456265 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908041020 con 0x7fa90c06d8f0 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa901ffb640 1 -- 172.21.15.55:0/1751456265 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa908003c50 con 0x7fa90c06d8f0 2024-05-20T22:10:28.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa901ffb640 1 -- 172.21.15.55:0/1751456265 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908004030 con 0x7fa90c06d8f0 2024-05-20T22:10:28.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa9138b8640 1 -- 172.21.15.55:0/1751456265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90c09f4b0 con 0x7fa90c06d8f0 2024-05-20T22:10:28.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.493+0000 7fa9138b8640 1 -- 172.21.15.55:0/1751456265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90c09fa70 con 0x7fa90c06d8f0 2024-05-20T22:10:28.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.494+0000 7fa9138b8640 1 -- 172.21.15.55:0/1751456265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90c069090 con 0x7fa90c06d8f0 2024-05-20T22:10:28.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.495+0000 7fa901ffb640 1 -- 172.21.15.55:0/1751456265 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa908044020 con 0x7fa90c06d8f0 2024-05-20T22:10:28.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.495+0000 7fa901ffb640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa8f006be60 0x7fa8f006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.496+0000 7fa901ffb640 1 -- 172.21.15.55:0/1751456265 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7fa9080abbe0 con 0x7fa90c06d8f0 2024-05-20T22:10:28.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.496+0000 7fa910e53640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa8f006be60 0x7fa8f006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.496+0000 7fa910e53640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa8f006be60 0x7fa8f006e370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0c9410 tx=0x7fa8fc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:28.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.500+0000 7fa901ffb640 1 -- 172.21.15.55:0/1751456265 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa90804b030 con 0x7fa90c06d8f0 2024-05-20T22:10:28.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.683+0000 7fa9138b8640 1 -- 172.21.15.55:0/1751456265 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa90c0ceb00 con 0x7fa8f006be60 2024-05-20T22:10:28.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.689+0000 7fa901ffb640 1 -- 172.21.15.55:0/1751456265 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251923 (secure 0 0 0) 0x7fa90c0ceb00 con 0x7fa8f006be60 2024-05-20T22:10:28.693 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:28.695 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:10:28.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 -- 172.21.15.55:0/1751456265 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa8f006be60 msgr2=0x7fa8f006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa8f006be60 0x7fa8f006e370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0c9410 tx=0x7fa8fc008040 comp rx=0 tx=0).stop 2024-05-20T22:10:28.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 -- 172.21.15.55:0/1751456265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 msgr2=0x7fa90c0cc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 0x7fa90c0cc2b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa90802a200 tx=0x7fa9080046c0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 -- 172.21.15.55:0/1751456265 shutdown_connections 2024-05-20T22:10:28.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa8f006be60 0x7fa8f006e370 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa90c163e60 0x7fa90c0b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa90c163510 0x7fa90c09e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 --2- 172.21.15.55:0/1751456265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa90c06d8f0 0x7fa90c0cc2b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 -- 172.21.15.55:0/1751456265 >> 172.21.15.55:0/1751456265 conn(0x7fa90c0b4340 msgr2=0x7fa90c0b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:28.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 -- 172.21.15.55:0/1751456265 shutdown_connections 2024-05-20T22:10:28.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.693+0000 7fa9138b8640 1 -- 172.21.15.55:0/1751456265 wait complete. 2024-05-20T22:10:28.705 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-05-20T22:10:28.363938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243240,"kb_used_data":6260,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731480,"statfs":{"total":773094113280,"available":772845035520,"internally_reserved":0,"allocated":6410240,"data_stored":3508904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.767463"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649069+0000","last_change":"2024-05-20T22:10:24.637185+0000","last_active":"2024-05-20T22:10:24.649069+0000","last_peered":"2024-05-20T22:10:24.649069+0000","last_clean":"2024-05-20T22:10:24.649069+0000","last_became_active":"2024-05-20T22:10:24.636736+0000","last_became_peered":"2024-05-20T22:10:24.636736+0000","last_unstale":"2024-05-20T22:10:24.649069+0000","last_undegraded":"2024-05-20T22:10:24.649069+0000","last_fullsized":"2024-05-20T22:10:24.649069+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:09.538298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619134+0000","last_change":"2024-05-20T22:10:24.625208+0000","last_active":"2024-05-20T22:10:25.619134+0000","last_peered":"2024-05-20T22:10:25.619134+0000","last_clean":"2024-05-20T22:10:25.619134+0000","last_became_active":"2024-05-20T22:10:24.624880+0000","last_became_peered":"2024-05-20T22:10:24.624880+0000","last_unstale":"2024-05-20T22:10:25.619134+0000","last_undegraded":"2024-05-20T22:10:25.619134+0000","last_fullsized":"2024-05-20T22:10:25.619134+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:37:18.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636411+0000","last_change":"2024-05-20T22:10:24.641686+0000","last_active":"2024-05-20T22:10:25.636411+0000","last_peered":"2024-05-20T22:10:25.636411+0000","last_clean":"2024-05-20T22:10:25.636411+0000","last_became_active":"2024-05-20T22:10:24.641225+0000","last_became_peered":"2024-05-20T22:10:24.641225+0000","last_unstale":"2024-05-20T22:10:25.636411+0000","last_undegraded":"2024-05-20T22:10:25.636411+0000","last_fullsized":"2024-05-20T22:10:25.636411+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:42:48.287564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618640+0000","last_change":"2024-05-20T22:10:24.646369+0000","last_active":"2024-05-20T22:10:25.618640+0000","last_peered":"2024-05-20T22:10:25.618640+0000","last_clean":"2024-05-20T22:10:25.618640+0000","last_became_active":"2024-05-20T22:10:24.646057+0000","last_became_peered":"2024-05-20T22:10:24.646057+0000","last_unstale":"2024-05-20T22:10:25.618640+0000","last_undegraded":"2024-05-20T22:10:25.618640+0000","last_fullsized":"2024-05-20T22:10:25.618640+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:09:11.489783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619029+0000","last_change":"2024-05-20T22:10:24.637171+0000","last_active":"2024-05-20T22:10:25.619029+0000","last_peered":"2024-05-20T22:10:25.619029+0000","last_clean":"2024-05-20T22:10:25.619029+0000","last_became_active":"2024-05-20T22:10:24.636816+0000","last_became_peered":"2024-05-20T22:10:24.636816+0000","last_unstale":"2024-05-20T22:10:25.619029+0000","last_undegraded":"2024-05-20T22:10:25.619029+0000","last_fullsized":"2024-05-20T22:10:25.619029+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:15:02.617975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649536+0000","last_change":"2024-05-20T22:10:24.637058+0000","last_active":"2024-05-20T22:10:24.649536+0000","last_peered":"2024-05-20T22:10:24.649536+0000","last_clean":"2024-05-20T22:10:24.649536+0000","last_became_active":"2024-05-20T22:10:24.636646+0000","last_became_peered":"2024-05-20T22:10:24.636646+0000","last_unstale":"2024-05-20T22:10:24.649536+0000","last_undegraded":"2024-05-20T22:10:24.649536+0000","last_fullsized":"2024-05-20T22:10:24.649536+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:34:27.795802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636139+0000","last_change":"2024-05-20T22:10:24.646018+0000","last_active":"2024-05-20T22:10:25.636139+0000","last_peered":"2024-05-20T22:10:25.636139+0000","last_clean":"2024-05-20T22:10:25.636139+0000","last_became_active":"2024-05-20T22:10:24.645680+0000","last_became_peered":"2024-05-20T22:10:24.645680+0000","last_unstale":"2024-05-20T22:10:25.636139+0000","last_undegraded":"2024-05-20T22:10:25.636139+0000","last_fullsized":"2024-05-20T22:10:25.636139+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:09:16.410294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620361+0000","last_change":"2024-05-20T22:10:24.658719+0000","last_active":"2024-05-20T22:10:25.620361+0000","last_peered":"2024-05-20T22:10:25.620361+0000","last_clean":"2024-05-20T22:10:25.620361+0000","last_became_active":"2024-05-20T22:10:24.658403+0000","last_became_peered":"2024-05-20T22:10:24.658403+0000","last_unstale":"2024-05-20T22:10:25.620361+0000","last_undegraded":"2024-05-20T22:10:25.620361+0000","last_fullsized":"2024-05-20T22:10:25.620361+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:27:02.138910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620704+0000","last_change":"2024-05-20T22:10:24.654515+0000","last_active":"2024-05-20T22:10:25.620704+0000","last_peered":"2024-05-20T22:10:25.620704+0000","last_clean":"2024-05-20T22:10:25.620704+0000","last_became_active":"2024-05-20T22:10:24.654185+0000","last_became_peered":"2024-05-20T22:10:24.654185+0000","last_unstale":"2024-05-20T22:10:25.620704+0000","last_undegraded":"2024-05-20T22:10:25.620704+0000","last_fullsized":"2024-05-20T22:10:25.620704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:42.948141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649602+0000","last_change":"2024-05-20T22:10:24.637396+0000","last_active":"2024-05-20T22:10:24.649602+0000","last_peered":"2024-05-20T22:10:24.649602+0000","last_clean":"2024-05-20T22:10:24.649602+0000","last_became_active":"2024-05-20T22:10:24.636685+0000","last_became_peered":"2024-05-20T22:10:24.636685+0000","last_unstale":"2024-05-20T22:10:24.649602+0000","last_undegraded":"2024-05-20T22:10:24.649602+0000","last_fullsized":"2024-05-20T22:10:24.649602+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:44.372720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.646536+0000","last_change":"2024-05-20T22:10:24.635025+0000","last_active":"2024-05-20T22:10:24.646536+0000","last_peered":"2024-05-20T22:10:24.646536+0000","last_clean":"2024-05-20T22:10:24.646536+0000","last_became_active":"2024-05-20T22:10:24.634689+0000","last_became_peered":"2024-05-20T22:10:24.634689+0000","last_unstale":"2024-05-20T22:10:24.646536+0000","last_undegraded":"2024-05-20T22:10:24.646536+0000","last_fullsized":"2024-05-20T22:10:24.646536+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:55.858769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620717+0000","last_change":"2024-05-20T22:10:24.645036+0000","last_active":"2024-05-20T22:10:25.620717+0000","last_peered":"2024-05-20T22:10:25.620717+0000","last_clean":"2024-05-20T22:10:25.620717+0000","last_became_active":"2024-05-20T22:10:24.644675+0000","last_became_peered":"2024-05-20T22:10:24.644675+0000","last_unstale":"2024-05-20T22:10:25.620717+0000","last_undegraded":"2024-05-20T22:10:25.620717+0000","last_fullsized":"2024-05-20T22:10:25.620717+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:27:01.866406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649734+0000","last_change":"2024-05-20T22:10:24.637220+0000","last_active":"2024-05-20T22:10:24.649734+0000","last_peered":"2024-05-20T22:10:24.649734+0000","last_clean":"2024-05-20T22:10:24.649734+0000","last_became_active":"2024-05-20T22:10:24.636770+0000","last_became_peered":"2024-05-20T22:10:24.636770+0000","last_unstale":"2024-05-20T22:10:24.649734+0000","last_undegraded":"2024-05-20T22:10:24.649734+0000","last_fullsized":"2024-05-20T22:10:24.649734+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:27:55.130179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636444+0000","last_change":"2024-05-20T22:10:24.646047+0000","last_active":"2024-05-20T22:10:25.636444+0000","last_peered":"2024-05-20T22:10:25.636444+0000","last_clean":"2024-05-20T22:10:25.636444+0000","last_became_active":"2024-05-20T22:10:24.645695+0000","last_became_peered":"2024-05-20T22:10:24.645695+0000","last_unstale":"2024-05-20T22:10:25.636444+0000","last_undegraded":"2024-05-20T22:10:25.636444+0000","last_fullsized":"2024-05-20T22:10:25.636444+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:54:44.645915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617999+0000","last_change":"2024-05-20T22:10:24.629115+0000","last_active":"2024-05-20T22:10:25.617999+0000","last_peered":"2024-05-20T22:10:25.617999+0000","last_clean":"2024-05-20T22:10:25.617999+0000","last_became_active":"2024-05-20T22:10:24.628655+0000","last_became_peered":"2024-05-20T22:10:24.628655+0000","last_unstale":"2024-05-20T22:10:25.617999+0000","last_undegraded":"2024-05-20T22:10:25.617999+0000","last_fullsized":"2024-05-20T22:10:25.617999+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:42:53.054927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649685+0000","last_change":"2024-05-20T22:10:24.637788+0000","last_active":"2024-05-20T22:10:24.649685+0000","last_peered":"2024-05-20T22:10:24.649685+0000","last_clean":"2024-05-20T22:10:24.649685+0000","last_became_active":"2024-05-20T22:10:24.637460+0000","last_became_peered":"2024-05-20T22:10:24.637460+0000","last_unstale":"2024-05-20T22:10:24.649685+0000","last_undegraded":"2024-05-20T22:10:24.649685+0000","last_fullsized":"2024-05-20T22:10:24.649685+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:04.895434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636845+0000","last_change":"2024-05-20T22:10:24.643216+0000","last_active":"2024-05-20T22:10:25.636845+0000","last_peered":"2024-05-20T22:10:25.636845+0000","last_clean":"2024-05-20T22:10:25.636845+0000","last_became_active":"2024-05-20T22:10:24.642746+0000","last_became_peered":"2024-05-20T22:10:24.642746+0000","last_unstale":"2024-05-20T22:10:25.636845+0000","last_undegraded":"2024-05-20T22:10:25.636845+0000","last_fullsized":"2024-05-20T22:10:25.636845+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:27:02.138910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619339+0000","last_change":"2024-05-20T22:10:24.626819+0000","last_active":"2024-05-20T22:10:25.619339+0000","last_peered":"2024-05-20T22:10:25.619339+0000","last_clean":"2024-05-20T22:10:25.619339+0000","last_became_active":"2024-05-20T22:10:24.626158+0000","last_became_peered":"2024-05-20T22:10:24.626158+0000","last_unstale":"2024-05-20T22:10:25.619339+0000","last_undegraded":"2024-05-20T22:10:25.619339+0000","last_fullsized":"2024-05-20T22:10:25.619339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:42:48.287564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636479+0000","last_change":"2024-05-20T22:10:24.637398+0000","last_active":"2024-05-20T22:10:25.636479+0000","last_peered":"2024-05-20T22:10:25.636479+0000","last_clean":"2024-05-20T22:10:25.636479+0000","last_became_active":"2024-05-20T22:10:24.637082+0000","last_became_peered":"2024-05-20T22:10:24.637082+0000","last_unstale":"2024-05-20T22:10:25.636479+0000","last_undegraded":"2024-05-20T22:10:25.636479+0000","last_fullsized":"2024-05-20T22:10:25.636479+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:06:52.487566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.639960+0000","last_change":"2024-05-20T22:10:24.637410+0000","last_active":"2024-05-20T22:10:24.639960+0000","last_peered":"2024-05-20T22:10:24.639960+0000","last_clean":"2024-05-20T22:10:24.639960+0000","last_became_active":"2024-05-20T22:10:24.637103+0000","last_became_peered":"2024-05-20T22:10:24.637103+0000","last_unstale":"2024-05-20T22:10:24.639960+0000","last_undegraded":"2024-05-20T22:10:24.639960+0000","last_fullsized":"2024-05-20T22:10:24.639960+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:38.332210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.634629+0000","last_change":"2024-05-20T22:10:24.630950+0000","last_active":"2024-05-20T22:10:24.634629+0000","last_peered":"2024-05-20T22:10:24.634629+0000","last_clean":"2024-05-20T22:10:24.634629+0000","last_became_active":"2024-05-20T22:10:24.630579+0000","last_became_peered":"2024-05-20T22:10:24.630579+0000","last_unstale":"2024-05-20T22:10:24.634629+0000","last_undegraded":"2024-05-20T22:10:24.634629+0000","last_fullsized":"2024-05-20T22:10:24.634629+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:27:55.130179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620476+0000","last_change":"2024-05-20T22:10:24.660055+0000","last_active":"2024-05-20T22:10:25.620476+0000","last_peered":"2024-05-20T22:10:25.620476+0000","last_clean":"2024-05-20T22:10:25.620476+0000","last_became_active":"2024-05-20T22:10:24.659142+0000","last_became_peered":"2024-05-20T22:10:24.659142+0000","last_unstale":"2024-05-20T22:10:25.620476+0000","last_undegraded":"2024-05-20T22:10:25.620476+0000","last_fullsized":"2024-05-20T22:10:25.620476+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:53:56.291962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.646577+0000","last_change":"2024-05-20T22:10:24.634294+0000","last_active":"2024-05-20T22:10:24.646577+0000","last_peered":"2024-05-20T22:10:24.646577+0000","last_clean":"2024-05-20T22:10:24.646577+0000","last_became_active":"2024-05-20T22:10:24.633962+0000","last_became_peered":"2024-05-20T22:10:24.633962+0000","last_unstale":"2024-05-20T22:10:24.646577+0000","last_undegraded":"2024-05-20T22:10:24.646577+0000","last_fullsized":"2024-05-20T22:10:24.646577+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:43:17.088437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636094+0000","last_change":"2024-05-20T22:10:24.650199+0000","last_active":"2024-05-20T22:10:25.636094+0000","last_peered":"2024-05-20T22:10:25.636094+0000","last_clean":"2024-05-20T22:10:25.636094+0000","last_became_active":"2024-05-20T22:10:24.644859+0000","last_became_peered":"2024-05-20T22:10:24.644859+0000","last_unstale":"2024-05-20T22:10:25.636094+0000","last_undegraded":"2024-05-20T22:10:25.636094+0000","last_fullsized":"2024-05-20T22:10:25.636094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:36.792344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636961+0000","last_change":"2024-05-20T22:10:24.649012+0000","last_active":"2024-05-20T22:10:25.636961+0000","last_peered":"2024-05-20T22:10:25.636961+0000","last_clean":"2024-05-20T22:10:25.636961+0000","last_became_active":"2024-05-20T22:10:24.648675+0000","last_became_peered":"2024-05-20T22:10:24.648675+0000","last_unstale":"2024-05-20T22:10:25.636961+0000","last_undegraded":"2024-05-20T22:10:25.636961+0000","last_fullsized":"2024-05-20T22:10:25.636961+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:31.732256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620431+0000","last_change":"2024-05-20T22:10:24.653378+0000","last_active":"2024-05-20T22:10:25.620431+0000","last_peered":"2024-05-20T22:10:25.620431+0000","last_clean":"2024-05-20T22:10:25.620431+0000","last_became_active":"2024-05-20T22:10:24.651745+0000","last_became_peered":"2024-05-20T22:10:24.651745+0000","last_unstale":"2024-05-20T22:10:25.620431+0000","last_undegraded":"2024-05-20T22:10:25.620431+0000","last_fullsized":"2024-05-20T22:10:25.620431+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:31.732256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.650482+0000","last_change":"2024-05-20T22:10:24.642165+0000","last_active":"2024-05-20T22:10:24.650482+0000","last_peered":"2024-05-20T22:10:24.650482+0000","last_clean":"2024-05-20T22:10:24.650482+0000","last_became_active":"2024-05-20T22:10:24.641408+0000","last_became_peered":"2024-05-20T22:10:24.641408+0000","last_unstale":"2024-05-20T22:10:24.650482+0000","last_undegraded":"2024-05-20T22:10:24.650482+0000","last_fullsized":"2024-05-20T22:10:24.650482+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:15:02.617975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.640533+0000","last_change":"2024-05-20T22:10:24.638993+0000","last_active":"2024-05-20T22:10:24.640533+0000","last_peered":"2024-05-20T22:10:24.640533+0000","last_clean":"2024-05-20T22:10:24.640533+0000","last_became_active":"2024-05-20T22:10:24.638687+0000","last_became_peered":"2024-05-20T22:10:24.638687+0000","last_unstale":"2024-05-20T22:10:24.640533+0000","last_undegraded":"2024-05-20T22:10:24.640533+0000","last_fullsized":"2024-05-20T22:10:24.640533+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:15:02.617975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636714+0000","last_change":"2024-05-20T22:10:24.642365+0000","last_active":"2024-05-20T22:10:25.636714+0000","last_peered":"2024-05-20T22:10:25.636714+0000","last_clean":"2024-05-20T22:10:25.636714+0000","last_became_active":"2024-05-20T22:10:24.642032+0000","last_became_peered":"2024-05-20T22:10:24.642032+0000","last_unstale":"2024-05-20T22:10:25.636714+0000","last_undegraded":"2024-05-20T22:10:25.636714+0000","last_fullsized":"2024-05-20T22:10:25.636714+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:19:52.710100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618337+0000","last_change":"2024-05-20T22:10:24.629429+0000","last_active":"2024-05-20T22:10:25.618337+0000","last_peered":"2024-05-20T22:10:25.618337+0000","last_clean":"2024-05-20T22:10:25.618337+0000","last_became_active":"2024-05-20T22:10:24.628979+0000","last_became_peered":"2024-05-20T22:10:24.628979+0000","last_unstale":"2024-05-20T22:10:25.618337+0000","last_undegraded":"2024-05-20T22:10:25.618337+0000","last_fullsized":"2024-05-20T22:10:25.618337+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:40:24.202923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636210+0000","last_change":"2024-05-20T22:10:24.645238+0000","last_active":"2024-05-20T22:10:25.636210+0000","last_peered":"2024-05-20T22:10:25.636210+0000","last_clean":"2024-05-20T22:10:25.636210+0000","last_became_active":"2024-05-20T22:10:24.644612+0000","last_became_peered":"2024-05-20T22:10:24.644612+0000","last_unstale":"2024-05-20T22:10:25.636210+0000","last_undegraded":"2024-05-20T22:10:25.636210+0000","last_fullsized":"2024-05-20T22:10:25.636210+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:37:18.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.652894+0000","last_change":"2024-05-20T22:10:24.648561+0000","last_active":"2024-05-20T22:10:24.652894+0000","last_peered":"2024-05-20T22:10:24.652894+0000","last_clean":"2024-05-20T22:10:24.652894+0000","last_became_active":"2024-05-20T22:10:24.648253+0000","last_became_peered":"2024-05-20T22:10:24.648253+0000","last_unstale":"2024-05-20T22:10:24.652894+0000","last_undegraded":"2024-05-20T22:10:24.652894+0000","last_fullsized":"2024-05-20T22:10:24.652894+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:06.964355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617716+0000","last_change":"2024-05-20T22:10:24.622527+0000","last_active":"2024-05-20T22:10:25.617716+0000","last_peered":"2024-05-20T22:10:25.617716+0000","last_clean":"2024-05-20T22:10:25.617716+0000","last_became_active":"2024-05-20T22:10:24.622146+0000","last_became_peered":"2024-05-20T22:10:24.622146+0000","last_unstale":"2024-05-20T22:10:25.617716+0000","last_undegraded":"2024-05-20T22:10:25.617716+0000","last_fullsized":"2024-05-20T22:10:25.617716+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:06.964355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.650278+0000","last_change":"2024-05-20T22:10:23.617582+0000","last_active":"2024-05-20T22:10:24.650278+0000","last_peered":"2024-05-20T22:10:24.650278+0000","last_clean":"2024-05-20T22:10:24.650278+0000","last_became_active":"2024-05-20T22:10:23.617267+0000","last_became_peered":"2024-05-20T22:10:23.617267+0000","last_unstale":"2024-05-20T22:10:24.650278+0000","last_undegraded":"2024-05-20T22:10:24.650278+0000","last_fullsized":"2024-05-20T22:10:24.650278+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:06:34.084033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.625950+0000","last_change":"2024-05-20T22:10:24.623360+0000","last_active":"2024-05-20T22:10:24.625950+0000","last_peered":"2024-05-20T22:10:24.625950+0000","last_clean":"2024-05-20T22:10:24.625950+0000","last_became_active":"2024-05-20T22:10:24.623022+0000","last_became_peered":"2024-05-20T22:10:24.623022+0000","last_unstale":"2024-05-20T22:10:24.625950+0000","last_undegraded":"2024-05-20T22:10:24.625950+0000","last_fullsized":"2024-05-20T22:10:24.625950+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:05.215640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.658252+0000","last_change":"2024-05-20T22:10:23.623800+0000","last_active":"2024-05-20T22:10:24.658252+0000","last_peered":"2024-05-20T22:10:24.658252+0000","last_clean":"2024-05-20T22:10:24.658252+0000","last_became_active":"2024-05-20T22:10:23.623481+0000","last_became_peered":"2024-05-20T22:10:23.623481+0000","last_unstale":"2024-05-20T22:10:24.658252+0000","last_undegraded":"2024-05-20T22:10:24.658252+0000","last_fullsized":"2024-05-20T22:10:24.658252+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:06:34.084033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.634782+0000","last_change":"2024-05-20T22:10:24.630951+0000","last_active":"2024-05-20T22:10:24.634782+0000","last_peered":"2024-05-20T22:10:24.634782+0000","last_clean":"2024-05-20T22:10:24.634782+0000","last_became_active":"2024-05-20T22:10:24.630327+0000","last_became_peered":"2024-05-20T22:10:24.630327+0000","last_unstale":"2024-05-20T22:10:24.634782+0000","last_undegraded":"2024-05-20T22:10:24.634782+0000","last_fullsized":"2024-05-20T22:10:24.634782+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:04.895434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.650329+0000","last_change":"2024-05-20T22:10:23.620076+0000","last_active":"2024-05-20T22:10:24.650329+0000","last_peered":"2024-05-20T22:10:24.650329+0000","last_clean":"2024-05-20T22:10:24.650329+0000","last_became_active":"2024-05-20T22:10:23.619722+0000","last_became_peered":"2024-05-20T22:10:23.619722+0000","last_unstale":"2024-05-20T22:10:24.650329+0000","last_undegraded":"2024-05-20T22:10:24.650329+0000","last_fullsized":"2024-05-20T22:10:24.650329+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:53.964769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620441+0000","last_change":"2024-05-20T22:10:24.650265+0000","last_active":"2024-05-20T22:10:25.620441+0000","last_peered":"2024-05-20T22:10:25.620441+0000","last_clean":"2024-05-20T22:10:25.620441+0000","last_became_active":"2024-05-20T22:10:24.647516+0000","last_became_peered":"2024-05-20T22:10:24.647516+0000","last_unstale":"2024-05-20T22:10:25.620441+0000","last_undegraded":"2024-05-20T22:10:25.620441+0000","last_fullsized":"2024-05-20T22:10:25.620441+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:15.642998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.660159+0000","last_change":"2024-05-20T22:10:23.628714+0000","last_active":"2024-05-20T22:10:24.660159+0000","last_peered":"2024-05-20T22:10:24.660159+0000","last_clean":"2024-05-20T22:10:24.660159+0000","last_became_active":"2024-05-20T22:10:23.628411+0000","last_became_peered":"2024-05-20T22:10:23.628411+0000","last_unstale":"2024-05-20T22:10:24.660159+0000","last_undegraded":"2024-05-20T22:10:24.660159+0000","last_fullsized":"2024-05-20T22:10:24.660159+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:53.351292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618477+0000","last_change":"2024-05-20T22:10:24.654011+0000","last_active":"2024-05-20T22:10:25.618477+0000","last_peered":"2024-05-20T22:10:25.618477+0000","last_clean":"2024-05-20T22:10:25.618477+0000","last_became_active":"2024-05-20T22:10:24.653737+0000","last_became_peered":"2024-05-20T22:10:24.653737+0000","last_unstale":"2024-05-20T22:10:25.618477+0000","last_undegraded":"2024-05-20T22:10:25.618477+0000","last_fullsized":"2024-05-20T22:10:25.618477+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:01.712783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636410+0000","last_change":"2024-05-20T22:10:23.626863+0000","last_active":"2024-05-20T22:10:25.636410+0000","last_peered":"2024-05-20T22:10:25.636410+0000","last_clean":"2024-05-20T22:10:25.636410+0000","last_became_active":"2024-05-20T22:10:23.625662+0000","last_became_peered":"2024-05-20T22:10:23.625662+0000","last_unstale":"2024-05-20T22:10:25.636410+0000","last_undegraded":"2024-05-20T22:10:25.636410+0000","last_fullsized":"2024-05-20T22:10:25.636410+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:15:01.610969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618877+0000","last_change":"2024-05-20T22:10:24.628495+0000","last_active":"2024-05-20T22:10:25.618877+0000","last_peered":"2024-05-20T22:10:25.618877+0000","last_clean":"2024-05-20T22:10:25.618877+0000","last_became_active":"2024-05-20T22:10:24.628180+0000","last_became_peered":"2024-05-20T22:10:24.628180+0000","last_unstale":"2024-05-20T22:10:25.618877+0000","last_undegraded":"2024-05-20T22:10:25.618877+0000","last_fullsized":"2024-05-20T22:10:25.618877+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:36.792344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619092+0000","last_change":"2024-05-20T22:10:23.619235+0000","last_active":"2024-05-20T22:10:25.619092+0000","last_peered":"2024-05-20T22:10:25.619092+0000","last_clean":"2024-05-20T22:10:25.619092+0000","last_became_active":"2024-05-20T22:10:23.618913+0000","last_became_peered":"2024-05-20T22:10:23.618913+0000","last_unstale":"2024-05-20T22:10:25.619092+0000","last_undegraded":"2024-05-20T22:10:25.619092+0000","last_fullsized":"2024-05-20T22:10:25.619092+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:53.964769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620871+0000","last_change":"2024-05-20T22:10:24.654129+0000","last_active":"2024-05-20T22:10:25.620871+0000","last_peered":"2024-05-20T22:10:25.620871+0000","last_clean":"2024-05-20T22:10:25.620871+0000","last_became_active":"2024-05-20T22:10:24.653796+0000","last_became_peered":"2024-05-20T22:10:24.653796+0000","last_unstale":"2024-05-20T22:10:25.620871+0000","last_undegraded":"2024-05-20T22:10:25.620871+0000","last_fullsized":"2024-05-20T22:10:25.620871+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:03:22.672075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.625076+0000","last_change":"2024-05-20T22:10:23.619196+0000","last_active":"2024-05-20T22:10:24.625076+0000","last_peered":"2024-05-20T22:10:24.625076+0000","last_clean":"2024-05-20T22:10:24.625076+0000","last_became_active":"2024-05-20T22:10:23.618607+0000","last_became_peered":"2024-05-20T22:10:23.618607+0000","last_unstale":"2024-05-20T22:10:24.625076+0000","last_undegraded":"2024-05-20T22:10:24.625076+0000","last_fullsized":"2024-05-20T22:10:24.625076+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:34:26.788796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618988+0000","last_change":"2024-05-20T22:10:24.644938+0000","last_active":"2024-05-20T22:10:25.618988+0000","last_peered":"2024-05-20T22:10:25.618988+0000","last_clean":"2024-05-20T22:10:25.618988+0000","last_became_active":"2024-05-20T22:10:24.644627+0000","last_became_peered":"2024-05-20T22:10:24.644627+0000","last_unstale":"2024-05-20T22:10:25.618988+0000","last_undegraded":"2024-05-20T22:10:25.618988+0000","last_fullsized":"2024-05-20T22:10:25.618988+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:09:16.410294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636106+0000","last_change":"2024-05-20T22:10:23.621369+0000","last_active":"2024-05-20T22:10:25.636106+0000","last_peered":"2024-05-20T22:10:25.636106+0000","last_clean":"2024-05-20T22:10:25.636106+0000","last_became_active":"2024-05-20T22:10:23.621058+0000","last_became_peered":"2024-05-20T22:10:23.621058+0000","last_unstale":"2024-05-20T22:10:25.636106+0000","last_undegraded":"2024-05-20T22:10:25.636106+0000","last_fullsized":"2024-05-20T22:10:25.636106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:03.888428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617998+0000","last_change":"2024-05-20T22:10:24.643807+0000","last_active":"2024-05-20T22:10:25.617998+0000","last_peered":"2024-05-20T22:10:25.617998+0000","last_clean":"2024-05-20T22:10:25.617998+0000","last_became_active":"2024-05-20T22:10:24.642813+0000","last_became_peered":"2024-05-20T22:10:24.642813+0000","last_unstale":"2024-05-20T22:10:25.617998+0000","last_undegraded":"2024-05-20T22:10:25.617998+0000","last_fullsized":"2024-05-20T22:10:25.617998+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:09:05.390222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619376+0000","last_change":"2024-05-20T22:10:23.618509+0000","last_active":"2024-05-20T22:10:25.619376+0000","last_peered":"2024-05-20T22:10:25.619376+0000","last_clean":"2024-05-20T22:10:25.619376+0000","last_became_active":"2024-05-20T22:10:23.618175+0000","last_became_peered":"2024-05-20T22:10:23.618175+0000","last_unstale":"2024-05-20T22:10:25.619376+0000","last_undegraded":"2024-05-20T22:10:25.619376+0000","last_fullsized":"2024-05-20T22:10:25.619376+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:55:00.552671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.638792+0000","last_change":"2024-05-20T22:10:24.636478+0000","last_active":"2024-05-20T22:10:24.638792+0000","last_peered":"2024-05-20T22:10:24.638792+0000","last_clean":"2024-05-20T22:10:24.638792+0000","last_became_active":"2024-05-20T22:10:24.636144+0000","last_became_peered":"2024-05-20T22:10:24.636144+0000","last_unstale":"2024-05-20T22:10:24.638792+0000","last_undegraded":"2024-05-20T22:10:24.638792+0000","last_fullsized":"2024-05-20T22:10:24.638792+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:37.478610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617991+0000","last_change":"2024-05-20T22:10:23.612855+0000","last_active":"2024-05-20T22:10:25.617991+0000","last_peered":"2024-05-20T22:10:25.617991+0000","last_clean":"2024-05-20T22:10:25.617991+0000","last_became_active":"2024-05-20T22:10:23.612550+0000","last_became_peered":"2024-05-20T22:10:23.612550+0000","last_unstale":"2024-05-20T22:10:25.617991+0000","last_undegraded":"2024-05-20T22:10:25.617991+0000","last_fullsized":"2024-05-20T22:10:25.617991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:50.824414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636499+0000","last_change":"2024-05-20T22:10:24.638769+0000","last_active":"2024-05-20T22:10:25.636499+0000","last_peered":"2024-05-20T22:10:25.636499+0000","last_clean":"2024-05-20T22:10:25.636499+0000","last_became_active":"2024-05-20T22:10:24.638451+0000","last_became_peered":"2024-05-20T22:10:24.638451+0000","last_unstale":"2024-05-20T22:10:25.636499+0000","last_undegraded":"2024-05-20T22:10:25.636499+0000","last_fullsized":"2024-05-20T22:10:25.636499+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:13:30.588413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619121+0000","last_change":"2024-05-20T22:10:23.621269+0000","last_active":"2024-05-20T22:10:25.619121+0000","last_peered":"2024-05-20T22:10:25.619121+0000","last_clean":"2024-05-20T22:10:25.619121+0000","last_became_active":"2024-05-20T22:10:23.620952+0000","last_became_peered":"2024-05-20T22:10:23.620952+0000","last_unstale":"2024-05-20T22:10:25.619121+0000","last_undegraded":"2024-05-20T22:10:25.619121+0000","last_fullsized":"2024-05-20T22:10:25.619121+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:21.960465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619042+0000","last_change":"2024-05-20T22:10:24.647371+0000","last_active":"2024-05-20T22:10:25.619042+0000","last_peered":"2024-05-20T22:10:25.619042+0000","last_clean":"2024-05-20T22:10:25.619042+0000","last_became_active":"2024-05-20T22:10:24.647139+0000","last_became_peered":"2024-05-20T22:10:24.647139+0000","last_unstale":"2024-05-20T22:10:25.619042+0000","last_undegraded":"2024-05-20T22:10:25.619042+0000","last_fullsized":"2024-05-20T22:10:25.619042+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:37.099887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":20,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620906+0000","last_change":"2024-05-20T22:10:23.625931+0000","last_active":"2024-05-20T22:10:25.620906+0000","last_peered":"2024-05-20T22:10:25.620906+0000","last_clean":"2024-05-20T22:10:25.620906+0000","last_became_active":"2024-05-20T22:10:23.625593+0000","last_became_peered":"2024-05-20T22:10:23.625593+0000","last_unstale":"2024-05-20T22:10:25.620906+0000","last_undegraded":"2024-05-20T22:10:25.620906+0000","last_fullsized":"2024-05-20T22:10:25.620906+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:35.785338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.630199+0000","last_change":"2024-05-20T22:10:24.625101+0000","last_active":"2024-05-20T22:10:24.630199+0000","last_peered":"2024-05-20T22:10:24.630199+0000","last_clean":"2024-05-20T22:10:24.630199+0000","last_became_active":"2024-05-20T22:10:24.623588+0000","last_became_peered":"2024-05-20T22:10:24.623588+0000","last_unstale":"2024-05-20T22:10:24.630199+0000","last_undegraded":"2024-05-20T22:10:24.630199+0000","last_fullsized":"2024-05-20T22:10:24.630199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:47.850915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636541+0000","last_change":"2024-05-20T22:10:23.619729+0000","last_active":"2024-05-20T22:10:25.636541+0000","last_peered":"2024-05-20T22:10:25.636541+0000","last_clean":"2024-05-20T22:10:25.636541+0000","last_became_active":"2024-05-20T22:10:23.619414+0000","last_became_peered":"2024-05-20T22:10:23.619414+0000","last_unstale":"2024-05-20T22:10:25.636541+0000","last_undegraded":"2024-05-20T22:10:25.636541+0000","last_fullsized":"2024-05-20T22:10:25.636541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:37.325204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636621+0000","last_change":"2024-05-20T22:10:24.648257+0000","last_active":"2024-05-20T22:10:25.636621+0000","last_peered":"2024-05-20T22:10:25.636621+0000","last_clean":"2024-05-20T22:10:25.636621+0000","last_became_active":"2024-05-20T22:10:24.647910+0000","last_became_peered":"2024-05-20T22:10:24.647910+0000","last_unstale":"2024-05-20T22:10:25.636621+0000","last_undegraded":"2024-05-20T22:10:25.636621+0000","last_fullsized":"2024-05-20T22:10:25.636621+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:21:10.915657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.625128+0000","last_change":"2024-05-20T22:10:23.627363+0000","last_active":"2024-05-20T22:10:24.625128+0000","last_peered":"2024-05-20T22:10:24.625128+0000","last_clean":"2024-05-20T22:10:24.625128+0000","last_became_active":"2024-05-20T22:10:23.627071+0000","last_became_peered":"2024-05-20T22:10:23.627071+0000","last_unstale":"2024-05-20T22:10:24.625128+0000","last_undegraded":"2024-05-20T22:10:24.625128+0000","last_fullsized":"2024-05-20T22:10:24.625128+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:08.531292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620707+0000","last_change":"2024-05-20T22:10:24.639544+0000","last_active":"2024-05-20T22:10:25.620707+0000","last_peered":"2024-05-20T22:10:25.620707+0000","last_clean":"2024-05-20T22:10:25.620707+0000","last_became_active":"2024-05-20T22:10:24.639197+0000","last_became_peered":"2024-05-20T22:10:24.639197+0000","last_unstale":"2024-05-20T22:10:25.620707+0000","last_undegraded":"2024-05-20T22:10:25.620707+0000","last_fullsized":"2024-05-20T22:10:25.620707+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:36:53.606875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618102+0000","last_change":"2024-05-20T22:10:23.619196+0000","last_active":"2024-05-20T22:10:25.618102+0000","last_peered":"2024-05-20T22:10:25.618102+0000","last_clean":"2024-05-20T22:10:25.618102+0000","last_became_active":"2024-05-20T22:10:23.618882+0000","last_became_peered":"2024-05-20T22:10:23.618882+0000","last_unstale":"2024-05-20T22:10:25.618102+0000","last_undegraded":"2024-05-20T22:10:25.618102+0000","last_fullsized":"2024-05-20T22:10:25.618102+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:36.092881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636957+0000","last_change":"2024-05-20T22:10:24.636153+0000","last_active":"2024-05-20T22:10:25.636957+0000","last_peered":"2024-05-20T22:10:25.636957+0000","last_clean":"2024-05-20T22:10:25.636957+0000","last_became_active":"2024-05-20T22:10:24.635808+0000","last_became_peered":"2024-05-20T22:10:24.635808+0000","last_unstale":"2024-05-20T22:10:25.636957+0000","last_undegraded":"2024-05-20T22:10:25.636957+0000","last_fullsized":"2024-05-20T22:10:25.636957+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:29:26.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":22,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620522+0000","last_change":"2024-05-20T22:10:23.628057+0000","last_active":"2024-05-20T22:10:25.620522+0000","last_peered":"2024-05-20T22:10:25.620522+0000","last_clean":"2024-05-20T22:10:25.620522+0000","last_became_active":"2024-05-20T22:10:23.626835+0000","last_became_peered":"2024-05-20T22:10:23.626835+0000","last_unstale":"2024-05-20T22:10:25.620522+0000","last_undegraded":"2024-05-20T22:10:25.620522+0000","last_fullsized":"2024-05-20T22:10:25.620522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:09:10.482777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620613+0000","last_change":"2024-05-20T22:10:24.649307+0000","last_active":"2024-05-20T22:10:25.620613+0000","last_peered":"2024-05-20T22:10:25.620613+0000","last_clean":"2024-05-20T22:10:25.620613+0000","last_became_active":"2024-05-20T22:10:24.648978+0000","last_became_peered":"2024-05-20T22:10:24.648978+0000","last_unstale":"2024-05-20T22:10:25.620613+0000","last_undegraded":"2024-05-20T22:10:25.620613+0000","last_fullsized":"2024-05-20T22:10:25.620613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:36:31.103907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.637025+0000","last_change":"2024-05-20T22:10:23.627338+0000","last_active":"2024-05-20T22:10:25.637025+0000","last_peered":"2024-05-20T22:10:25.637025+0000","last_clean":"2024-05-20T22:10:25.637025+0000","last_became_active":"2024-05-20T22:10:23.626207+0000","last_became_peered":"2024-05-20T22:10:23.626207+0000","last_unstale":"2024-05-20T22:10:25.637025+0000","last_undegraded":"2024-05-20T22:10:25.637025+0000","last_fullsized":"2024-05-20T22:10:25.637025+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:36.092881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649652+0000","last_change":"2024-05-20T22:10:24.642583+0000","last_active":"2024-05-20T22:10:24.649652+0000","last_peered":"2024-05-20T22:10:24.649652+0000","last_clean":"2024-05-20T22:10:24.649652+0000","last_became_active":"2024-05-20T22:10:24.642253+0000","last_became_peered":"2024-05-20T22:10:24.642253+0000","last_unstale":"2024-05-20T22:10:24.649652+0000","last_undegraded":"2024-05-20T22:10:24.649652+0000","last_fullsized":"2024-05-20T22:10:24.649652+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:37.099887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.653139+0000","last_change":"2024-05-20T22:10:23.621005+0000","last_active":"2024-05-20T22:10:24.653139+0000","last_peered":"2024-05-20T22:10:24.653139+0000","last_clean":"2024-05-20T22:10:24.653139+0000","last_became_active":"2024-05-20T22:10:23.620567+0000","last_became_peered":"2024-05-20T22:10:23.620567+0000","last_unstale":"2024-05-20T22:10:24.653139+0000","last_undegraded":"2024-05-20T22:10:24.653139+0000","last_fullsized":"2024-05-20T22:10:24.653139+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:56:15.656528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618728+0000","last_change":"2024-05-20T22:10:24.654083+0000","last_active":"2024-05-20T22:10:25.618728+0000","last_peered":"2024-05-20T22:10:25.618728+0000","last_clean":"2024-05-20T22:10:25.618728+0000","last_became_active":"2024-05-20T22:10:24.653774+0000","last_became_peered":"2024-05-20T22:10:24.653774+0000","last_unstale":"2024-05-20T22:10:25.618728+0000","last_undegraded":"2024-05-20T22:10:25.618728+0000","last_fullsized":"2024-05-20T22:10:25.618728+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:38.332210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.650856+0000","last_change":"2024-05-20T22:10:23.622913+0000","last_active":"2024-05-20T22:10:24.650856+0000","last_peered":"2024-05-20T22:10:24.650856+0000","last_clean":"2024-05-20T22:10:24.650856+0000","last_became_active":"2024-05-20T22:10:23.622559+0000","last_became_peered":"2024-05-20T22:10:23.622559+0000","last_unstale":"2024-05-20T22:10:24.650856+0000","last_undegraded":"2024-05-20T22:10:24.650856+0000","last_fullsized":"2024-05-20T22:10:24.650856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:43.365714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649980+0000","last_change":"2024-05-20T22:10:24.640570+0000","last_active":"2024-05-20T22:10:24.649980+0000","last_peered":"2024-05-20T22:10:24.649980+0000","last_clean":"2024-05-20T22:10:24.649980+0000","last_became_active":"2024-05-20T22:10:24.639972+0000","last_became_peered":"2024-05-20T22:10:24.639972+0000","last_unstale":"2024-05-20T22:10:24.649980+0000","last_undegraded":"2024-05-20T22:10:24.649980+0000","last_fullsized":"2024-05-20T22:10:24.649980+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:01.712783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.624481+0000","last_change":"2024-05-20T22:10:23.623034+0000","last_active":"2024-05-20T22:10:24.624481+0000","last_peered":"2024-05-20T22:10:24.624481+0000","last_clean":"2024-05-20T22:10:24.624481+0000","last_became_active":"2024-05-20T22:10:23.622605+0000","last_became_peered":"2024-05-20T22:10:23.622605+0000","last_unstale":"2024-05-20T22:10:24.624481+0000","last_undegraded":"2024-05-20T22:10:24.624481+0000","last_fullsized":"2024-05-20T22:10:24.624481+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:50.824414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.638753+0000","last_change":"2024-05-20T22:10:24.637293+0000","last_active":"2024-05-20T22:10:24.638753+0000","last_peered":"2024-05-20T22:10:24.638753+0000","last_clean":"2024-05-20T22:10:24.638753+0000","last_became_active":"2024-05-20T22:10:24.636981+0000","last_became_peered":"2024-05-20T22:10:24.636981+0000","last_unstale":"2024-05-20T22:10:24.638753+0000","last_undegraded":"2024-05-20T22:10:24.638753+0000","last_fullsized":"2024-05-20T22:10:24.638753+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:06.749616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649906+0000","last_change":"2024-05-20T22:10:23.620498+0000","last_active":"2024-05-20T22:10:24.649906+0000","last_peered":"2024-05-20T22:10:24.649906+0000","last_clean":"2024-05-20T22:10:24.649906+0000","last_became_active":"2024-05-20T22:10:23.620146+0000","last_became_peered":"2024-05-20T22:10:23.620146+0000","last_unstale":"2024-05-20T22:10:24.649906+0000","last_undegraded":"2024-05-20T22:10:24.649906+0000","last_fullsized":"2024-05-20T22:10:24.649906+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:50.824414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.638921+0000","last_change":"2024-05-20T22:10:24.636304+0000","last_active":"2024-05-20T22:10:24.638921+0000","last_peered":"2024-05-20T22:10:24.638921+0000","last_clean":"2024-05-20T22:10:24.638921+0000","last_became_active":"2024-05-20T22:10:24.635983+0000","last_became_peered":"2024-05-20T22:10:24.635983+0000","last_unstale":"2024-05-20T22:10:24.638921+0000","last_undegraded":"2024-05-20T22:10:24.638921+0000","last_fullsized":"2024-05-20T22:10:24.638921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:01.712783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620257+0000","last_change":"2024-05-20T22:10:23.630021+0000","last_active":"2024-05-20T22:10:25.620257+0000","last_peered":"2024-05-20T22:10:25.620257+0000","last_clean":"2024-05-20T22:10:25.620257+0000","last_became_active":"2024-05-20T22:10:23.629231+0000","last_became_peered":"2024-05-20T22:10:23.629231+0000","last_unstale":"2024-05-20T22:10:25.620257+0000","last_undegraded":"2024-05-20T22:10:25.620257+0000","last_fullsized":"2024-05-20T22:10:25.620257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:42:52.047921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636643+0000","last_change":"2024-05-20T22:10:24.633829+0000","last_active":"2024-05-20T22:10:25.636643+0000","last_peered":"2024-05-20T22:10:25.636643+0000","last_clean":"2024-05-20T22:10:25.636643+0000","last_became_active":"2024-05-20T22:10:24.633516+0000","last_became_peered":"2024-05-20T22:10:24.633516+0000","last_unstale":"2024-05-20T22:10:25.636643+0000","last_undegraded":"2024-05-20T22:10:25.636643+0000","last_fullsized":"2024-05-20T22:10:25.636643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:24.021946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636270+0000","last_change":"2024-05-20T22:10:23.622926+0000","last_active":"2024-05-20T22:10:25.636270+0000","last_peered":"2024-05-20T22:10:25.636270+0000","last_clean":"2024-05-20T22:10:25.636270+0000","last_became_active":"2024-05-20T22:10:23.622595+0000","last_became_peered":"2024-05-20T22:10:23.622595+0000","last_unstale":"2024-05-20T22:10:25.636270+0000","last_undegraded":"2024-05-20T22:10:25.636270+0000","last_fullsized":"2024-05-20T22:10:25.636270+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:00.705778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649098+0000","last_change":"2024-05-20T22:10:24.642105+0000","last_active":"2024-05-20T22:10:24.649098+0000","last_peered":"2024-05-20T22:10:24.649098+0000","last_clean":"2024-05-20T22:10:24.649098+0000","last_became_active":"2024-05-20T22:10:24.641788+0000","last_became_peered":"2024-05-20T22:10:24.641788+0000","last_unstale":"2024-05-20T22:10:24.649098+0000","last_undegraded":"2024-05-20T22:10:24.649098+0000","last_fullsized":"2024-05-20T22:10:24.649098+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:38.332210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.626525+0000","last_change":"2024-05-20T22:10:23.624437+0000","last_active":"2024-05-20T22:10:24.626525+0000","last_peered":"2024-05-20T22:10:24.626525+0000","last_clean":"2024-05-20T22:10:24.626525+0000","last_became_active":"2024-05-20T22:10:23.622824+0000","last_became_peered":"2024-05-20T22:10:23.622824+0000","last_unstale":"2024-05-20T22:10:24.626525+0000","last_undegraded":"2024-05-20T22:10:24.626525+0000","last_fullsized":"2024-05-20T22:10:24.626525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:26.244777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.652880+0000","last_change":"2024-05-20T22:10:24.648539+0000","last_active":"2024-05-20T22:10:24.652880+0000","last_peered":"2024-05-20T22:10:24.652880+0000","last_clean":"2024-05-20T22:10:24.652880+0000","last_became_active":"2024-05-20T22:10:24.648230+0000","last_became_peered":"2024-05-20T22:10:24.648230+0000","last_unstale":"2024-05-20T22:10:24.652880+0000","last_undegraded":"2024-05-20T22:10:24.652880+0000","last_fullsized":"2024-05-20T22:10:24.652880+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:50:00.634939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.655864+0000","last_change":"2024-05-20T22:10:23.615216+0000","last_active":"2024-05-20T22:10:24.655864+0000","last_peered":"2024-05-20T22:10:24.655864+0000","last_clean":"2024-05-20T22:10:24.655864+0000","last_became_active":"2024-05-20T22:10:23.614877+0000","last_became_peered":"2024-05-20T22:10:23.614877+0000","last_unstale":"2024-05-20T22:10:24.655864+0000","last_undegraded":"2024-05-20T22:10:24.655864+0000","last_fullsized":"2024-05-20T22:10:24.655864+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:53.964769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649561+0000","last_change":"2024-05-20T22:10:24.638337+0000","last_active":"2024-05-20T22:10:24.649561+0000","last_peered":"2024-05-20T22:10:24.649561+0000","last_clean":"2024-05-20T22:10:24.649561+0000","last_became_active":"2024-05-20T22:10:24.638011+0000","last_became_peered":"2024-05-20T22:10:24.638011+0000","last_unstale":"2024-05-20T22:10:24.649561+0000","last_undegraded":"2024-05-20T22:10:24.649561+0000","last_fullsized":"2024-05-20T22:10:24.649561+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:09:05.390222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619239+0000","last_change":"2024-05-20T22:10:23.625806+0000","last_active":"2024-05-20T22:10:25.619239+0000","last_peered":"2024-05-20T22:10:25.619239+0000","last_clean":"2024-05-20T22:10:25.619239+0000","last_became_active":"2024-05-20T22:10:23.625513+0000","last_became_peered":"2024-05-20T22:10:23.625513+0000","last_unstale":"2024-05-20T22:10:25.619239+0000","last_undegraded":"2024-05-20T22:10:25.619239+0000","last_fullsized":"2024-05-20T22:10:25.619239+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:34:52.256297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620422+0000","last_change":"2024-05-20T22:10:24.647060+0000","last_active":"2024-05-20T22:10:25.620422+0000","last_peered":"2024-05-20T22:10:25.620422+0000","last_clean":"2024-05-20T22:10:25.620422+0000","last_became_active":"2024-05-20T22:10:24.644261+0000","last_became_peered":"2024-05-20T22:10:24.644261+0000","last_unstale":"2024-05-20T22:10:25.620422+0000","last_undegraded":"2024-05-20T22:10:25.620422+0000","last_fullsized":"2024-05-20T22:10:25.620422+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:29:26.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636796+0000","last_change":"2024-05-20T22:10:23.612870+0000","last_active":"2024-05-20T22:10:25.636796+0000","last_peered":"2024-05-20T22:10:25.636796+0000","last_clean":"2024-05-20T22:10:25.636796+0000","last_became_active":"2024-05-20T22:10:23.612534+0000","last_became_peered":"2024-05-20T22:10:23.612534+0000","last_unstale":"2024-05-20T22:10:25.636796+0000","last_undegraded":"2024-05-20T22:10:25.636796+0000","last_fullsized":"2024-05-20T22:10:25.636796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:09:04.383216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636690+0000","last_change":"2024-05-20T22:10:24.649287+0000","last_active":"2024-05-20T22:10:25.636690+0000","last_peered":"2024-05-20T22:10:25.636690+0000","last_clean":"2024-05-20T22:10:25.636690+0000","last_became_active":"2024-05-20T22:10:24.648939+0000","last_became_peered":"2024-05-20T22:10:24.648939+0000","last_unstale":"2024-05-20T22:10:25.636690+0000","last_undegraded":"2024-05-20T22:10:25.636690+0000","last_fullsized":"2024-05-20T22:10:25.636690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:11.888028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620553+0000","last_change":"2024-05-20T22:10:23.629627+0000","last_active":"2024-05-20T22:10:25.620553+0000","last_peered":"2024-05-20T22:10:25.620553+0000","last_clean":"2024-05-20T22:10:25.620553+0000","last_became_active":"2024-05-20T22:10:23.629252+0000","last_became_peered":"2024-05-20T22:10:23.629252+0000","last_unstale":"2024-05-20T22:10:25.620553+0000","last_undegraded":"2024-05-20T22:10:25.620553+0000","last_fullsized":"2024-05-20T22:10:25.620553+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:05.957349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.626141+0000","last_change":"2024-05-20T22:10:24.624042+0000","last_active":"2024-05-20T22:10:24.626141+0000","last_peered":"2024-05-20T22:10:24.626141+0000","last_clean":"2024-05-20T22:10:24.626141+0000","last_became_active":"2024-05-20T22:10:24.623716+0000","last_became_peered":"2024-05-20T22:10:24.623716+0000","last_unstale":"2024-05-20T22:10:24.626141+0000","last_undegraded":"2024-05-20T22:10:24.626141+0000","last_fullsized":"2024-05-20T22:10:24.626141+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:08:03.319713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620381+0000","last_change":"2024-05-20T22:10:23.628520+0000","last_active":"2024-05-20T22:10:25.620381+0000","last_peered":"2024-05-20T22:10:25.620381+0000","last_clean":"2024-05-20T22:10:25.620381+0000","last_became_active":"2024-05-20T22:10:23.628198+0000","last_became_peered":"2024-05-20T22:10:23.628198+0000","last_unstale":"2024-05-20T22:10:25.620381+0000","last_undegraded":"2024-05-20T22:10:25.620381+0000","last_fullsized":"2024-05-20T22:10:25.620381+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:40:23.195917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636457+0000","last_change":"2024-05-20T22:10:24.646294+0000","last_active":"2024-05-20T22:10:25.636457+0000","last_peered":"2024-05-20T22:10:25.636457+0000","last_clean":"2024-05-20T22:10:25.636457+0000","last_became_active":"2024-05-20T22:10:24.645374+0000","last_became_peered":"2024-05-20T22:10:24.645374+0000","last_unstale":"2024-05-20T22:10:25.636457+0000","last_undegraded":"2024-05-20T22:10:25.636457+0000","last_fullsized":"2024-05-20T22:10:25.636457+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:50.138737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617809+0000","last_change":"2024-05-20T22:10:23.625961+0000","last_active":"2024-05-20T22:10:25.617809+0000","last_peered":"2024-05-20T22:10:25.617809+0000","last_clean":"2024-05-20T22:10:25.617809+0000","last_became_active":"2024-05-20T22:10:23.625606+0000","last_became_peered":"2024-05-20T22:10:23.625606+0000","last_unstale":"2024-05-20T22:10:25.617809+0000","last_undegraded":"2024-05-20T22:10:25.617809+0000","last_fullsized":"2024-05-20T22:10:25.617809+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:06:34.084033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620797+0000","last_change":"2024-05-20T22:10:24.648685+0000","last_active":"2024-05-20T22:10:25.620797+0000","last_peered":"2024-05-20T22:10:25.620797+0000","last_clean":"2024-05-20T22:10:25.620797+0000","last_became_active":"2024-05-20T22:10:24.647432+0000","last_became_peered":"2024-05-20T22:10:24.647432+0000","last_unstale":"2024-05-20T22:10:25.620797+0000","last_undegraded":"2024-05-20T22:10:25.620797+0000","last_fullsized":"2024-05-20T22:10:25.620797+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:35.226715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.617811+0000","last_change":"2024-05-20T22:10:23.622905+0000","last_active":"2024-05-20T22:10:24.617811+0000","last_peered":"2024-05-20T22:10:24.617811+0000","last_clean":"2024-05-20T22:10:24.617811+0000","last_became_active":"2024-05-20T22:10:23.622595+0000","last_became_peered":"2024-05-20T22:10:23.622595+0000","last_unstale":"2024-05-20T22:10:24.617811+0000","last_undegraded":"2024-05-20T22:10:24.617811+0000","last_fullsized":"2024-05-20T22:10:24.617811+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:22:09.244992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.635232+0000","last_change":"2024-05-20T22:10:24.634218+0000","last_active":"2024-05-20T22:10:24.635232+0000","last_peered":"2024-05-20T22:10:24.635232+0000","last_clean":"2024-05-20T22:10:24.635232+0000","last_became_active":"2024-05-20T22:10:24.633896+0000","last_became_peered":"2024-05-20T22:10:24.633896+0000","last_unstale":"2024-05-20T22:10:24.635232+0000","last_undegraded":"2024-05-20T22:10:24.635232+0000","last_fullsized":"2024-05-20T22:10:24.635232+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:09:05.390222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"55'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.638375+0000","last_change":"2024-05-20T22:10:23.620237+0000","last_active":"2024-05-20T22:10:25.638375+0000","last_peered":"2024-05-20T22:10:25.638375+0000","last_clean":"2024-05-20T22:10:25.638375+0000","last_became_active":"2024-05-20T22:10:23.619876+0000","last_became_peered":"2024-05-20T22:10:23.619876+0000","last_unstale":"2024-05-20T22:10:25.638375+0000","last_undegraded":"2024-05-20T22:10:25.638375+0000","last_fullsized":"2024-05-20T22:10:25.638375+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:27:54.123173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":241,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617732+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:10:25.617732+0000","last_peered":"2024-05-20T22:10:25.617732+0000","last_clean":"2024-05-20T22:10:25.617732+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:10:25.617732+0000","last_undegraded":"2024-05-20T22:10:25.617732+0000","last_fullsized":"2024-05-20T22:10:25.617732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":886766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":290234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":289582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":290164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":659456,"data_stored":291216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":655360,"data_stored":291194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":651264,"data_stored":289742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1245184,"data_stored":880006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:10:28.710 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:10:28.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.860+0000 7f563a491640 1 Processor -- start 2024-05-20T22:10:28.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.860+0000 7f563a491640 1 -- start start 2024-05-20T22:10:28.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f563a491640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56340b3b30 0x7f56340b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f563a491640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56340d3dd0 con 0x7f563415f100 2024-05-20T22:10:28.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f563a491640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56340d40b0 con 0x7f563406c790 2024-05-20T22:10:28.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f563a491640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56340d4390 con 0x7f56340b3b30 2024-05-20T22:10:28.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f56337fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f563415f100 0x7f563406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59678/0 (socket says 172.21.15.55:59678) 2024-05-20T22:10:28.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f56337fe640 1 -- 172.21.15.55:0/2149734644 learned_addr learned my addr 172.21.15.55:0/2149734644 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:28.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5633fff640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56340b3b30 0x7f56340b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1384080107 0 0) 0x7f56340d3dd0 con 0x7f563415f100 2024-05-20T22:10:28.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f561c003520 con 0x7f563415f100 2024-05-20T22:10:28.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1036952581 0 0) 0x7f56340d40b0 con 0x7f563406c790 2024-05-20T22:10:28.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56340d3dd0 con 0x7f563406c790 2024-05-20T22:10:28.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1194040649 0 0) 0x7f561c003520 con 0x7f563415f100 2024-05-20T22:10:28.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56340b3b30 msgr2=0x7f56340b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:28.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56340b3b30 0x7f56340b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 >> v1:172.21.15.138:6789/0 conn(0x7f563406c790 legacy=0x7f56340cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563406b5a0 con 0x7f563415f100 2024-05-20T22:10:28.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f5628001670 con 0x7f563415f100 2024-05-20T22:10:28.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.862+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 >> v1:172.21.15.55:6789/0 conn(0x7f563415f100 legacy=0x7f563406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f5632ffd640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56340b3b30 0x7f561c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f5632ffd640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f561c0040d0 0x7f561c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f5632ffd640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f561c004d40 0x7f561c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56340d4390 con 0x7f561c0040d0 2024-05-20T22:10:28.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f561c00bb20 con 0x7f561c004d40 2024-05-20T22:10:28.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f561c00bd20 con 0x7f56340b3b30 2024-05-20T22:10:28.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f56337fe640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f561c0040d0 0x7f561c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f5633fff640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56340b3b30 0x7f561c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f5638a2e640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f561c004d40 0x7f561c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f56337fe640 1 -- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56340b3b30 msgr2=0x7f561c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:28.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f56337fe640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56340b3b30 0x7f561c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f56337fe640 1 -- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f561c004d40 msgr2=0x7f561c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:28.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f56337fe640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f561c004d40 0x7f561c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f56337fe640 1 -- 172.21.15.55:0/2149734644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c003520 con 0x7f561c0040d0 2024-05-20T22:10:28.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.863+0000 7f5633fff640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56340b3b30 0x7f561c003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:28.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.864+0000 7f56337fe640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f561c0040d0 0x7f561c004800 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f562800c190 tx=0x7f561c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212587ffc971b3a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:28.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.864+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628007cc0 con 0x7f561c0040d0 2024-05-20T22:10:28.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.864+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5628005af0 con 0x7f561c0040d0 2024-05-20T22:10:28.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.864+0000 7f5632ffd640 1 -- 172.21.15.55:0/2149734644 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628007070 con 0x7f561c0040d0 2024-05-20T22:10:28.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.865+0000 7f563a491640 1 -- 172.21.15.55:0/2149734644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f561c0040d0 msgr2=0x7f561c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.865+0000 7f563a491640 1 --2- 172.21.15.55:0/2149734644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f561c0040d0 0x7f561c004800 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f562800c190 tx=0x7f561c005830 comp rx=0 tx=0).stop 2024-05-20T22:10:28.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.865+0000 7f5638a2e640 1 -- 172.21.15.55:0/2149734644 reap_dead start 2024-05-20T22:10:28.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.865+0000 7f563a491640 1 -- 172.21.15.55:0/2149734644 shutdown_connections 2024-05-20T22:10:28.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.865+0000 7f563a491640 1 -- 172.21.15.55:0/2149734644 >> 172.21.15.55:0/2149734644 conn(0x7f56340d51d0 msgr2=0x7f56340b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:28.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.865+0000 7f563a491640 1 -- 172.21.15.55:0/2149734644 shutdown_connections 2024-05-20T22:10:28.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.865+0000 7f563a491640 1 -- 172.21.15.55:0/2149734644 wait complete. 2024-05-20T22:10:28.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.866+0000 7f563a491640 1 Processor -- start 2024-05-20T22:10:28.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.867+0000 7f563a491640 1 -- start start 2024-05-20T22:10:28.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f563a491640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f561c0040d0 0x7f563415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f563a491640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56340c9970 0x7f5634158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f563a491640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5634158e70 0x7f56341f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f563a491640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f563406cba0 con 0x7f5634158e70 2024-05-20T22:10:28.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f563a491640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f563406bb60 con 0x7f56340c9970 2024-05-20T22:10:28.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f563a491640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f563406cd20 con 0x7f561c0040d0 2024-05-20T22:10:28.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f5638a2e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5634158e70 0x7f56341f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f5638a2e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5634158e70 0x7f56341f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57296/0 (socket says 172.21.15.55:57296) 2024-05-20T22:10:28.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f5638a2e640 1 -- 172.21.15.55:0/3725820876 learned_addr learned my addr 172.21.15.55:0/3725820876 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:28.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f56337fe640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56340c9970 0x7f5634158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f5633fff640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f561c0040d0 0x7f563415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f5638a2e640 1 -- 172.21.15.55:0/3725820876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f561c0040d0 msgr2=0x7f563415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f5638a2e640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f561c0040d0 0x7f563415af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f5638a2e640 1 -- 172.21.15.55:0/3725820876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56340c9970 msgr2=0x7f5634158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f5638a2e640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56340c9970 0x7f5634158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f5638a2e640 1 -- 172.21.15.55:0/3725820876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340c88d0 con 0x7f5634158e70 2024-05-20T22:10:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.868+0000 7f56337fe640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56340c9970 0x7f5634158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.869+0000 7f5638a2e640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5634158e70 0x7f56341f5700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f56340d1320 tx=0x7f562c00fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.869+0000 7f5630ff9640 1 -- 172.21.15.55:0/3725820876 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c019070 con 0x7f5634158e70 2024-05-20T22:10:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.869+0000 7f5630ff9640 1 -- 172.21.15.55:0/3725820876 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c006e20 con 0x7f5634158e70 2024-05-20T22:10:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.869+0000 7f5630ff9640 1 -- 172.21.15.55:0/3725820876 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c003b70 con 0x7f5634158e70 2024-05-20T22:10:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.869+0000 7f563a491640 1 -- 172.21.15.55:0/3725820876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56340c8bb0 con 0x7f5634158e70 2024-05-20T22:10:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.869+0000 7f563a491640 1 -- 172.21.15.55:0/3725820876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56340c9170 con 0x7f5634158e70 2024-05-20T22:10:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.870+0000 7f563a491640 1 -- 172.21.15.55:0/3725820876 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634069090 con 0x7f5634158e70 2024-05-20T22:10:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.871+0000 7f5630ff9640 1 -- 172.21.15.55:0/3725820876 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f562c003d10 con 0x7f5634158e70 2024-05-20T22:10:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.871+0000 7f5630ff9640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f562006be30 0x7f562006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.871+0000 7f5630ff9640 1 -- 172.21.15.55:0/3725820876 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f562c08a4a0 con 0x7f5634158e70 2024-05-20T22:10:28.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.871+0000 7f5633fff640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f562006be30 0x7f562006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:28.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.872+0000 7f5633fff640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f562006be30 0x7f562006e340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5624009160 tx=0x7f5624008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:28.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:28.876+0000 7f5630ff9640 1 -- 172.21.15.55:0/3725820876 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f562c058c30 con 0x7f5634158e70 2024-05-20T22:10:29.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.055+0000 7f563a491640 1 -- 172.21.15.55:0/3725820876 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56340d2ee0 con 0x7f562006be30 2024-05-20T22:10:29.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.062+0000 7f5630ff9640 1 -- 172.21.15.55:0/3725820876 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251923 (secure 0 0 0) 0x7f56340d2ee0 con 0x7f562006be30 2024-05-20T22:10:29.065 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:29.067 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:10:29.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 -- 172.21.15.55:0/3725820876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f562006be30 msgr2=0x7f562006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f562006be30 0x7f562006e340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5624009160 tx=0x7f5624008040 comp rx=0 tx=0).stop 2024-05-20T22:10:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 -- 172.21.15.55:0/3725820876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5634158e70 msgr2=0x7f56341f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5634158e70 0x7f56341f5700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f56340d1320 tx=0x7f562c00fd60 comp rx=0 tx=0).stop 2024-05-20T22:10:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 -- 172.21.15.55:0/3725820876 shutdown_connections 2024-05-20T22:10:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f562006be30 0x7f562006e340 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5634158e70 0x7f56341f5700 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56340c9970 0x7f5634158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 --2- 172.21.15.55:0/3725820876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f561c0040d0 0x7f563415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 -- 172.21.15.55:0/3725820876 >> 172.21.15.55:0/3725820876 conn(0x7f56340d51d0 msgr2=0x7f563415e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 -- 172.21.15.55:0/3725820876 shutdown_connections 2024-05-20T22:10:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.065+0000 7f563a491640 1 -- 172.21.15.55:0/3725820876 wait complete. 2024-05-20T22:10:29.078 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-05-20T22:10:28.363938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243240,"kb_used_data":6260,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731480,"statfs":{"total":773094113280,"available":772845035520,"internally_reserved":0,"allocated":6410240,"data_stored":3508904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.767463"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649069+0000","last_change":"2024-05-20T22:10:24.637185+0000","last_active":"2024-05-20T22:10:24.649069+0000","last_peered":"2024-05-20T22:10:24.649069+0000","last_clean":"2024-05-20T22:10:24.649069+0000","last_became_active":"2024-05-20T22:10:24.636736+0000","last_became_peered":"2024-05-20T22:10:24.636736+0000","last_unstale":"2024-05-20T22:10:24.649069+0000","last_undegraded":"2024-05-20T22:10:24.649069+0000","last_fullsized":"2024-05-20T22:10:24.649069+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:09.538298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619134+0000","last_change":"2024-05-20T22:10:24.625208+0000","last_active":"2024-05-20T22:10:25.619134+0000","last_peered":"2024-05-20T22:10:25.619134+0000","last_clean":"2024-05-20T22:10:25.619134+0000","last_became_active":"2024-05-20T22:10:24.624880+0000","last_became_peered":"2024-05-20T22:10:24.624880+0000","last_unstale":"2024-05-20T22:10:25.619134+0000","last_undegraded":"2024-05-20T22:10:25.619134+0000","last_fullsized":"2024-05-20T22:10:25.619134+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:37:18.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636411+0000","last_change":"2024-05-20T22:10:24.641686+0000","last_active":"2024-05-20T22:10:25.636411+0000","last_peered":"2024-05-20T22:10:25.636411+0000","last_clean":"2024-05-20T22:10:25.636411+0000","last_became_active":"2024-05-20T22:10:24.641225+0000","last_became_peered":"2024-05-20T22:10:24.641225+0000","last_unstale":"2024-05-20T22:10:25.636411+0000","last_undegraded":"2024-05-20T22:10:25.636411+0000","last_fullsized":"2024-05-20T22:10:25.636411+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:42:48.287564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618640+0000","last_change":"2024-05-20T22:10:24.646369+0000","last_active":"2024-05-20T22:10:25.618640+0000","last_peered":"2024-05-20T22:10:25.618640+0000","last_clean":"2024-05-20T22:10:25.618640+0000","last_became_active":"2024-05-20T22:10:24.646057+0000","last_became_peered":"2024-05-20T22:10:24.646057+0000","last_unstale":"2024-05-20T22:10:25.618640+0000","last_undegraded":"2024-05-20T22:10:25.618640+0000","last_fullsized":"2024-05-20T22:10:25.618640+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:09:11.489783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619029+0000","last_change":"2024-05-20T22:10:24.637171+0000","last_active":"2024-05-20T22:10:25.619029+0000","last_peered":"2024-05-20T22:10:25.619029+0000","last_clean":"2024-05-20T22:10:25.619029+0000","last_became_active":"2024-05-20T22:10:24.636816+0000","last_became_peered":"2024-05-20T22:10:24.636816+0000","last_unstale":"2024-05-20T22:10:25.619029+0000","last_undegraded":"2024-05-20T22:10:25.619029+0000","last_fullsized":"2024-05-20T22:10:25.619029+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:15:02.617975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649536+0000","last_change":"2024-05-20T22:10:24.637058+0000","last_active":"2024-05-20T22:10:24.649536+0000","last_peered":"2024-05-20T22:10:24.649536+0000","last_clean":"2024-05-20T22:10:24.649536+0000","last_became_active":"2024-05-20T22:10:24.636646+0000","last_became_peered":"2024-05-20T22:10:24.636646+0000","last_unstale":"2024-05-20T22:10:24.649536+0000","last_undegraded":"2024-05-20T22:10:24.649536+0000","last_fullsized":"2024-05-20T22:10:24.649536+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:34:27.795802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636139+0000","last_change":"2024-05-20T22:10:24.646018+0000","last_active":"2024-05-20T22:10:25.636139+0000","last_peered":"2024-05-20T22:10:25.636139+0000","last_clean":"2024-05-20T22:10:25.636139+0000","last_became_active":"2024-05-20T22:10:24.645680+0000","last_became_peered":"2024-05-20T22:10:24.645680+0000","last_unstale":"2024-05-20T22:10:25.636139+0000","last_undegraded":"2024-05-20T22:10:25.636139+0000","last_fullsized":"2024-05-20T22:10:25.636139+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:09:16.410294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620361+0000","last_change":"2024-05-20T22:10:24.658719+0000","last_active":"2024-05-20T22:10:25.620361+0000","last_peered":"2024-05-20T22:10:25.620361+0000","last_clean":"2024-05-20T22:10:25.620361+0000","last_became_active":"2024-05-20T22:10:24.658403+0000","last_became_peered":"2024-05-20T22:10:24.658403+0000","last_unstale":"2024-05-20T22:10:25.620361+0000","last_undegraded":"2024-05-20T22:10:25.620361+0000","last_fullsized":"2024-05-20T22:10:25.620361+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:27:02.138910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620704+0000","last_change":"2024-05-20T22:10:24.654515+0000","last_active":"2024-05-20T22:10:25.620704+0000","last_peered":"2024-05-20T22:10:25.620704+0000","last_clean":"2024-05-20T22:10:25.620704+0000","last_became_active":"2024-05-20T22:10:24.654185+0000","last_became_peered":"2024-05-20T22:10:24.654185+0000","last_unstale":"2024-05-20T22:10:25.620704+0000","last_undegraded":"2024-05-20T22:10:25.620704+0000","last_fullsized":"2024-05-20T22:10:25.620704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:05:42.948141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649602+0000","last_change":"2024-05-20T22:10:24.637396+0000","last_active":"2024-05-20T22:10:24.649602+0000","last_peered":"2024-05-20T22:10:24.649602+0000","last_clean":"2024-05-20T22:10:24.649602+0000","last_became_active":"2024-05-20T22:10:24.636685+0000","last_became_peered":"2024-05-20T22:10:24.636685+0000","last_unstale":"2024-05-20T22:10:24.649602+0000","last_undegraded":"2024-05-20T22:10:24.649602+0000","last_fullsized":"2024-05-20T22:10:24.649602+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:44.372720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.646536+0000","last_change":"2024-05-20T22:10:24.635025+0000","last_active":"2024-05-20T22:10:24.646536+0000","last_peered":"2024-05-20T22:10:24.646536+0000","last_clean":"2024-05-20T22:10:24.646536+0000","last_became_active":"2024-05-20T22:10:24.634689+0000","last_became_peered":"2024-05-20T22:10:24.634689+0000","last_unstale":"2024-05-20T22:10:24.646536+0000","last_undegraded":"2024-05-20T22:10:24.646536+0000","last_fullsized":"2024-05-20T22:10:24.646536+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:08:55.858769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620717+0000","last_change":"2024-05-20T22:10:24.645036+0000","last_active":"2024-05-20T22:10:25.620717+0000","last_peered":"2024-05-20T22:10:25.620717+0000","last_clean":"2024-05-20T22:10:25.620717+0000","last_became_active":"2024-05-20T22:10:24.644675+0000","last_became_peered":"2024-05-20T22:10:24.644675+0000","last_unstale":"2024-05-20T22:10:25.620717+0000","last_undegraded":"2024-05-20T22:10:25.620717+0000","last_fullsized":"2024-05-20T22:10:25.620717+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:27:01.866406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649734+0000","last_change":"2024-05-20T22:10:24.637220+0000","last_active":"2024-05-20T22:10:24.649734+0000","last_peered":"2024-05-20T22:10:24.649734+0000","last_clean":"2024-05-20T22:10:24.649734+0000","last_became_active":"2024-05-20T22:10:24.636770+0000","last_became_peered":"2024-05-20T22:10:24.636770+0000","last_unstale":"2024-05-20T22:10:24.649734+0000","last_undegraded":"2024-05-20T22:10:24.649734+0000","last_fullsized":"2024-05-20T22:10:24.649734+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:27:55.130179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636444+0000","last_change":"2024-05-20T22:10:24.646047+0000","last_active":"2024-05-20T22:10:25.636444+0000","last_peered":"2024-05-20T22:10:25.636444+0000","last_clean":"2024-05-20T22:10:25.636444+0000","last_became_active":"2024-05-20T22:10:24.645695+0000","last_became_peered":"2024-05-20T22:10:24.645695+0000","last_unstale":"2024-05-20T22:10:25.636444+0000","last_undegraded":"2024-05-20T22:10:25.636444+0000","last_fullsized":"2024-05-20T22:10:25.636444+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:54:44.645915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617999+0000","last_change":"2024-05-20T22:10:24.629115+0000","last_active":"2024-05-20T22:10:25.617999+0000","last_peered":"2024-05-20T22:10:25.617999+0000","last_clean":"2024-05-20T22:10:25.617999+0000","last_became_active":"2024-05-20T22:10:24.628655+0000","last_became_peered":"2024-05-20T22:10:24.628655+0000","last_unstale":"2024-05-20T22:10:25.617999+0000","last_undegraded":"2024-05-20T22:10:25.617999+0000","last_fullsized":"2024-05-20T22:10:25.617999+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:42:53.054927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649685+0000","last_change":"2024-05-20T22:10:24.637788+0000","last_active":"2024-05-20T22:10:24.649685+0000","last_peered":"2024-05-20T22:10:24.649685+0000","last_clean":"2024-05-20T22:10:24.649685+0000","last_became_active":"2024-05-20T22:10:24.637460+0000","last_became_peered":"2024-05-20T22:10:24.637460+0000","last_unstale":"2024-05-20T22:10:24.649685+0000","last_undegraded":"2024-05-20T22:10:24.649685+0000","last_fullsized":"2024-05-20T22:10:24.649685+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:04.895434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636845+0000","last_change":"2024-05-20T22:10:24.643216+0000","last_active":"2024-05-20T22:10:25.636845+0000","last_peered":"2024-05-20T22:10:25.636845+0000","last_clean":"2024-05-20T22:10:25.636845+0000","last_became_active":"2024-05-20T22:10:24.642746+0000","last_became_peered":"2024-05-20T22:10:24.642746+0000","last_unstale":"2024-05-20T22:10:25.636845+0000","last_undegraded":"2024-05-20T22:10:25.636845+0000","last_fullsized":"2024-05-20T22:10:25.636845+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:27:02.138910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619339+0000","last_change":"2024-05-20T22:10:24.626819+0000","last_active":"2024-05-20T22:10:25.619339+0000","last_peered":"2024-05-20T22:10:25.619339+0000","last_clean":"2024-05-20T22:10:25.619339+0000","last_became_active":"2024-05-20T22:10:24.626158+0000","last_became_peered":"2024-05-20T22:10:24.626158+0000","last_unstale":"2024-05-20T22:10:25.619339+0000","last_undegraded":"2024-05-20T22:10:25.619339+0000","last_fullsized":"2024-05-20T22:10:25.619339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:42:48.287564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636479+0000","last_change":"2024-05-20T22:10:24.637398+0000","last_active":"2024-05-20T22:10:25.636479+0000","last_peered":"2024-05-20T22:10:25.636479+0000","last_clean":"2024-05-20T22:10:25.636479+0000","last_became_active":"2024-05-20T22:10:24.637082+0000","last_became_peered":"2024-05-20T22:10:24.637082+0000","last_unstale":"2024-05-20T22:10:25.636479+0000","last_undegraded":"2024-05-20T22:10:25.636479+0000","last_fullsized":"2024-05-20T22:10:25.636479+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:06:52.487566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.639960+0000","last_change":"2024-05-20T22:10:24.637410+0000","last_active":"2024-05-20T22:10:24.639960+0000","last_peered":"2024-05-20T22:10:24.639960+0000","last_clean":"2024-05-20T22:10:24.639960+0000","last_became_active":"2024-05-20T22:10:24.637103+0000","last_became_peered":"2024-05-20T22:10:24.637103+0000","last_unstale":"2024-05-20T22:10:24.639960+0000","last_undegraded":"2024-05-20T22:10:24.639960+0000","last_fullsized":"2024-05-20T22:10:24.639960+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:38.332210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.634629+0000","last_change":"2024-05-20T22:10:24.630950+0000","last_active":"2024-05-20T22:10:24.634629+0000","last_peered":"2024-05-20T22:10:24.634629+0000","last_clean":"2024-05-20T22:10:24.634629+0000","last_became_active":"2024-05-20T22:10:24.630579+0000","last_became_peered":"2024-05-20T22:10:24.630579+0000","last_unstale":"2024-05-20T22:10:24.634629+0000","last_undegraded":"2024-05-20T22:10:24.634629+0000","last_fullsized":"2024-05-20T22:10:24.634629+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:27:55.130179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620476+0000","last_change":"2024-05-20T22:10:24.660055+0000","last_active":"2024-05-20T22:10:25.620476+0000","last_peered":"2024-05-20T22:10:25.620476+0000","last_clean":"2024-05-20T22:10:25.620476+0000","last_became_active":"2024-05-20T22:10:24.659142+0000","last_became_peered":"2024-05-20T22:10:24.659142+0000","last_unstale":"2024-05-20T22:10:25.620476+0000","last_undegraded":"2024-05-20T22:10:25.620476+0000","last_fullsized":"2024-05-20T22:10:25.620476+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:53:56.291962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.646577+0000","last_change":"2024-05-20T22:10:24.634294+0000","last_active":"2024-05-20T22:10:24.646577+0000","last_peered":"2024-05-20T22:10:24.646577+0000","last_clean":"2024-05-20T22:10:24.646577+0000","last_became_active":"2024-05-20T22:10:24.633962+0000","last_became_peered":"2024-05-20T22:10:24.633962+0000","last_unstale":"2024-05-20T22:10:24.646577+0000","last_undegraded":"2024-05-20T22:10:24.646577+0000","last_fullsized":"2024-05-20T22:10:24.646577+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:43:17.088437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636094+0000","last_change":"2024-05-20T22:10:24.650199+0000","last_active":"2024-05-20T22:10:25.636094+0000","last_peered":"2024-05-20T22:10:25.636094+0000","last_clean":"2024-05-20T22:10:25.636094+0000","last_became_active":"2024-05-20T22:10:24.644859+0000","last_became_peered":"2024-05-20T22:10:24.644859+0000","last_unstale":"2024-05-20T22:10:25.636094+0000","last_undegraded":"2024-05-20T22:10:25.636094+0000","last_fullsized":"2024-05-20T22:10:25.636094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:36.792344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636961+0000","last_change":"2024-05-20T22:10:24.649012+0000","last_active":"2024-05-20T22:10:25.636961+0000","last_peered":"2024-05-20T22:10:25.636961+0000","last_clean":"2024-05-20T22:10:25.636961+0000","last_became_active":"2024-05-20T22:10:24.648675+0000","last_became_peered":"2024-05-20T22:10:24.648675+0000","last_unstale":"2024-05-20T22:10:25.636961+0000","last_undegraded":"2024-05-20T22:10:25.636961+0000","last_fullsized":"2024-05-20T22:10:25.636961+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:31.732256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620431+0000","last_change":"2024-05-20T22:10:24.653378+0000","last_active":"2024-05-20T22:10:25.620431+0000","last_peered":"2024-05-20T22:10:25.620431+0000","last_clean":"2024-05-20T22:10:25.620431+0000","last_became_active":"2024-05-20T22:10:24.651745+0000","last_became_peered":"2024-05-20T22:10:24.651745+0000","last_unstale":"2024-05-20T22:10:25.620431+0000","last_undegraded":"2024-05-20T22:10:25.620431+0000","last_fullsized":"2024-05-20T22:10:25.620431+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:24:31.732256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.650482+0000","last_change":"2024-05-20T22:10:24.642165+0000","last_active":"2024-05-20T22:10:24.650482+0000","last_peered":"2024-05-20T22:10:24.650482+0000","last_clean":"2024-05-20T22:10:24.650482+0000","last_became_active":"2024-05-20T22:10:24.641408+0000","last_became_peered":"2024-05-20T22:10:24.641408+0000","last_unstale":"2024-05-20T22:10:24.650482+0000","last_undegraded":"2024-05-20T22:10:24.650482+0000","last_fullsized":"2024-05-20T22:10:24.650482+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:15:02.617975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.640533+0000","last_change":"2024-05-20T22:10:24.638993+0000","last_active":"2024-05-20T22:10:24.640533+0000","last_peered":"2024-05-20T22:10:24.640533+0000","last_clean":"2024-05-20T22:10:24.640533+0000","last_became_active":"2024-05-20T22:10:24.638687+0000","last_became_peered":"2024-05-20T22:10:24.638687+0000","last_unstale":"2024-05-20T22:10:24.640533+0000","last_undegraded":"2024-05-20T22:10:24.640533+0000","last_fullsized":"2024-05-20T22:10:24.640533+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:15:02.617975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636714+0000","last_change":"2024-05-20T22:10:24.642365+0000","last_active":"2024-05-20T22:10:25.636714+0000","last_peered":"2024-05-20T22:10:25.636714+0000","last_clean":"2024-05-20T22:10:25.636714+0000","last_became_active":"2024-05-20T22:10:24.642032+0000","last_became_peered":"2024-05-20T22:10:24.642032+0000","last_unstale":"2024-05-20T22:10:25.636714+0000","last_undegraded":"2024-05-20T22:10:25.636714+0000","last_fullsized":"2024-05-20T22:10:25.636714+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:19:52.710100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618337+0000","last_change":"2024-05-20T22:10:24.629429+0000","last_active":"2024-05-20T22:10:25.618337+0000","last_peered":"2024-05-20T22:10:25.618337+0000","last_clean":"2024-05-20T22:10:25.618337+0000","last_became_active":"2024-05-20T22:10:24.628979+0000","last_became_peered":"2024-05-20T22:10:24.628979+0000","last_unstale":"2024-05-20T22:10:25.618337+0000","last_undegraded":"2024-05-20T22:10:25.618337+0000","last_fullsized":"2024-05-20T22:10:25.618337+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:40:24.202923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636210+0000","last_change":"2024-05-20T22:10:24.645238+0000","last_active":"2024-05-20T22:10:25.636210+0000","last_peered":"2024-05-20T22:10:25.636210+0000","last_clean":"2024-05-20T22:10:25.636210+0000","last_became_active":"2024-05-20T22:10:24.644612+0000","last_became_peered":"2024-05-20T22:10:24.644612+0000","last_unstale":"2024-05-20T22:10:25.636210+0000","last_undegraded":"2024-05-20T22:10:25.636210+0000","last_fullsized":"2024-05-20T22:10:25.636210+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:37:18.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.652894+0000","last_change":"2024-05-20T22:10:24.648561+0000","last_active":"2024-05-20T22:10:24.652894+0000","last_peered":"2024-05-20T22:10:24.652894+0000","last_clean":"2024-05-20T22:10:24.652894+0000","last_became_active":"2024-05-20T22:10:24.648253+0000","last_became_peered":"2024-05-20T22:10:24.648253+0000","last_unstale":"2024-05-20T22:10:24.652894+0000","last_undegraded":"2024-05-20T22:10:24.652894+0000","last_fullsized":"2024-05-20T22:10:24.652894+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:06.964355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617716+0000","last_change":"2024-05-20T22:10:24.622527+0000","last_active":"2024-05-20T22:10:25.617716+0000","last_peered":"2024-05-20T22:10:25.617716+0000","last_clean":"2024-05-20T22:10:25.617716+0000","last_became_active":"2024-05-20T22:10:24.622146+0000","last_became_peered":"2024-05-20T22:10:24.622146+0000","last_unstale":"2024-05-20T22:10:25.617716+0000","last_undegraded":"2024-05-20T22:10:25.617716+0000","last_fullsized":"2024-05-20T22:10:25.617716+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:06.964355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.650278+0000","last_change":"2024-05-20T22:10:23.617582+0000","last_active":"2024-05-20T22:10:24.650278+0000","last_peered":"2024-05-20T22:10:24.650278+0000","last_clean":"2024-05-20T22:10:24.650278+0000","last_became_active":"2024-05-20T22:10:23.617267+0000","last_became_peered":"2024-05-20T22:10:23.617267+0000","last_unstale":"2024-05-20T22:10:24.650278+0000","last_undegraded":"2024-05-20T22:10:24.650278+0000","last_fullsized":"2024-05-20T22:10:24.650278+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:06:34.084033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.625950+0000","last_change":"2024-05-20T22:10:24.623360+0000","last_active":"2024-05-20T22:10:24.625950+0000","last_peered":"2024-05-20T22:10:24.625950+0000","last_clean":"2024-05-20T22:10:24.625950+0000","last_became_active":"2024-05-20T22:10:24.623022+0000","last_became_peered":"2024-05-20T22:10:24.623022+0000","last_unstale":"2024-05-20T22:10:24.625950+0000","last_undegraded":"2024-05-20T22:10:24.625950+0000","last_fullsized":"2024-05-20T22:10:24.625950+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:05.215640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.658252+0000","last_change":"2024-05-20T22:10:23.623800+0000","last_active":"2024-05-20T22:10:24.658252+0000","last_peered":"2024-05-20T22:10:24.658252+0000","last_clean":"2024-05-20T22:10:24.658252+0000","last_became_active":"2024-05-20T22:10:23.623481+0000","last_became_peered":"2024-05-20T22:10:23.623481+0000","last_unstale":"2024-05-20T22:10:24.658252+0000","last_undegraded":"2024-05-20T22:10:24.658252+0000","last_fullsized":"2024-05-20T22:10:24.658252+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:06:34.084033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.634782+0000","last_change":"2024-05-20T22:10:24.630951+0000","last_active":"2024-05-20T22:10:24.634782+0000","last_peered":"2024-05-20T22:10:24.634782+0000","last_clean":"2024-05-20T22:10:24.634782+0000","last_became_active":"2024-05-20T22:10:24.630327+0000","last_became_peered":"2024-05-20T22:10:24.630327+0000","last_unstale":"2024-05-20T22:10:24.634782+0000","last_undegraded":"2024-05-20T22:10:24.634782+0000","last_fullsized":"2024-05-20T22:10:24.634782+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:04.895434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.650329+0000","last_change":"2024-05-20T22:10:23.620076+0000","last_active":"2024-05-20T22:10:24.650329+0000","last_peered":"2024-05-20T22:10:24.650329+0000","last_clean":"2024-05-20T22:10:24.650329+0000","last_became_active":"2024-05-20T22:10:23.619722+0000","last_became_peered":"2024-05-20T22:10:23.619722+0000","last_unstale":"2024-05-20T22:10:24.650329+0000","last_undegraded":"2024-05-20T22:10:24.650329+0000","last_fullsized":"2024-05-20T22:10:24.650329+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:53.964769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620441+0000","last_change":"2024-05-20T22:10:24.650265+0000","last_active":"2024-05-20T22:10:25.620441+0000","last_peered":"2024-05-20T22:10:25.620441+0000","last_clean":"2024-05-20T22:10:25.620441+0000","last_became_active":"2024-05-20T22:10:24.647516+0000","last_became_peered":"2024-05-20T22:10:24.647516+0000","last_unstale":"2024-05-20T22:10:25.620441+0000","last_undegraded":"2024-05-20T22:10:25.620441+0000","last_fullsized":"2024-05-20T22:10:25.620441+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:15.642998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.660159+0000","last_change":"2024-05-20T22:10:23.628714+0000","last_active":"2024-05-20T22:10:24.660159+0000","last_peered":"2024-05-20T22:10:24.660159+0000","last_clean":"2024-05-20T22:10:24.660159+0000","last_became_active":"2024-05-20T22:10:23.628411+0000","last_became_peered":"2024-05-20T22:10:23.628411+0000","last_unstale":"2024-05-20T22:10:24.660159+0000","last_undegraded":"2024-05-20T22:10:24.660159+0000","last_fullsized":"2024-05-20T22:10:24.660159+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:53.351292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618477+0000","last_change":"2024-05-20T22:10:24.654011+0000","last_active":"2024-05-20T22:10:25.618477+0000","last_peered":"2024-05-20T22:10:25.618477+0000","last_clean":"2024-05-20T22:10:25.618477+0000","last_became_active":"2024-05-20T22:10:24.653737+0000","last_became_peered":"2024-05-20T22:10:24.653737+0000","last_unstale":"2024-05-20T22:10:25.618477+0000","last_undegraded":"2024-05-20T22:10:25.618477+0000","last_fullsized":"2024-05-20T22:10:25.618477+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:01.712783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636410+0000","last_change":"2024-05-20T22:10:23.626863+0000","last_active":"2024-05-20T22:10:25.636410+0000","last_peered":"2024-05-20T22:10:25.636410+0000","last_clean":"2024-05-20T22:10:25.636410+0000","last_became_active":"2024-05-20T22:10:23.625662+0000","last_became_peered":"2024-05-20T22:10:23.625662+0000","last_unstale":"2024-05-20T22:10:25.636410+0000","last_undegraded":"2024-05-20T22:10:25.636410+0000","last_fullsized":"2024-05-20T22:10:25.636410+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:15:01.610969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618877+0000","last_change":"2024-05-20T22:10:24.628495+0000","last_active":"2024-05-20T22:10:25.618877+0000","last_peered":"2024-05-20T22:10:25.618877+0000","last_clean":"2024-05-20T22:10:25.618877+0000","last_became_active":"2024-05-20T22:10:24.628180+0000","last_became_peered":"2024-05-20T22:10:24.628180+0000","last_unstale":"2024-05-20T22:10:25.618877+0000","last_undegraded":"2024-05-20T22:10:25.618877+0000","last_fullsized":"2024-05-20T22:10:25.618877+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:36.792344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619092+0000","last_change":"2024-05-20T22:10:23.619235+0000","last_active":"2024-05-20T22:10:25.619092+0000","last_peered":"2024-05-20T22:10:25.619092+0000","last_clean":"2024-05-20T22:10:25.619092+0000","last_became_active":"2024-05-20T22:10:23.618913+0000","last_became_peered":"2024-05-20T22:10:23.618913+0000","last_unstale":"2024-05-20T22:10:25.619092+0000","last_undegraded":"2024-05-20T22:10:25.619092+0000","last_fullsized":"2024-05-20T22:10:25.619092+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:53.964769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620871+0000","last_change":"2024-05-20T22:10:24.654129+0000","last_active":"2024-05-20T22:10:25.620871+0000","last_peered":"2024-05-20T22:10:25.620871+0000","last_clean":"2024-05-20T22:10:25.620871+0000","last_became_active":"2024-05-20T22:10:24.653796+0000","last_became_peered":"2024-05-20T22:10:24.653796+0000","last_unstale":"2024-05-20T22:10:25.620871+0000","last_undegraded":"2024-05-20T22:10:25.620871+0000","last_fullsized":"2024-05-20T22:10:25.620871+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:03:22.672075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.625076+0000","last_change":"2024-05-20T22:10:23.619196+0000","last_active":"2024-05-20T22:10:24.625076+0000","last_peered":"2024-05-20T22:10:24.625076+0000","last_clean":"2024-05-20T22:10:24.625076+0000","last_became_active":"2024-05-20T22:10:23.618607+0000","last_became_peered":"2024-05-20T22:10:23.618607+0000","last_unstale":"2024-05-20T22:10:24.625076+0000","last_undegraded":"2024-05-20T22:10:24.625076+0000","last_fullsized":"2024-05-20T22:10:24.625076+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:34:26.788796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618988+0000","last_change":"2024-05-20T22:10:24.644938+0000","last_active":"2024-05-20T22:10:25.618988+0000","last_peered":"2024-05-20T22:10:25.618988+0000","last_clean":"2024-05-20T22:10:25.618988+0000","last_became_active":"2024-05-20T22:10:24.644627+0000","last_became_peered":"2024-05-20T22:10:24.644627+0000","last_unstale":"2024-05-20T22:10:25.618988+0000","last_undegraded":"2024-05-20T22:10:25.618988+0000","last_fullsized":"2024-05-20T22:10:25.618988+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:09:16.410294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636106+0000","last_change":"2024-05-20T22:10:23.621369+0000","last_active":"2024-05-20T22:10:25.636106+0000","last_peered":"2024-05-20T22:10:25.636106+0000","last_clean":"2024-05-20T22:10:25.636106+0000","last_became_active":"2024-05-20T22:10:23.621058+0000","last_became_peered":"2024-05-20T22:10:23.621058+0000","last_unstale":"2024-05-20T22:10:25.636106+0000","last_undegraded":"2024-05-20T22:10:25.636106+0000","last_fullsized":"2024-05-20T22:10:25.636106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:03.888428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617998+0000","last_change":"2024-05-20T22:10:24.643807+0000","last_active":"2024-05-20T22:10:25.617998+0000","last_peered":"2024-05-20T22:10:25.617998+0000","last_clean":"2024-05-20T22:10:25.617998+0000","last_became_active":"2024-05-20T22:10:24.642813+0000","last_became_peered":"2024-05-20T22:10:24.642813+0000","last_unstale":"2024-05-20T22:10:25.617998+0000","last_undegraded":"2024-05-20T22:10:25.617998+0000","last_fullsized":"2024-05-20T22:10:25.617998+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:09:05.390222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619376+0000","last_change":"2024-05-20T22:10:23.618509+0000","last_active":"2024-05-20T22:10:25.619376+0000","last_peered":"2024-05-20T22:10:25.619376+0000","last_clean":"2024-05-20T22:10:25.619376+0000","last_became_active":"2024-05-20T22:10:23.618175+0000","last_became_peered":"2024-05-20T22:10:23.618175+0000","last_unstale":"2024-05-20T22:10:25.619376+0000","last_undegraded":"2024-05-20T22:10:25.619376+0000","last_fullsized":"2024-05-20T22:10:25.619376+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:55:00.552671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.638792+0000","last_change":"2024-05-20T22:10:24.636478+0000","last_active":"2024-05-20T22:10:24.638792+0000","last_peered":"2024-05-20T22:10:24.638792+0000","last_clean":"2024-05-20T22:10:24.638792+0000","last_became_active":"2024-05-20T22:10:24.636144+0000","last_became_peered":"2024-05-20T22:10:24.636144+0000","last_unstale":"2024-05-20T22:10:24.638792+0000","last_undegraded":"2024-05-20T22:10:24.638792+0000","last_fullsized":"2024-05-20T22:10:24.638792+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:37.478610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617991+0000","last_change":"2024-05-20T22:10:23.612855+0000","last_active":"2024-05-20T22:10:25.617991+0000","last_peered":"2024-05-20T22:10:25.617991+0000","last_clean":"2024-05-20T22:10:25.617991+0000","last_became_active":"2024-05-20T22:10:23.612550+0000","last_became_peered":"2024-05-20T22:10:23.612550+0000","last_unstale":"2024-05-20T22:10:25.617991+0000","last_undegraded":"2024-05-20T22:10:25.617991+0000","last_fullsized":"2024-05-20T22:10:25.617991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:50.824414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636499+0000","last_change":"2024-05-20T22:10:24.638769+0000","last_active":"2024-05-20T22:10:25.636499+0000","last_peered":"2024-05-20T22:10:25.636499+0000","last_clean":"2024-05-20T22:10:25.636499+0000","last_became_active":"2024-05-20T22:10:24.638451+0000","last_became_peered":"2024-05-20T22:10:24.638451+0000","last_unstale":"2024-05-20T22:10:25.636499+0000","last_undegraded":"2024-05-20T22:10:25.636499+0000","last_fullsized":"2024-05-20T22:10:25.636499+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:13:30.588413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619121+0000","last_change":"2024-05-20T22:10:23.621269+0000","last_active":"2024-05-20T22:10:25.619121+0000","last_peered":"2024-05-20T22:10:25.619121+0000","last_clean":"2024-05-20T22:10:25.619121+0000","last_became_active":"2024-05-20T22:10:23.620952+0000","last_became_peered":"2024-05-20T22:10:23.620952+0000","last_unstale":"2024-05-20T22:10:25.619121+0000","last_undegraded":"2024-05-20T22:10:25.619121+0000","last_fullsized":"2024-05-20T22:10:25.619121+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:21.960465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619042+0000","last_change":"2024-05-20T22:10:24.647371+0000","last_active":"2024-05-20T22:10:25.619042+0000","last_peered":"2024-05-20T22:10:25.619042+0000","last_clean":"2024-05-20T22:10:25.619042+0000","last_became_active":"2024-05-20T22:10:24.647139+0000","last_became_peered":"2024-05-20T22:10:24.647139+0000","last_unstale":"2024-05-20T22:10:25.619042+0000","last_undegraded":"2024-05-20T22:10:25.619042+0000","last_fullsized":"2024-05-20T22:10:25.619042+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:37.099887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":20,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620906+0000","last_change":"2024-05-20T22:10:23.625931+0000","last_active":"2024-05-20T22:10:25.620906+0000","last_peered":"2024-05-20T22:10:25.620906+0000","last_clean":"2024-05-20T22:10:25.620906+0000","last_became_active":"2024-05-20T22:10:23.625593+0000","last_became_peered":"2024-05-20T22:10:23.625593+0000","last_unstale":"2024-05-20T22:10:25.620906+0000","last_undegraded":"2024-05-20T22:10:25.620906+0000","last_fullsized":"2024-05-20T22:10:25.620906+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:46:35.785338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.630199+0000","last_change":"2024-05-20T22:10:24.625101+0000","last_active":"2024-05-20T22:10:24.630199+0000","last_peered":"2024-05-20T22:10:24.630199+0000","last_clean":"2024-05-20T22:10:24.630199+0000","last_became_active":"2024-05-20T22:10:24.623588+0000","last_became_peered":"2024-05-20T22:10:24.623588+0000","last_unstale":"2024-05-20T22:10:24.630199+0000","last_undegraded":"2024-05-20T22:10:24.630199+0000","last_fullsized":"2024-05-20T22:10:24.630199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:47.850915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636541+0000","last_change":"2024-05-20T22:10:23.619729+0000","last_active":"2024-05-20T22:10:25.636541+0000","last_peered":"2024-05-20T22:10:25.636541+0000","last_clean":"2024-05-20T22:10:25.636541+0000","last_became_active":"2024-05-20T22:10:23.619414+0000","last_became_peered":"2024-05-20T22:10:23.619414+0000","last_unstale":"2024-05-20T22:10:25.636541+0000","last_undegraded":"2024-05-20T22:10:25.636541+0000","last_fullsized":"2024-05-20T22:10:25.636541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:37.325204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636621+0000","last_change":"2024-05-20T22:10:24.648257+0000","last_active":"2024-05-20T22:10:25.636621+0000","last_peered":"2024-05-20T22:10:25.636621+0000","last_clean":"2024-05-20T22:10:25.636621+0000","last_became_active":"2024-05-20T22:10:24.647910+0000","last_became_peered":"2024-05-20T22:10:24.647910+0000","last_unstale":"2024-05-20T22:10:25.636621+0000","last_undegraded":"2024-05-20T22:10:25.636621+0000","last_fullsized":"2024-05-20T22:10:25.636621+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:21:10.915657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.625128+0000","last_change":"2024-05-20T22:10:23.627363+0000","last_active":"2024-05-20T22:10:24.625128+0000","last_peered":"2024-05-20T22:10:24.625128+0000","last_clean":"2024-05-20T22:10:24.625128+0000","last_became_active":"2024-05-20T22:10:23.627071+0000","last_became_peered":"2024-05-20T22:10:23.627071+0000","last_unstale":"2024-05-20T22:10:24.625128+0000","last_undegraded":"2024-05-20T22:10:24.625128+0000","last_fullsized":"2024-05-20T22:10:24.625128+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:08.531292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620707+0000","last_change":"2024-05-20T22:10:24.639544+0000","last_active":"2024-05-20T22:10:25.620707+0000","last_peered":"2024-05-20T22:10:25.620707+0000","last_clean":"2024-05-20T22:10:25.620707+0000","last_became_active":"2024-05-20T22:10:24.639197+0000","last_became_peered":"2024-05-20T22:10:24.639197+0000","last_unstale":"2024-05-20T22:10:25.620707+0000","last_undegraded":"2024-05-20T22:10:25.620707+0000","last_fullsized":"2024-05-20T22:10:25.620707+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:36:53.606875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618102+0000","last_change":"2024-05-20T22:10:23.619196+0000","last_active":"2024-05-20T22:10:25.618102+0000","last_peered":"2024-05-20T22:10:25.618102+0000","last_clean":"2024-05-20T22:10:25.618102+0000","last_became_active":"2024-05-20T22:10:23.618882+0000","last_became_peered":"2024-05-20T22:10:23.618882+0000","last_unstale":"2024-05-20T22:10:25.618102+0000","last_undegraded":"2024-05-20T22:10:25.618102+0000","last_fullsized":"2024-05-20T22:10:25.618102+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:36.092881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636957+0000","last_change":"2024-05-20T22:10:24.636153+0000","last_active":"2024-05-20T22:10:25.636957+0000","last_peered":"2024-05-20T22:10:25.636957+0000","last_clean":"2024-05-20T22:10:25.636957+0000","last_became_active":"2024-05-20T22:10:24.635808+0000","last_became_peered":"2024-05-20T22:10:24.635808+0000","last_unstale":"2024-05-20T22:10:25.636957+0000","last_undegraded":"2024-05-20T22:10:25.636957+0000","last_fullsized":"2024-05-20T22:10:25.636957+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:29:26.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":22,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620522+0000","last_change":"2024-05-20T22:10:23.628057+0000","last_active":"2024-05-20T22:10:25.620522+0000","last_peered":"2024-05-20T22:10:25.620522+0000","last_clean":"2024-05-20T22:10:25.620522+0000","last_became_active":"2024-05-20T22:10:23.626835+0000","last_became_peered":"2024-05-20T22:10:23.626835+0000","last_unstale":"2024-05-20T22:10:25.620522+0000","last_undegraded":"2024-05-20T22:10:25.620522+0000","last_fullsized":"2024-05-20T22:10:25.620522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:09:10.482777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620613+0000","last_change":"2024-05-20T22:10:24.649307+0000","last_active":"2024-05-20T22:10:25.620613+0000","last_peered":"2024-05-20T22:10:25.620613+0000","last_clean":"2024-05-20T22:10:25.620613+0000","last_became_active":"2024-05-20T22:10:24.648978+0000","last_became_peered":"2024-05-20T22:10:24.648978+0000","last_unstale":"2024-05-20T22:10:25.620613+0000","last_undegraded":"2024-05-20T22:10:25.620613+0000","last_fullsized":"2024-05-20T22:10:25.620613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:36:31.103907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.637025+0000","last_change":"2024-05-20T22:10:23.627338+0000","last_active":"2024-05-20T22:10:25.637025+0000","last_peered":"2024-05-20T22:10:25.637025+0000","last_clean":"2024-05-20T22:10:25.637025+0000","last_became_active":"2024-05-20T22:10:23.626207+0000","last_became_peered":"2024-05-20T22:10:23.626207+0000","last_unstale":"2024-05-20T22:10:25.637025+0000","last_undegraded":"2024-05-20T22:10:25.637025+0000","last_fullsized":"2024-05-20T22:10:25.637025+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:36.092881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649652+0000","last_change":"2024-05-20T22:10:24.642583+0000","last_active":"2024-05-20T22:10:24.649652+0000","last_peered":"2024-05-20T22:10:24.649652+0000","last_clean":"2024-05-20T22:10:24.649652+0000","last_became_active":"2024-05-20T22:10:24.642253+0000","last_became_peered":"2024-05-20T22:10:24.642253+0000","last_unstale":"2024-05-20T22:10:24.649652+0000","last_undegraded":"2024-05-20T22:10:24.649652+0000","last_fullsized":"2024-05-20T22:10:24.649652+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:37.099887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.653139+0000","last_change":"2024-05-20T22:10:23.621005+0000","last_active":"2024-05-20T22:10:24.653139+0000","last_peered":"2024-05-20T22:10:24.653139+0000","last_clean":"2024-05-20T22:10:24.653139+0000","last_became_active":"2024-05-20T22:10:23.620567+0000","last_became_peered":"2024-05-20T22:10:23.620567+0000","last_unstale":"2024-05-20T22:10:24.653139+0000","last_undegraded":"2024-05-20T22:10:24.653139+0000","last_fullsized":"2024-05-20T22:10:24.653139+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:56:15.656528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.618728+0000","last_change":"2024-05-20T22:10:24.654083+0000","last_active":"2024-05-20T22:10:25.618728+0000","last_peered":"2024-05-20T22:10:25.618728+0000","last_clean":"2024-05-20T22:10:25.618728+0000","last_became_active":"2024-05-20T22:10:24.653774+0000","last_became_peered":"2024-05-20T22:10:24.653774+0000","last_unstale":"2024-05-20T22:10:25.618728+0000","last_undegraded":"2024-05-20T22:10:25.618728+0000","last_fullsized":"2024-05-20T22:10:25.618728+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:38.332210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.650856+0000","last_change":"2024-05-20T22:10:23.622913+0000","last_active":"2024-05-20T22:10:24.650856+0000","last_peered":"2024-05-20T22:10:24.650856+0000","last_clean":"2024-05-20T22:10:24.650856+0000","last_became_active":"2024-05-20T22:10:23.622559+0000","last_became_peered":"2024-05-20T22:10:23.622559+0000","last_unstale":"2024-05-20T22:10:24.650856+0000","last_undegraded":"2024-05-20T22:10:24.650856+0000","last_fullsized":"2024-05-20T22:10:24.650856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:43.365714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649980+0000","last_change":"2024-05-20T22:10:24.640570+0000","last_active":"2024-05-20T22:10:24.649980+0000","last_peered":"2024-05-20T22:10:24.649980+0000","last_clean":"2024-05-20T22:10:24.649980+0000","last_became_active":"2024-05-20T22:10:24.639972+0000","last_became_peered":"2024-05-20T22:10:24.639972+0000","last_unstale":"2024-05-20T22:10:24.649980+0000","last_undegraded":"2024-05-20T22:10:24.649980+0000","last_fullsized":"2024-05-20T22:10:24.649980+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:01.712783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.624481+0000","last_change":"2024-05-20T22:10:23.623034+0000","last_active":"2024-05-20T22:10:24.624481+0000","last_peered":"2024-05-20T22:10:24.624481+0000","last_clean":"2024-05-20T22:10:24.624481+0000","last_became_active":"2024-05-20T22:10:23.622605+0000","last_became_peered":"2024-05-20T22:10:23.622605+0000","last_unstale":"2024-05-20T22:10:24.624481+0000","last_undegraded":"2024-05-20T22:10:24.624481+0000","last_fullsized":"2024-05-20T22:10:24.624481+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:50.824414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.638753+0000","last_change":"2024-05-20T22:10:24.637293+0000","last_active":"2024-05-20T22:10:24.638753+0000","last_peered":"2024-05-20T22:10:24.638753+0000","last_clean":"2024-05-20T22:10:24.638753+0000","last_became_active":"2024-05-20T22:10:24.636981+0000","last_became_peered":"2024-05-20T22:10:24.636981+0000","last_unstale":"2024-05-20T22:10:24.638753+0000","last_undegraded":"2024-05-20T22:10:24.638753+0000","last_fullsized":"2024-05-20T22:10:24.638753+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:06.749616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649906+0000","last_change":"2024-05-20T22:10:23.620498+0000","last_active":"2024-05-20T22:10:24.649906+0000","last_peered":"2024-05-20T22:10:24.649906+0000","last_clean":"2024-05-20T22:10:24.649906+0000","last_became_active":"2024-05-20T22:10:23.620146+0000","last_became_peered":"2024-05-20T22:10:23.620146+0000","last_unstale":"2024-05-20T22:10:24.649906+0000","last_undegraded":"2024-05-20T22:10:24.649906+0000","last_fullsized":"2024-05-20T22:10:24.649906+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:50.824414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.638921+0000","last_change":"2024-05-20T22:10:24.636304+0000","last_active":"2024-05-20T22:10:24.638921+0000","last_peered":"2024-05-20T22:10:24.638921+0000","last_clean":"2024-05-20T22:10:24.638921+0000","last_became_active":"2024-05-20T22:10:24.635983+0000","last_became_peered":"2024-05-20T22:10:24.635983+0000","last_unstale":"2024-05-20T22:10:24.638921+0000","last_undegraded":"2024-05-20T22:10:24.638921+0000","last_fullsized":"2024-05-20T22:10:24.638921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:01.712783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620257+0000","last_change":"2024-05-20T22:10:23.630021+0000","last_active":"2024-05-20T22:10:25.620257+0000","last_peered":"2024-05-20T22:10:25.620257+0000","last_clean":"2024-05-20T22:10:25.620257+0000","last_became_active":"2024-05-20T22:10:23.629231+0000","last_became_peered":"2024-05-20T22:10:23.629231+0000","last_unstale":"2024-05-20T22:10:25.620257+0000","last_undegraded":"2024-05-20T22:10:25.620257+0000","last_fullsized":"2024-05-20T22:10:25.620257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:42:52.047921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636643+0000","last_change":"2024-05-20T22:10:24.633829+0000","last_active":"2024-05-20T22:10:25.636643+0000","last_peered":"2024-05-20T22:10:25.636643+0000","last_clean":"2024-05-20T22:10:25.636643+0000","last_became_active":"2024-05-20T22:10:24.633516+0000","last_became_peered":"2024-05-20T22:10:24.633516+0000","last_unstale":"2024-05-20T22:10:25.636643+0000","last_undegraded":"2024-05-20T22:10:25.636643+0000","last_fullsized":"2024-05-20T22:10:25.636643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:24.021946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636270+0000","last_change":"2024-05-20T22:10:23.622926+0000","last_active":"2024-05-20T22:10:25.636270+0000","last_peered":"2024-05-20T22:10:25.636270+0000","last_clean":"2024-05-20T22:10:25.636270+0000","last_became_active":"2024-05-20T22:10:23.622595+0000","last_became_peered":"2024-05-20T22:10:23.622595+0000","last_unstale":"2024-05-20T22:10:25.636270+0000","last_undegraded":"2024-05-20T22:10:25.636270+0000","last_fullsized":"2024-05-20T22:10:25.636270+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:00.705778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649098+0000","last_change":"2024-05-20T22:10:24.642105+0000","last_active":"2024-05-20T22:10:24.649098+0000","last_peered":"2024-05-20T22:10:24.649098+0000","last_clean":"2024-05-20T22:10:24.649098+0000","last_became_active":"2024-05-20T22:10:24.641788+0000","last_became_peered":"2024-05-20T22:10:24.641788+0000","last_unstale":"2024-05-20T22:10:24.649098+0000","last_undegraded":"2024-05-20T22:10:24.649098+0000","last_fullsized":"2024-05-20T22:10:24.649098+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:38.332210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.626525+0000","last_change":"2024-05-20T22:10:23.624437+0000","last_active":"2024-05-20T22:10:24.626525+0000","last_peered":"2024-05-20T22:10:24.626525+0000","last_clean":"2024-05-20T22:10:24.626525+0000","last_became_active":"2024-05-20T22:10:23.622824+0000","last_became_peered":"2024-05-20T22:10:23.622824+0000","last_unstale":"2024-05-20T22:10:24.626525+0000","last_undegraded":"2024-05-20T22:10:24.626525+0000","last_fullsized":"2024-05-20T22:10:24.626525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:26.244777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.652880+0000","last_change":"2024-05-20T22:10:24.648539+0000","last_active":"2024-05-20T22:10:24.652880+0000","last_peered":"2024-05-20T22:10:24.652880+0000","last_clean":"2024-05-20T22:10:24.652880+0000","last_became_active":"2024-05-20T22:10:24.648230+0000","last_became_peered":"2024-05-20T22:10:24.648230+0000","last_unstale":"2024-05-20T22:10:24.652880+0000","last_undegraded":"2024-05-20T22:10:24.652880+0000","last_fullsized":"2024-05-20T22:10:24.652880+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:50:00.634939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.655864+0000","last_change":"2024-05-20T22:10:23.615216+0000","last_active":"2024-05-20T22:10:24.655864+0000","last_peered":"2024-05-20T22:10:24.655864+0000","last_clean":"2024-05-20T22:10:24.655864+0000","last_became_active":"2024-05-20T22:10:23.614877+0000","last_became_peered":"2024-05-20T22:10:23.614877+0000","last_unstale":"2024-05-20T22:10:24.655864+0000","last_undegraded":"2024-05-20T22:10:24.655864+0000","last_fullsized":"2024-05-20T22:10:24.655864+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:53.964769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.649561+0000","last_change":"2024-05-20T22:10:24.638337+0000","last_active":"2024-05-20T22:10:24.649561+0000","last_peered":"2024-05-20T22:10:24.649561+0000","last_clean":"2024-05-20T22:10:24.649561+0000","last_became_active":"2024-05-20T22:10:24.638011+0000","last_became_peered":"2024-05-20T22:10:24.638011+0000","last_unstale":"2024-05-20T22:10:24.649561+0000","last_undegraded":"2024-05-20T22:10:24.649561+0000","last_fullsized":"2024-05-20T22:10:24.649561+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:09:05.390222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.619239+0000","last_change":"2024-05-20T22:10:23.625806+0000","last_active":"2024-05-20T22:10:25.619239+0000","last_peered":"2024-05-20T22:10:25.619239+0000","last_clean":"2024-05-20T22:10:25.619239+0000","last_became_active":"2024-05-20T22:10:23.625513+0000","last_became_peered":"2024-05-20T22:10:23.625513+0000","last_unstale":"2024-05-20T22:10:25.619239+0000","last_undegraded":"2024-05-20T22:10:25.619239+0000","last_fullsized":"2024-05-20T22:10:25.619239+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:34:52.256297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620422+0000","last_change":"2024-05-20T22:10:24.647060+0000","last_active":"2024-05-20T22:10:25.620422+0000","last_peered":"2024-05-20T22:10:25.620422+0000","last_clean":"2024-05-20T22:10:25.620422+0000","last_became_active":"2024-05-20T22:10:24.644261+0000","last_became_peered":"2024-05-20T22:10:24.644261+0000","last_unstale":"2024-05-20T22:10:25.620422+0000","last_undegraded":"2024-05-20T22:10:25.620422+0000","last_fullsized":"2024-05-20T22:10:25.620422+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:29:26.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636796+0000","last_change":"2024-05-20T22:10:23.612870+0000","last_active":"2024-05-20T22:10:25.636796+0000","last_peered":"2024-05-20T22:10:25.636796+0000","last_clean":"2024-05-20T22:10:25.636796+0000","last_became_active":"2024-05-20T22:10:23.612534+0000","last_became_peered":"2024-05-20T22:10:23.612534+0000","last_unstale":"2024-05-20T22:10:25.636796+0000","last_undegraded":"2024-05-20T22:10:25.636796+0000","last_fullsized":"2024-05-20T22:10:25.636796+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:09:04.383216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636690+0000","last_change":"2024-05-20T22:10:24.649287+0000","last_active":"2024-05-20T22:10:25.636690+0000","last_peered":"2024-05-20T22:10:25.636690+0000","last_clean":"2024-05-20T22:10:25.636690+0000","last_became_active":"2024-05-20T22:10:24.648939+0000","last_became_peered":"2024-05-20T22:10:24.648939+0000","last_unstale":"2024-05-20T22:10:25.636690+0000","last_undegraded":"2024-05-20T22:10:25.636690+0000","last_fullsized":"2024-05-20T22:10:25.636690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:49:11.888028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620553+0000","last_change":"2024-05-20T22:10:23.629627+0000","last_active":"2024-05-20T22:10:25.620553+0000","last_peered":"2024-05-20T22:10:25.620553+0000","last_clean":"2024-05-20T22:10:25.620553+0000","last_became_active":"2024-05-20T22:10:23.629252+0000","last_became_peered":"2024-05-20T22:10:23.629252+0000","last_unstale":"2024-05-20T22:10:25.620553+0000","last_undegraded":"2024-05-20T22:10:25.620553+0000","last_fullsized":"2024-05-20T22:10:25.620553+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:46:05.957349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.626141+0000","last_change":"2024-05-20T22:10:24.624042+0000","last_active":"2024-05-20T22:10:24.626141+0000","last_peered":"2024-05-20T22:10:24.626141+0000","last_clean":"2024-05-20T22:10:24.626141+0000","last_became_active":"2024-05-20T22:10:24.623716+0000","last_became_peered":"2024-05-20T22:10:24.623716+0000","last_unstale":"2024-05-20T22:10:24.626141+0000","last_undegraded":"2024-05-20T22:10:24.626141+0000","last_fullsized":"2024-05-20T22:10:24.626141+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:08:03.319713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620381+0000","last_change":"2024-05-20T22:10:23.628520+0000","last_active":"2024-05-20T22:10:25.620381+0000","last_peered":"2024-05-20T22:10:25.620381+0000","last_clean":"2024-05-20T22:10:25.620381+0000","last_became_active":"2024-05-20T22:10:23.628198+0000","last_became_peered":"2024-05-20T22:10:23.628198+0000","last_unstale":"2024-05-20T22:10:25.620381+0000","last_undegraded":"2024-05-20T22:10:25.620381+0000","last_fullsized":"2024-05-20T22:10:25.620381+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:40:23.195917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.636457+0000","last_change":"2024-05-20T22:10:24.646294+0000","last_active":"2024-05-20T22:10:25.636457+0000","last_peered":"2024-05-20T22:10:25.636457+0000","last_clean":"2024-05-20T22:10:25.636457+0000","last_became_active":"2024-05-20T22:10:24.645374+0000","last_became_peered":"2024-05-20T22:10:24.645374+0000","last_unstale":"2024-05-20T22:10:25.636457+0000","last_undegraded":"2024-05-20T22:10:25.636457+0000","last_fullsized":"2024-05-20T22:10:25.636457+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:50.138737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617809+0000","last_change":"2024-05-20T22:10:23.625961+0000","last_active":"2024-05-20T22:10:25.617809+0000","last_peered":"2024-05-20T22:10:25.617809+0000","last_clean":"2024-05-20T22:10:25.617809+0000","last_became_active":"2024-05-20T22:10:23.625606+0000","last_became_peered":"2024-05-20T22:10:23.625606+0000","last_unstale":"2024-05-20T22:10:25.617809+0000","last_undegraded":"2024-05-20T22:10:25.617809+0000","last_fullsized":"2024-05-20T22:10:25.617809+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:06:34.084033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.620797+0000","last_change":"2024-05-20T22:10:24.648685+0000","last_active":"2024-05-20T22:10:25.620797+0000","last_peered":"2024-05-20T22:10:25.620797+0000","last_clean":"2024-05-20T22:10:25.620797+0000","last_became_active":"2024-05-20T22:10:24.647432+0000","last_became_peered":"2024-05-20T22:10:24.647432+0000","last_unstale":"2024-05-20T22:10:25.620797+0000","last_undegraded":"2024-05-20T22:10:25.620797+0000","last_fullsized":"2024-05-20T22:10:25.620797+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:35.226715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.617811+0000","last_change":"2024-05-20T22:10:23.622905+0000","last_active":"2024-05-20T22:10:24.617811+0000","last_peered":"2024-05-20T22:10:24.617811+0000","last_clean":"2024-05-20T22:10:24.617811+0000","last_became_active":"2024-05-20T22:10:23.622595+0000","last_became_peered":"2024-05-20T22:10:23.622595+0000","last_unstale":"2024-05-20T22:10:24.617811+0000","last_undegraded":"2024-05-20T22:10:24.617811+0000","last_fullsized":"2024-05-20T22:10:24.617811+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:22:09.244992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-20T22:10:24.635232+0000","last_change":"2024-05-20T22:10:24.634218+0000","last_active":"2024-05-20T22:10:24.635232+0000","last_peered":"2024-05-20T22:10:24.635232+0000","last_clean":"2024-05-20T22:10:24.635232+0000","last_became_active":"2024-05-20T22:10:24.633896+0000","last_became_peered":"2024-05-20T22:10:24.633896+0000","last_unstale":"2024-05-20T22:10:24.635232+0000","last_undegraded":"2024-05-20T22:10:24.635232+0000","last_fullsized":"2024-05-20T22:10:24.635232+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:23.595799+0000","last_clean_scrub_stamp":"2024-05-20T22:10:23.595799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:09:05.390222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"55'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.638375+0000","last_change":"2024-05-20T22:10:23.620237+0000","last_active":"2024-05-20T22:10:25.638375+0000","last_peered":"2024-05-20T22:10:25.638375+0000","last_clean":"2024-05-20T22:10:25.638375+0000","last_became_active":"2024-05-20T22:10:23.619876+0000","last_became_peered":"2024-05-20T22:10:23.619876+0000","last_unstale":"2024-05-20T22:10:25.638375+0000","last_undegraded":"2024-05-20T22:10:25.638375+0000","last_fullsized":"2024-05-20T22:10:25.638375+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:10:22.588793+0000","last_clean_scrub_stamp":"2024-05-20T22:10:22.588793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:27:54.123173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":241,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-20T22:10:25.617732+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:10:25.617732+0000","last_peered":"2024-05-20T22:10:25.617732+0000","last_clean":"2024-05-20T22:10:25.617732+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:10:25.617732+0000","last_undegraded":"2024-05-20T22:10:25.617732+0000","last_fullsized":"2024-05-20T22:10:25.617732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":886766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":290234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":289582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":290164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":659456,"data_stored":291216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":655360,"data_stored":291194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":651264,"data_stored":289742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1245184,"data_stored":880006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:10:29.083 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T22:10:29.083 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-20T22:10:29.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.239+0000 7fd065f45640 1 Processor -- start 2024-05-20T22:10:29.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.239+0000 7fd065f45640 1 -- start start 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.240+0000 7fd065f45640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd0600cf680 0x7fd06015f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.240+0000 7fd065f45640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0600b7fb0 0x7fd0600b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.240+0000 7fd065f45640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd06006b6a0 con 0x7fd06006c460 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.240+0000 7fd065f45640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd06006b980 con 0x7fd0600cf680 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.240+0000 7fd065f45640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd06006bc60 con 0x7fd0600b7fb0 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.240+0000 7fd05ffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd06006c460 0x7fd06006c870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59688/0 (socket says 172.21.15.55:59688) 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.240+0000 7fd05ffff640 1 -- 172.21.15.55:0/1386160084 learned_addr learned my addr 172.21.15.55:0/1386160084 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.241+0000 7fd05f7fe640 1 --2- 172.21.15.55:0/1386160084 >> v2:172.21.15.138:3300/0 conn(0x7fd0600cf680 0x7fd06015f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.241+0000 7fd05effd640 1 --2- 172.21.15.55:0/1386160084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0600b7fb0 0x7fd0600b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.241+0000 7fd05e7fc640 1 -- 172.21.15.55:0/1386160084 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4988960 0 0) 0x7fd06006b6a0 con 0x7fd06006c460 2024-05-20T22:10:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.241+0000 7fd05e7fc640 1 -- 172.21.15.55:0/1386160084 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd048003520 con 0x7fd06006c460 2024-05-20T22:10:29.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.241+0000 7fd05e7fc640 1 -- 172.21.15.55:0/1386160084 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2402365713 0 0) 0x7fd048003520 con 0x7fd06006c460 2024-05-20T22:10:29.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.241+0000 7fd05f7fe640 1 -- 172.21.15.55:0/1386160084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0600b7fb0 msgr2=0x7fd0600b83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:29.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.241+0000 7fd05f7fe640 1 --2- 172.21.15.55:0/1386160084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0600b7fb0 0x7fd0600b83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.241+0000 7fd05f7fe640 1 -- 172.21.15.55:0/1386160084 >> v1:172.21.15.55:6789/0 conn(0x7fd06006c460 legacy=0x7fd06006c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.241+0000 7fd05f7fe640 1 -- 172.21.15.55:0/1386160084 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06006bf40 con 0x7fd0600cf680 2024-05-20T22:10:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.242+0000 7fd05f7fe640 1 --2- 172.21.15.55:0/1386160084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0600cf680 0x7fd06015f080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd0500097a0 tx=0x7fd05002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aefb928d50b98615 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.242+0000 7fd05e7fc640 1 -- 172.21.15.55:0/1386160084 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050037070 con 0x7fd0600cf680 2024-05-20T22:10:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.242+0000 7fd05e7fc640 1 -- 172.21.15.55:0/1386160084 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05002ab10 con 0x7fd0600cf680 2024-05-20T22:10:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.242+0000 7fd05e7fc640 1 -- 172.21.15.55:0/1386160084 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0580019f0 con 0x7fd0600cf680 2024-05-20T22:10:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.243+0000 7fd065f45640 1 -- 172.21.15.55:0/1386160084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0600cf680 msgr2=0x7fd06015f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.243+0000 7fd065f45640 1 --2- 172.21.15.55:0/1386160084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0600cf680 0x7fd06015f080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd0500097a0 tx=0x7fd05002a080 comp rx=0 tx=0).stop 2024-05-20T22:10:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.243+0000 7fd065f45640 1 -- 172.21.15.55:0/1386160084 shutdown_connections 2024-05-20T22:10:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.243+0000 7fd065f45640 1 --2- 172.21.15.55:0/1386160084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0600b7fb0 0x7fd0600b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.243+0000 7fd065f45640 1 --2- 172.21.15.55:0/1386160084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0600cf680 0x7fd06015f080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.243+0000 7fd065f45640 1 -- 172.21.15.55:0/1386160084 >> 172.21.15.55:0/1386160084 conn(0x7fd0600d5450 msgr2=0x7fd0600b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:29.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.243+0000 7fd065f45640 1 -- 172.21.15.55:0/1386160084 shutdown_connections 2024-05-20T22:10:29.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.243+0000 7fd065f45640 1 -- 172.21.15.55:0/1386160084 wait complete. 2024-05-20T22:10:29.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.245+0000 7fd065f45640 1 Processor -- start 2024-05-20T22:10:29.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.245+0000 7fd065f45640 1 -- start start 2024-05-20T22:10:29.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd065f45640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd06006c460 0x7fd0601f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd065f45640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0600b7fb0 0x7fd0600c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd065f45640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0600cf680 0x7fd060161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd065f45640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd0600d3740 con 0x7fd0600b7fb0 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd065f45640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0600d35c0 con 0x7fd06006c460 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd065f45640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd0600d38c0 con 0x7fd0600cf680 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd05effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0600b7fb0 0x7fd0600c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd05effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0600b7fb0 0x7fd0600c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57304/0 (socket says 172.21.15.55:57304) 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd05effd640 1 -- 172.21.15.55:0/2949576695 learned_addr learned my addr 172.21.15.55:0/2949576695 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd05f7fe640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd06006c460 0x7fd0601f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.246+0000 7fd05ffff640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0600cf680 0x7fd060161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd05effd640 1 -- 172.21.15.55:0/2949576695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0600cf680 msgr2=0x7fd060161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd05effd640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0600cf680 0x7fd060161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd05effd640 1 -- 172.21.15.55:0/2949576695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd06006c460 msgr2=0x7fd0601f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd05effd640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd06006c460 0x7fd0601f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd05effd640 1 -- 172.21.15.55:0/2949576695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0600c8f70 con 0x7fd0600b7fb0 2024-05-20T22:10:29.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd05effd640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0600b7fb0 0x7fd0600c8590 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd05400e870 tx=0x7fd05400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:29.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd047fff640 1 -- 172.21.15.55:0/2949576695 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054017070 con 0x7fd0600b7fb0 2024-05-20T22:10:29.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd047fff640 1 -- 172.21.15.55:0/2949576695 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05400c940 con 0x7fd0600b7fb0 2024-05-20T22:10:29.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd047fff640 1 -- 172.21.15.55:0/2949576695 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05400cd30 con 0x7fd0600b7fb0 2024-05-20T22:10:29.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd065f45640 1 -- 172.21.15.55:0/2949576695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0600c9250 con 0x7fd0600b7fb0 2024-05-20T22:10:29.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.247+0000 7fd065f45640 1 -- 172.21.15.55:0/2949576695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06014ce80 con 0x7fd0600b7fb0 2024-05-20T22:10:29.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.248+0000 7fd065f45640 1 -- 172.21.15.55:0/2949576695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06014c790 con 0x7fd0600b7fb0 2024-05-20T22:10:29.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.249+0000 7fd047fff640 1 -- 172.21.15.55:0/2949576695 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd05401a070 con 0x7fd0600b7fb0 2024-05-20T22:10:29.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.249+0000 7fd047fff640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd04c06be30 0x7fd04c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.249+0000 7fd047fff640 1 -- 172.21.15.55:0/2949576695 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7fd05408b210 con 0x7fd0600b7fb0 2024-05-20T22:10:29.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.249+0000 7fd05f7fe640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd04c06be30 0x7fd04c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.250+0000 7fd05f7fe640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd04c06be30 0x7fd04c06e340 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd050034000 tx=0x7fd05002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.254+0000 7fd047fff640 1 -- 172.21.15.55:0/2949576695 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd06014c790 con 0x7fd0600b7fb0 2024-05-20T22:10:29.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.513+0000 7fd065f45640 1 -- 172.21.15.55:0/2949576695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd0600c8d00 con 0x7fd0600b7fb0 2024-05-20T22:10:29.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.515+0000 7fd047fff640 1 -- 172.21.15.55:0/2949576695 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fd054059a30 con 0x7fd0600b7fb0 2024-05-20T22:10:29.516 INFO:teuthology.orchestra.run.smithi055.stdout:[client.0] 2024-05-20T22:10:29.516 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB2yUtm592RAxAAiaDYBHgkpVfsQHm+jLOy3g== 2024-05-20T22:10:29.517 INFO:teuthology.orchestra.run.smithi055.stdout: caps mds = "allow" 2024-05-20T22:10:29.517 INFO:teuthology.orchestra.run.smithi055.stdout: caps mon = "allow r" 2024-05-20T22:10:29.517 INFO:teuthology.orchestra.run.smithi055.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-20T22:10:29.517 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.518+0000 7fd065f45640 1 -- 172.21.15.55:0/2949576695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd04c06be30 msgr2=0x7fd04c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.518+0000 7fd065f45640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd04c06be30 0x7fd04c06e340 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd050034000 tx=0x7fd05002e000 comp rx=0 tx=0).stop 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.518+0000 7fd065f45640 1 -- 172.21.15.55:0/2949576695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0600b7fb0 msgr2=0x7fd0600c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.518+0000 7fd065f45640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0600b7fb0 0x7fd0600c8590 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd05400e870 tx=0x7fd05400ed30 comp rx=0 tx=0).stop 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.519+0000 7fd065f45640 1 -- 172.21.15.55:0/2949576695 shutdown_connections 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.519+0000 7fd065f45640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd04c06be30 0x7fd04c06e340 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.519+0000 7fd065f45640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0600cf680 0x7fd060161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.519+0000 7fd065f45640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0600b7fb0 0x7fd0600c8590 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.519+0000 7fd065f45640 1 --2- 172.21.15.55:0/2949576695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd06006c460 0x7fd0601f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.519+0000 7fd065f45640 1 -- 172.21.15.55:0/2949576695 >> 172.21.15.55:0/2949576695 conn(0x7fd0600d5450 msgr2=0x7fd0600b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:29.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.519+0000 7fd065f45640 1 -- 172.21.15.55:0/2949576695 shutdown_connections 2024-05-20T22:10:29.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.519+0000 7fd065f45640 1 -- 172.21.15.55:0/2949576695 wait complete. 2024-05-20T22:10:29.531 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:29.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.645+0000 7f35f6fa3640 1 Processor -- start 2024-05-20T22:10:29.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.645+0000 7f35f6fa3640 1 -- start start 2024-05-20T22:10:29.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.646+0000 7f35f6fa3640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f35f00b5830 0x7f35f00b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.646+0000 7f35f6fa3640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35f015da90 con 0x7f35f00d2e10 2024-05-20T22:10:29.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.646+0000 7f35f6fa3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35f00b5d80 con 0x7f35f00cd840 2024-05-20T22:10:29.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.646+0000 7f35f6fa3640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f35f00b6990 con 0x7f35f00b5830 2024-05-20T22:10:29.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.646+0000 7f35e7fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f35f00b5830 0x7f35f00b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.646+0000 7f35f5540640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f35f00d2e10 0x7f35f00b6e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59700/0 (socket says 172.21.15.55:59700) 2024-05-20T22:10:29.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.646+0000 7f35f5540640 1 -- 172.21.15.55:0/1044582318 learned_addr learned my addr 172.21.15.55:0/1044582318 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:29.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.647+0000 7f35e7fff640 1 -- 172.21.15.55:0/1044582318 >> v1:172.21.15.138:6789/0 conn(0x7f35f00cd840 legacy=0x7f35f00d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.647+0000 7f35e7fff640 1 -- 172.21.15.55:0/1044582318 >> v1:172.21.15.55:6789/0 conn(0x7f35f00d2e10 legacy=0x7f35f00b6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.647+0000 7f35e7fff640 1 -- 172.21.15.55:0/1044582318 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f00b6060 con 0x7f35f00b5830 2024-05-20T22:10:29.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.647+0000 7f35e7fff640 1 --2- 172.21.15.55:0/1044582318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00b5830 0x7f35f00b1fa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f35f00b6b10 tx=0x7f35e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba98bf921a32e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:29.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.648+0000 7f35e77fe640 1 -- 172.21.15.55:0/1044582318 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0037070 con 0x7f35f00b5830 2024-05-20T22:10:29.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.648+0000 7f35e77fe640 1 -- 172.21.15.55:0/1044582318 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e0032440 con 0x7f35f00b5830 2024-05-20T22:10:29.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.648+0000 7f35e77fe640 1 -- 172.21.15.55:0/1044582318 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0032810 con 0x7f35f00b5830 2024-05-20T22:10:29.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.648+0000 7f35f6fa3640 1 -- 172.21.15.55:0/1044582318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00b5830 msgr2=0x7f35f00b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.648+0000 7f35f6fa3640 1 --2- 172.21.15.55:0/1044582318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00b5830 0x7f35f00b1fa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f35f00b6b10 tx=0x7f35e002a180 comp rx=0 tx=0).stop 2024-05-20T22:10:29.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.648+0000 7f35f6fa3640 1 -- 172.21.15.55:0/1044582318 shutdown_connections 2024-05-20T22:10:29.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.648+0000 7f35f6fa3640 1 --2- 172.21.15.55:0/1044582318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00b5830 0x7f35f00b1fa0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.648+0000 7f35f6fa3640 1 -- 172.21.15.55:0/1044582318 >> 172.21.15.55:0/1044582318 conn(0x7f35f008bff0 msgr2=0x7f35f008c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:29.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.649+0000 7f35f6fa3640 1 -- 172.21.15.55:0/1044582318 shutdown_connections 2024-05-20T22:10:29.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.649+0000 7f35f6fa3640 1 -- 172.21.15.55:0/1044582318 wait complete. 2024-05-20T22:10:29.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.650+0000 7f35f6fa3640 1 Processor -- start 2024-05-20T22:10:29.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.650+0000 7f35f6fa3640 1 -- start start 2024-05-20T22:10:29.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f6fa3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35f00b5830 0x7f35f0160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f6fa3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f00cd840 0x7f35f00cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f6fa3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00d2e10 0x7f35f01f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f6fa3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f35f00b8160 con 0x7f35f00d2e10 2024-05-20T22:10:29.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f6fa3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35f00b6620 con 0x7f35f00cd840 2024-05-20T22:10:29.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f6fa3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f35f00d1890 con 0x7f35f00b5830 2024-05-20T22:10:29.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f5540640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00d2e10 0x7f35f01f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f5540640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00d2e10 0x7f35f01f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57318/0 (socket says 172.21.15.55:57318) 2024-05-20T22:10:29.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f5540640 1 -- 172.21.15.55:0/801333849 learned_addr learned my addr 172.21.15.55:0/801333849 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:29.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35e7fff640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f00cd840 0x7f35f00cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f4d3f640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35f00b5830 0x7f35f0160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f5540640 1 -- 172.21.15.55:0/801333849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35f00b5830 msgr2=0x7f35f0160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f5540640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35f00b5830 0x7f35f0160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f5540640 1 -- 172.21.15.55:0/801333849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f00cd840 msgr2=0x7f35f00cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f5540640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f00cd840 0x7f35f00cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.652+0000 7f35f5540640 1 -- 172.21.15.55:0/801333849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f0157f30 con 0x7f35f00d2e10 2024-05-20T22:10:29.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.653+0000 7f35f5540640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00d2e10 0x7f35f01f7060 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f35e800cab0 tx=0x7f35e800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:29.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.654+0000 7f35e57fa640 1 -- 172.21.15.55:0/801333849 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8017070 con 0x7f35f00d2e10 2024-05-20T22:10:29.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.654+0000 7f35e57fa640 1 -- 172.21.15.55:0/801333849 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e800ac30 con 0x7f35f00d2e10 2024-05-20T22:10:29.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.654+0000 7f35f6fa3640 1 -- 172.21.15.55:0/801333849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f0158210 con 0x7f35f00d2e10 2024-05-20T22:10:29.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.654+0000 7f35e57fa640 1 -- 172.21.15.55:0/801333849 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8012400 con 0x7f35f00d2e10 2024-05-20T22:10:29.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.654+0000 7f35f6fa3640 1 -- 172.21.15.55:0/801333849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01587d0 con 0x7f35f00d2e10 2024-05-20T22:10:29.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.655+0000 7f35f6fa3640 1 -- 172.21.15.55:0/801333849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f0069090 con 0x7f35f00d2e10 2024-05-20T22:10:29.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.656+0000 7f35e57fa640 1 -- 172.21.15.55:0/801333849 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35e801a070 con 0x7f35f00d2e10 2024-05-20T22:10:29.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.656+0000 7f35e57fa640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f35dc06be60 0x7f35dc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:29.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.656+0000 7f35e57fa640 1 -- 172.21.15.55:0/801333849 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f35e808abc0 con 0x7f35f00d2e10 2024-05-20T22:10:29.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.656+0000 7f35f4d3f640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f35dc06be60 0x7f35dc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:29.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.657+0000 7f35f4d3f640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f35dc06be60 0x7f35dc06e370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f35ec007fd0 tx=0x7f35ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:29.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.661+0000 7f35e57fa640 1 -- 172.21.15.55:0/801333849 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f35e8058d90 con 0x7f35f00d2e10 2024-05-20T22:10:29.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.883+0000 7f35f6fa3640 1 -- 172.21.15.55:0/801333849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35f0067920 con 0x7f35f00d2e10 2024-05-20T22:10:29.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.884+0000 7f35e57fa640 1 -- 172.21.15.55:0/801333849 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5192 (secure 0 0 0) 0x7f35e805d090 con 0x7f35f00d2e10 2024-05-20T22:10:29.885 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:29.885 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":35,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2682361422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2682361422},{"type":"v1","addr":"172.21.15.138:6834","nonce":2682361422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5647,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1792037486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1792037486},{"type":"v1","addr":"172.21.15.138:6835","nonce":1792037486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6836/3650250553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":3650250553},{"type":"v1","addr":"172.21.15.55:6836","nonce":3650250553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:27.016303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":35231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35231":{"gid":35231,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6837/2445634740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6835","nonce":2445634740},{"type":"v1","addr":"172.21.15.55:6837","nonce":2445634740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":35231,"qdb_cluster":[35231]},"id":3}]} 2024-05-20T22:10:29.885 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 35 2024-05-20T22:10:29.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 -- 172.21.15.55:0/801333849 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f35dc06be60 msgr2=0x7f35dc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f35dc06be60 0x7f35dc06e370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f35ec007fd0 tx=0x7f35ec009040 comp rx=0 tx=0).stop 2024-05-20T22:10:29.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 -- 172.21.15.55:0/801333849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00d2e10 msgr2=0x7f35f01f7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:29.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00d2e10 0x7f35f01f7060 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f35e800cab0 tx=0x7f35e800cf70 comp rx=0 tx=0).stop 2024-05-20T22:10:29.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 -- 172.21.15.55:0/801333849 shutdown_connections 2024-05-20T22:10:29.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f35dc06be60 0x7f35dc06e370 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f00d2e10 0x7f35f01f7060 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f00cd840 0x7f35f00cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 --2- 172.21.15.55:0/801333849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35f00b5830 0x7f35f0160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:29.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 -- 172.21.15.55:0/801333849 >> 172.21.15.55:0/801333849 conn(0x7f35f008bff0 msgr2=0x7f35f00719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:29.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 -- 172.21.15.55:0/801333849 shutdown_connections 2024-05-20T22:10:29.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:29.886+0000 7f35f6fa3640 1 -- 172.21.15.55:0/801333849 wait complete. 2024-05-20T22:10:30.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:10:23.995145+0000', 'modified': '2024-05-20T22:10:27.016303+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35231}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35231': {'gid': 35231, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6837/2445634740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6835', 'nonce': 2445634740}, {'type': 'v1', 'addr': '172.21.15.55:6837', 'nonce': 2445634740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 35231, 'qdb_cluster': [35231]} 2024-05-20T22:10:30.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T22:10:30.301 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-20T22:10:30.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.422+0000 7f45aec26640 1 Processor -- start 2024-05-20T22:10:30.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.422+0000 7f45aec26640 1 -- start start 2024-05-20T22:10:30.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.423+0000 7f45aec26640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45a80cf270 0x7f45a80cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.423+0000 7f45aec26640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45a808cd70 con 0x7f45a808c960 2024-05-20T22:10:30.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.423+0000 7f45aec26640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45a80cfbc0 con 0x7f45a80cd520 2024-05-20T22:10:30.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.423+0000 7f45aec26640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f45a80d1020 con 0x7f45a80cf270 2024-05-20T22:10:30.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.423+0000 7f45a7fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f45a808c960 0x7f45a80b19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59702/0 (socket says 172.21.15.55:59702) 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.423+0000 7f45a7fff640 1 -- 172.21.15.55:0/2265073596 learned_addr learned my addr 172.21.15.55:0/2265073596 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3965365381 0 0) 0x7f4594001110 con 0x7f45a808c960 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/2265073596 >> v2:172.21.15.138:3300/0 conn(0x7f45a80cf270 0x7f45a80cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4598003520 con 0x7f45a808c960 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3420806989 0 0) 0x7f4598003520 con 0x7f45a808c960 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 >> v2:172.21.15.138:3300/0 conn(0x7f45a80cf270 msgr2=0x7f45a80cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 --2- 172.21.15.55:0/2265073596 >> v2:172.21.15.138:3300/0 conn(0x7f45a80cf270 0x7f45a80cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 >> v1:172.21.15.138:6789/0 conn(0x7f45a80cd520 legacy=0x7f45a80d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a80d1300 con 0x7f45a808c960 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f4594001840 con 0x7f45a808c960 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 >> v1:172.21.15.55:6789/0 conn(0x7f45a808c960 legacy=0x7f45a80b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45980039b0 0x7f4598003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4598004800 0x7f4598004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4598005250 0x7f459800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f45a80d1020 con 0x7f4598005250 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f459800c1e0 con 0x7f4598004800 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f459800c3e0 con 0x7f45980039b0 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4598005250 0x7f459800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45a7fff640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4598004800 0x7f4598004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45980039b0 0x7f4598003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45ad1c3640 1 -- 172.21.15.55:0/2265073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45980039b0 msgr2=0x7f4598003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45980039b0 0x7f4598003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45ad1c3640 1 -- 172.21.15.55:0/2265073596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4598004800 msgr2=0x7f4598004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4598004800 0x7f4598004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.424+0000 7f45ad1c3640 1 -- 172.21.15.55:0/2265073596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4598003520 con 0x7f4598005250 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.425+0000 7f45a7fff640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4598004800 0x7f4598004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.425+0000 7f45ad1c3640 1 -- 172.21.15.55:0/2265073596 reap_dead start 2024-05-20T22:10:30.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.425+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4598005250 0x7f459800b9c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009f90 tx=0x7f4598005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc903c22d0916fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:30.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.425+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0035020 con 0x7f4598005250 2024-05-20T22:10:30.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.425+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a0003b90 con 0x7f4598005250 2024-05-20T22:10:30.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.425+0000 7f45a77fe640 1 -- 172.21.15.55:0/2265073596 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0009c10 con 0x7f4598005250 2024-05-20T22:10:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.426+0000 7f45aec26640 1 -- 172.21.15.55:0/2265073596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4598005250 msgr2=0x7f459800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.426+0000 7f45aec26640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4598005250 0x7f459800b9c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009f90 tx=0x7f4598005830 comp rx=0 tx=0).stop 2024-05-20T22:10:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.426+0000 7f45aec26640 1 -- 172.21.15.55:0/2265073596 shutdown_connections 2024-05-20T22:10:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.426+0000 7f45aec26640 1 --2- 172.21.15.55:0/2265073596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4598005250 0x7f459800b9c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.426+0000 7f45aec26640 1 -- 172.21.15.55:0/2265073596 >> 172.21.15.55:0/2265073596 conn(0x7f45a80d2860 msgr2=0x7f45a80d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.426+0000 7f45aec26640 1 -- 172.21.15.55:0/2265073596 shutdown_connections 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.426+0000 7f45aec26640 1 -- 172.21.15.55:0/2265073596 wait complete. 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.428+0000 7f45aec26640 1 Processor -- start 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.428+0000 7f45aec26640 1 -- start start 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.429+0000 7f45aec26640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4598005250 0x7f45a814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.429+0000 7f45aec26640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806e910 0x7f45a81f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.429+0000 7f45aec26640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815b590 0x7f45a815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.429+0000 7f45aec26640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f45a80b2cb0 con 0x7f45a815b590 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.429+0000 7f45aec26640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45a80d18c0 con 0x7f4598005250 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.429+0000 7f45aec26640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f45a80cf680 con 0x7f45a806e910 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45ad1c3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815b590 0x7f45a815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45ad1c3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815b590 0x7f45a815b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57338/0 (socket says 172.21.15.55:57338) 2024-05-20T22:10:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45ad1c3640 1 -- 172.21.15.55:0/1551702996 learned_addr learned my addr 172.21.15.55:0/1551702996 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45a7fff640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806e910 0x7f45a81f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4598005250 0x7f45a814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45ad1c3640 1 -- 172.21.15.55:0/1551702996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806e910 msgr2=0x7f45a81f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806e910 0x7f45a81f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45ad1c3640 1 -- 172.21.15.55:0/1551702996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4598005250 msgr2=0x7f45a814d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4598005250 0x7f45a814d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45ad1c3640 1 -- 172.21.15.55:0/1551702996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a815bee0 con 0x7f45a815b590 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815b590 0x7f45a815b9a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f45a000e230 tx=0x7f45a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45a57fa640 1 -- 172.21.15.55:0/1551702996 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0035030 con 0x7f45a815b590 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45a57fa640 1 -- 172.21.15.55:0/1551702996 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a0042070 con 0x7f45a815b590 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45aec26640 1 -- 172.21.15.55:0/1551702996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a814a530 con 0x7f45a815b590 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45a57fa640 1 -- 172.21.15.55:0/1551702996 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0009c10 con 0x7f45a815b590 2024-05-20T22:10:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.430+0000 7f45aec26640 1 -- 172.21.15.55:0/1551702996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a814aaf0 con 0x7f45a815b590 2024-05-20T22:10:30.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.432+0000 7f45a57fa640 1 -- 172.21.15.55:0/1551702996 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45a0005d10 con 0x7f45a815b590 2024-05-20T22:10:30.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.432+0000 7f45a57fa640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f458c06be60 0x7f458c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.432+0000 7f45a57fa640 1 -- 172.21.15.55:0/1551702996 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f45a00aba40 con 0x7f45a815b590 2024-05-20T22:10:30.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.433+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f458c06be60 0x7f458c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.433+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f458c06be60 0x7f458c06e370 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f459c007fd0 tx=0x7f459c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:30.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.434+0000 7f45b5baa740 1 Processor -- start 2024-05-20T22:10:30.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.434+0000 7f45b5baa740 1 -- start start 2024-05-20T22:10:30.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.435+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc7f270 0x56448c04a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.435+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc563b0 0x56448c0470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.435+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc5e590 0x56448c020360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.435+0000 7f45b5baa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56448bf834d0 con 0x56448bc5e590 2024-05-20T22:10:30.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.435+0000 7f45b5baa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56448bf2b3e0 con 0x56448bc563b0 2024-05-20T22:10:30.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.435+0000 7f45b5baa740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56448bf7f490 con 0x56448bc7f270 2024-05-20T22:10:30.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.435+0000 7f45ad1c3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc5e590 0x56448c020360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.435+0000 7f45ad1c3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc5e590 0x56448c020360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57348/0 (socket says 172.21.15.55:57348) 2024-05-20T22:10:30.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.435+0000 7f45ad1c3640 1 -- 172.21.15.55:0/896783101 learned_addr learned my addr 172.21.15.55:0/896783101 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45a7fff640 1 --2- 172.21.15.55:0/896783101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc563b0 0x56448c0470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/896783101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc7f270 0x56448c04a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45ad1c3640 1 -- 172.21.15.55:0/896783101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc7f270 msgr2=0x56448c04a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/896783101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc7f270 0x56448c04a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45ad1c3640 1 -- 172.21.15.55:0/896783101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc563b0 msgr2=0x56448c0470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/896783101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc563b0 0x56448c0470a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45ad1c3640 1 -- 172.21.15.55:0/896783101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448bc972d0 con 0x56448bc5e590 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/896783101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc5e590 0x56448c020360 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f45a0079b00 tx=0x7f45a0079b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc1ff532bbea87c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45927fc640 1 -- 172.21.15.55:0/896783101 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0077c10 con 0x56448bc5e590 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45927fc640 1 -- 172.21.15.55:0/896783101 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a0077e70 con 0x56448bc5e590 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.436+0000 7f45927fc640 1 -- 172.21.15.55:0/896783101 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0082210 con 0x56448bc5e590 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.437+0000 7f45b5baa740 1 -- 172.21.15.55:0/896783101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc5e590 msgr2=0x56448c020360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.437+0000 7f45b5baa740 1 --2- 172.21.15.55:0/896783101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc5e590 0x56448c020360 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f45a0079b00 tx=0x7f45a0079b70 comp rx=0 tx=0).stop 2024-05-20T22:10:30.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.437+0000 7f45b5baa740 1 -- 172.21.15.55:0/896783101 shutdown_connections 2024-05-20T22:10:30.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.437+0000 7f45b5baa740 1 --2- 172.21.15.55:0/896783101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc5e590 0x56448c020360 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.437+0000 7f45b5baa740 1 --2- 172.21.15.55:0/896783101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc563b0 0x56448c0470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.437+0000 7f45b5baa740 1 --2- 172.21.15.55:0/896783101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc7f270 0x56448c04a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.437+0000 7f45b5baa740 1 -- 172.21.15.55:0/896783101 >> 172.21.15.55:0/896783101 conn(0x56448bc733c0 msgr2=0x56448c01faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:30.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.437+0000 7f45b5baa740 1 -- 172.21.15.55:0/896783101 shutdown_connections 2024-05-20T22:10:30.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.437+0000 7f45b5baa740 1 -- 172.21.15.55:0/896783101 wait complete. 2024-05-20T22:10:30.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.438+0000 7f45b5baa740 1 -- start start 2024-05-20T22:10:30.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.438+0000 7f45b5baa740 1 Processor -- start 2024-05-20T22:10:30.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc5e590 0x56448c06b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 0x56448c04ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45b5baa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56448bf2b3e0 con 0x56448bc563b0 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45b5baa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56448bf3e8e0 con 0x56448bc7f270 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45b5baa740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56448bf834d0 con 0x56448bc5e590 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45b5baa740 10 client.0 _ll_register_callbacks cb 0x56448bf0b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45ac9c2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45ac9c2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57352/0 (socket says 172.21.15.55:57352) 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45ac9c2640 1 -- 172.21.15.55:0/841151131 learned_addr learned my addr 172.21.15.55:0/841151131 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 0x56448c04ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.439+0000 7f45a7fff640 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc5e590 0x56448c06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45ac9c2640 1 -- 172.21.15.55:0/841151131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc5e590 msgr2=0x56448c06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc5e590 0x56448c06b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45ac9c2640 1 -- 172.21.15.55:0/841151131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 msgr2=0x56448c04ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 0x56448c04ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45ac9c2640 1 -- 172.21.15.55:0/841151131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56448bd616b0 con 0x56448bc563b0 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0470b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f459c012010 tx=0x7f459c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4139b68d631a3542 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45877fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45877fe640 1 -- 172.21.15.55:0/841151131 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c025020 con 0x56448bc563b0 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45877fe640 1 -- 172.21.15.55:0/841151131 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f459c021630 con 0x56448bc563b0 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45b5baa740 1 -- 172.21.15.55:0/841151131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a0077e70 con 0x56448bc563b0 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45877fe640 1 -- 172.21.15.55:0/841151131 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f459c02f050 con 0x56448bc563b0 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45b5baa740 10 client.6138 fetch_fsmap learned FSMap version 35 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45b5baa740 1 -- 172.21.15.55:0/841151131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x56448c04b520 con 0x56448bc563b0 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.440+0000 7f45877fe640 1 -- 172.21.15.55:0/841151131 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a0077e70 con 0x56448bc563b0 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.441+0000 7f45877fe640 1 -- 172.21.15.55:0/841151131 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f459c032020 con 0x56448bc563b0 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.441+0000 7f45b5baa740 10 client.6138 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T22:10:30.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.441+0000 7f45b5baa740 10 client.6138 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.441+0000 7f45b5baa740 20 client.6138 populate_metadata read hostname 'smithi055' 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.441+0000 7f45b5baa740 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x56448bc3de40 0x56448c068860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.441+0000 7f45b5baa740 4 client.6138 mds_command: new command op to 35231 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.441+0000 7f45b5baa740 1 -- 172.21.15.55:0/841151131 --> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56448bf90c00 con 0x56448bc3de40 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.441+0000 7f45a7fff640 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x56448bc3de40 0x56448c068860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.441+0000 7f45877fe640 10 client.6138 ms_handle_connect on v2:172.21.15.55:6835/2445634740 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.441+0000 7f45a7fff640 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x56448bc3de40 0x56448c068860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88a556221b1e70d server_cookie=b81e77d75024b4d8 in_seq=0 out_seq=0 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45877fe640 1 -- 172.21.15.55:0/841151131 <== mds.0 v2:172.21.15.55:6835/2445634740 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56448bf90c00 con 0x56448bc3de40 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45877fe640 10 client.6138 handle_command_reply: tid=0 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 client.6138 shutdown 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 -- 172.21.15.55:0/841151131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 msgr2=0x56448c0470b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0470b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f459c012010 tx=0x7f459c001120 comp rx=0 tx=0).stop 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 -- 172.21.15.55:0/841151131 shutdown_connections 2024-05-20T22:10:30.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x56448bc3de40 0x56448c068860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 0x56448c04ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc5e590 0x56448c06b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45877fe640 0 client.6138 ms_handle_reset on v2:172.21.15.55:6835/2445634740 2024-05-20T22:10:30.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 --2- 172.21.15.55:0/841151131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0470b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 -- 172.21.15.55:0/841151131 >> 172.21.15.55:0/841151131 conn(0x56448bc733c0 msgr2=0x56448c019e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:30.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 -- 172.21.15.55:0/841151131 shutdown_connections 2024-05-20T22:10:30.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 1 -- 172.21.15.55:0/841151131 wait complete. 2024-05-20T22:10:30.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.442+0000 7f45b5baa740 20 client.6138 trim_cache size 0 max 16384 2024-05-20T22:10:30.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.461+0000 7f45b5baa740 1 Processor -- start 2024-05-20T22:10:30.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.461+0000 7f45b5baa740 1 -- start start 2024-05-20T22:10:30.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc5e590 0x56448c046610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc7f270 0x56448c05ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45b5baa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8201970 con 0x56448bc563b0 2024-05-20T22:10:30.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45b5baa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8200cd0 con 0x56448bc5e590 2024-05-20T22:10:30.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45b5baa740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f45a8201320 con 0x56448bc7f270 2024-05-20T22:10:30.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45ac9c2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45ac9c2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57364/0 (socket says 172.21.15.55:57364) 2024-05-20T22:10:30.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45ac9c2640 1 -- 172.21.15.55:0/90866609 learned_addr learned my addr 172.21.15.55:0/90866609 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45ac9c2640 1 -- 172.21.15.55:0/90866609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc7f270 msgr2=0x56448c05ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/90866609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc7f270 0x56448c05ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45ac9c2640 1 -- 172.21.15.55:0/90866609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc5e590 msgr2=0x56448c046610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45a7fff640 1 --2- 172.21.15.55:0/90866609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc5e590 0x56448c046610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/90866609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc5e590 0x56448c046610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45ac9c2640 1 -- 172.21.15.55:0/90866609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448c06b980 con 0x56448bc563b0 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/90866609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0460d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f459c011db0 tx=0x7f459c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42e83d1a2e9b931 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f4591ffb640 1 -- 172.21.15.55:0/90866609 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c038070 con 0x56448bc563b0 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f4591ffb640 1 -- 172.21.15.55:0/90866609 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f459c021850 con 0x56448bc563b0 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.463+0000 7f4591ffb640 1 -- 172.21.15.55:0/90866609 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c025070 con 0x56448bc563b0 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.464+0000 7f45b5baa740 1 -- 172.21.15.55:0/90866609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 msgr2=0x56448c0460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.464+0000 7f45b5baa740 1 --2- 172.21.15.55:0/90866609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0460d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f459c011db0 tx=0x7f459c018000 comp rx=0 tx=0).stop 2024-05-20T22:10:30.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.464+0000 7f45b5baa740 1 -- 172.21.15.55:0/90866609 shutdown_connections 2024-05-20T22:10:30.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.464+0000 7f45b5baa740 1 --2- 172.21.15.55:0/90866609 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc7f270 0x56448c05ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.464+0000 7f45b5baa740 1 --2- 172.21.15.55:0/90866609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc5e590 0x56448c046610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.464+0000 7f45b5baa740 1 --2- 172.21.15.55:0/90866609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c0460d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.464+0000 7f45b5baa740 1 -- 172.21.15.55:0/90866609 >> 172.21.15.55:0/90866609 conn(0x56448bc733c0 msgr2=0x56448c06c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:30.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.464+0000 7f45b5baa740 1 -- 172.21.15.55:0/90866609 shutdown_connections 2024-05-20T22:10:30.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.464+0000 7f45b5baa740 1 -- 172.21.15.55:0/90866609 wait complete. 2024-05-20T22:10:30.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.465+0000 7f45b5baa740 1 -- start start 2024-05-20T22:10:30.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.465+0000 7f45b5baa740 1 Processor -- start 2024-05-20T22:10:30.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c019fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc5e590 0x56448c0e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45b5baa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 0x56448c087d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45b5baa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56448c04d680 con 0x56448bc563b0 2024-05-20T22:10:30.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45b5baa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56448c06bcb0 con 0x56448bc7f270 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45b5baa740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56448c0677a0 con 0x56448bc5e590 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45b5baa740 10 client.0 _ll_register_callbacks cb 0x7f459c001470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ac9c2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c019fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ac9c2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c019fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57376/0 (socket says 172.21.15.55:57376) 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ac9c2640 1 -- 172.21.15.55:0/3895882983 learned_addr learned my addr 172.21.15.55:0/3895882983 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45a7fff640 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc5e590 0x56448c0e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 0x56448c087d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ac9c2640 1 -- 172.21.15.55:0/3895882983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc5e590 msgr2=0x56448c0e16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc5e590 0x56448c0e16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ac9c2640 1 -- 172.21.15.55:0/3895882983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 msgr2=0x56448c087d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 0x56448c087d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ac9c2640 1 -- 172.21.15.55:0/3895882983 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56448c04bbc0 con 0x56448bc563b0 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ad1c3640 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 0x56448c087d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:30.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45ac9c2640 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c019fa0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f459c024040 tx=0x7f459c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc18323d80b77087 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45877fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45877fe640 1 -- 172.21.15.55:0/3895882983 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c038040 con 0x56448bc563b0 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45877fe640 1 -- 172.21.15.55:0/3895882983 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f459c021630 con 0x56448bc563b0 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.467+0000 7f45b5baa740 1 -- 172.21.15.55:0/3895882983 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f459c021850 con 0x56448bc563b0 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45877fe640 1 -- 172.21.15.55:0/3895882983 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f459c04b050 con 0x56448bc563b0 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45b5baa740 10 client.6144 fetch_fsmap learned FSMap version 35 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45b5baa740 1 -- 172.21.15.55:0/3895882983 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x56448c066f80 con 0x56448bc563b0 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45877fe640 1 -- 172.21.15.55:0/3895882983 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f459c021850 con 0x56448bc563b0 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45877fe640 1 -- 172.21.15.55:0/3895882983 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f459c04e020 con 0x56448bc563b0 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45b5baa740 10 client.6144 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45b5baa740 10 client.6144 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45b5baa740 20 client.6144 populate_metadata read hostname 'smithi055' 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45b5baa740 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x56448c04c450 0x56448c04c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45b5baa740 4 client.6144 mds_command: new command op to 35231 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45b5baa740 1 -- 172.21.15.55:0/3895882983 --> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] -- command(tid 0: {"prefix": "status"}) -- 0x56448bf8af50 con 0x56448c04c450 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45a7fff640 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x56448c04c450 0x56448c04c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45877fe640 10 client.6144 ms_handle_connect on v2:172.21.15.55:6835/2445634740 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.468+0000 7f45a7fff640 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x56448c04c450 0x56448c04c860 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c8aa99af4f8f92 server_cookie=e841f93d65a044a7 in_seq=0 out_seq=0 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45877fe640 1 -- 172.21.15.55:0/3895882983 <== mds.0 v2:172.21.15.55:6835/2445634740 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56448bf8af50 con 0x56448c04c450 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45877fe640 10 client.6144 handle_command_reply: tid=0 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 client.6144 shutdown 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 -- 172.21.15.55:0/3895882983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 msgr2=0x56448c019fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c019fa0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f459c024040 tx=0x7f459c017000 comp rx=0 tx=0).stop 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 -- 172.21.15.55:0/3895882983 shutdown_connections 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x56448c04c450 0x56448c04c860 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56448bc7f270 0x56448c087d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45877fe640 0 client.6144 ms_handle_reset on v2:172.21.15.55:6835/2445634740 2024-05-20T22:10:30.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56448bc5e590 0x56448c0e16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 --2- 172.21.15.55:0/3895882983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56448bc563b0 0x56448c019fa0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 -- 172.21.15.55:0/3895882983 >> 172.21.15.55:0/3895882983 conn(0x56448bc733c0 msgr2=0x56448c015940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:30.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 -- 172.21.15.55:0/3895882983 shutdown_connections 2024-05-20T22:10:30.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 1 -- 172.21.15.55:0/3895882983 wait complete. 2024-05-20T22:10:30.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.469+0000 7f45b5baa740 20 client.6144 trim_cache size 0 max 16384 2024-05-20T22:10:30.475 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:10:30.475 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "de8ca569-3fd1-4cb2-8bdc-99f367745927", 2024-05-20T22:10:30.475 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-20T22:10:30.475 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 35231, 2024-05-20T22:10:30.476 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-20T22:10:30.476 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-20T22:10:30.476 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-20T22:10:30.476 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 5.8425612549999997, 2024-05-20T22:10:30.476 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 35, 2024-05-20T22:10:30.476 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 55, 2024-05-20T22:10:30.476 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 55, 2024-05-20T22:10:30.476 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 10.814849540000001 2024-05-20T22:10:30.476 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:10:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 -- 172.21.15.55:0/1551702996 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f458c06be60 msgr2=0x7f458c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f458c06be60 0x7f458c06e370 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f459c007fd0 tx=0x7f459c009040 comp rx=0 tx=0).stop 2024-05-20T22:10:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 -- 172.21.15.55:0/1551702996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815b590 msgr2=0x7f45a815b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815b590 0x7f45a815b9a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f45a000e230 tx=0x7f45a0009750 comp rx=0 tx=0).stop 2024-05-20T22:10:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 -- 172.21.15.55:0/1551702996 shutdown_connections 2024-05-20T22:10:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f458c06be60 0x7f458c06e370 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45a815b590 0x7f45a815b9a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45a806e910 0x7f45a81f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 --2- 172.21.15.55:0/1551702996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4598005250 0x7f45a814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 -- 172.21.15.55:0/1551702996 >> 172.21.15.55:0/1551702996 conn(0x7f45a80d2860 msgr2=0x7f45a8071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:30.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 -- 172.21.15.55:0/1551702996 shutdown_connections 2024-05-20T22:10:30.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.470+0000 7f45aec26640 1 -- 172.21.15.55:0/1551702996 wait complete. 2024-05-20T22:10:30.482 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T22:10:30.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.596+0000 7f4d4c63d640 1 Processor -- start 2024-05-20T22:10:30.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.596+0000 7f4d4c63d640 1 -- start start 2024-05-20T22:10:30.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d4c63d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d440b3b30 0x7f4d440b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d4c63d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4d4415f100 0x7f4d4406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d4c63d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d440d3dd0 con 0x7f4d4406c5c0 2024-05-20T22:10:30.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d4c63d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f4d440d40b0 con 0x7f4d4415f100 2024-05-20T22:10:30.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d4c63d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4d440d4390 con 0x7f4d440b3b30 2024-05-20T22:10:30.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d49bd8640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4d4415f100 0x7f4d4406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d4abda640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4d4406c5c0 0x7f4d44162b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59706/0 (socket says 172.21.15.55:59706) 2024-05-20T22:10:30.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d49bd8640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4d4415f100 0x7f4d4406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57382/0 (socket says 172.21.15.55:57382) 2024-05-20T22:10:30.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d4abda640 1 -- 172.21.15.55:0/3147279032 learned_addr learned my addr 172.21.15.55:0/3147279032 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:30.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d4a3d9640 1 --2- 172.21.15.55:0/3147279032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d440b3b30 0x7f4d440b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d49bd8640 1 -- 172.21.15.55:0/3147279032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d440b3b30 msgr2=0x7f4d440b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:30.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d49bd8640 1 --2- 172.21.15.55:0/3147279032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d440b3b30 0x7f4d440b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d49bd8640 1 -- 172.21.15.55:0/3147279032 >> v1:172.21.15.55:6789/0 conn(0x7f4d4406c5c0 legacy=0x7f4d44162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d49bd8640 1 -- 172.21.15.55:0/3147279032 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4406b5a0 con 0x7f4d4415f100 2024-05-20T22:10:30.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.598+0000 7f4d493d7640 1 -- 172.21.15.55:0/3147279032 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 363987232 0 0) 0x7f4d440d3dd0 con 0x7f4d4406c5c0 2024-05-20T22:10:30.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.599+0000 7f4d49bd8640 1 --2- 172.21.15.55:0/3147279032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 0x7f4d4406c080 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4d340097a0 tx=0x7f4d3402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f9224aaa920dfd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:30.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.599+0000 7f4d493d7640 1 -- 172.21.15.55:0/3147279032 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34037070 con 0x7f4d4415f100 2024-05-20T22:10:30.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.599+0000 7f4d493d7640 1 -- 172.21.15.55:0/3147279032 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d34032440 con 0x7f4d4415f100 2024-05-20T22:10:30.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.599+0000 7f4d493d7640 1 -- 172.21.15.55:0/3147279032 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34032810 con 0x7f4d4415f100 2024-05-20T22:10:30.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.600+0000 7f4d4c63d640 1 -- 172.21.15.55:0/3147279032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 msgr2=0x7f4d4406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.600+0000 7f4d4c63d640 1 --2- 172.21.15.55:0/3147279032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 0x7f4d4406c080 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4d340097a0 tx=0x7f4d3402a180 comp rx=0 tx=0).stop 2024-05-20T22:10:30.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.600+0000 7f4d4c63d640 1 -- 172.21.15.55:0/3147279032 shutdown_connections 2024-05-20T22:10:30.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.600+0000 7f4d4c63d640 1 --2- 172.21.15.55:0/3147279032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 0x7f4d4406c080 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.600+0000 7f4d4c63d640 1 --2- 172.21.15.55:0/3147279032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d440b3b30 0x7f4d440b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.600+0000 7f4d4c63d640 1 -- 172.21.15.55:0/3147279032 >> 172.21.15.55:0/3147279032 conn(0x7f4d440d51d0 msgr2=0x7f4d440b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:30.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.600+0000 7f4d4c63d640 1 -- 172.21.15.55:0/3147279032 shutdown_connections 2024-05-20T22:10:30.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.600+0000 7f4d4c63d640 1 -- 172.21.15.55:0/3147279032 wait complete. 2024-05-20T22:10:30.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.602+0000 7f4d4c63d640 1 Processor -- start 2024-05-20T22:10:30.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.602+0000 7f4d4c63d640 1 -- start start 2024-05-20T22:10:30.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.603+0000 7f4d4c63d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d4406c5c0 0x7f4d44154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.603+0000 7f4d4c63d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d440b3b30 0x7f4d441f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.603+0000 7f4d4c63d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 0x7f4d441562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.603+0000 7f4d4c63d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4406c9d0 con 0x7f4d4415f100 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.603+0000 7f4d4c63d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4406bb60 con 0x7f4d4406c5c0 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.603+0000 7f4d4c63d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4406cb50 con 0x7f4d440b3b30 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.603+0000 7f4d4abda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 0x7f4d441562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.603+0000 7f4d4abda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 0x7f4d441562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57398/0 (socket says 172.21.15.55:57398) 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.603+0000 7f4d4abda640 1 -- 172.21.15.55:0/383007271 learned_addr learned my addr 172.21.15.55:0/383007271 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d4a3d9640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d4406c5c0 0x7f4d44154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d49bd8640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d440b3b30 0x7f4d441f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d4abda640 1 -- 172.21.15.55:0/383007271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d440b3b30 msgr2=0x7f4d441f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d4abda640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d440b3b30 0x7f4d441f8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d4abda640 1 -- 172.21.15.55:0/383007271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d4406c5c0 msgr2=0x7f4d44154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d4abda640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d4406c5c0 0x7f4d44154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d4abda640 1 -- 172.21.15.55:0/383007271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d44154c80 con 0x7f4d4415f100 2024-05-20T22:10:30.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d4a3d9640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d4406c5c0 0x7f4d44154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:30.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d4abda640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 0x7f4d441562d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00c700 tx=0x7f4d3c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:30.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d3affd640 1 -- 172.21.15.55:0/383007271 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c018070 con 0x7f4d4415f100 2024-05-20T22:10:30.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d3affd640 1 -- 172.21.15.55:0/383007271 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c00ce10 con 0x7f4d4415f100 2024-05-20T22:10:30.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d3affd640 1 -- 172.21.15.55:0/383007271 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c00a860 con 0x7f4d4415f100 2024-05-20T22:10:30.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d4c63d640 1 -- 172.21.15.55:0/383007271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d44160530 con 0x7f4d4415f100 2024-05-20T22:10:30.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.604+0000 7f4d4c63d640 1 -- 172.21.15.55:0/383007271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d44160af0 con 0x7f4d4415f100 2024-05-20T22:10:30.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.605+0000 7f4d4c63d640 1 -- 172.21.15.55:0/383007271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d440b80c0 con 0x7f4d4415f100 2024-05-20T22:10:30.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.606+0000 7f4d3affd640 1 -- 172.21.15.55:0/383007271 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d3c00aa00 con 0x7f4d4415f100 2024-05-20T22:10:30.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.606+0000 7f4d3affd640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d2806be30 0x7f4d2806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:30.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.606+0000 7f4d3affd640 1 -- 172.21.15.55:0/383007271 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f4d3c08a540 con 0x7f4d4415f100 2024-05-20T22:10:30.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.606+0000 7f4d4a3d9640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d2806be30 0x7f4d2806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:30.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.607+0000 7f4d4a3d9640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d2806be30 0x7f4d2806e340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d40006fd0 tx=0x7f4d40008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:30.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.611+0000 7f4d3affd640 1 -- 172.21.15.55:0/383007271 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4d440b80c0 con 0x7f4d4415f100 2024-05-20T22:10:30.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.835+0000 7f4d4c63d640 1 -- 172.21.15.55:0/383007271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4d44069090 con 0x7f4d4415f100 2024-05-20T22:10:30.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.836+0000 7f4d3affd640 1 -- 172.21.15.55:0/383007271 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f4d3c058d60 con 0x7f4d4415f100 2024-05-20T22:10:30.837 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T22:10:30.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 -- 172.21.15.55:0/383007271 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d2806be30 msgr2=0x7f4d2806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d2806be30 0x7f4d2806e340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d40006fd0 tx=0x7f4d40008040 comp rx=0 tx=0).stop 2024-05-20T22:10:30.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 -- 172.21.15.55:0/383007271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 msgr2=0x7f4d441562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:30.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 0x7f4d441562d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00c700 tx=0x7f4d3c00cbc0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 -- 172.21.15.55:0/383007271 shutdown_connections 2024-05-20T22:10:30.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4d2806be30 0x7f4d2806e340 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4d4415f100 0x7f4d441562d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4d440b3b30 0x7f4d441f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 --2- 172.21.15.55:0/383007271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d4406c5c0 0x7f4d44154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:30.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 -- 172.21.15.55:0/383007271 >> 172.21.15.55:0/383007271 conn(0x7f4d440d51d0 msgr2=0x7f4d440b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:30.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 -- 172.21.15.55:0/383007271 shutdown_connections 2024-05-20T22:10:30.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:30.838+0000 7f4d4c63d640 1 -- 172.21.15.55:0/383007271 wait complete. 2024-05-20T22:10:30.850 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T22:10:30.850 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T22:10:30.850 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-20T22:10:30.850 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T22:10:30.850 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T22:10:30.850 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T22:10:30.850 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T22:10:30.850 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T22:10:30.851 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T22:10:30.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:30.851 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-20T22:10:30.867 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T22:10:30.867 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-20T22:10:30.868 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42561sec preferred_lft 42561sec 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 52:e2:c2:8c:12:3b brd ff:ff:ff:ff:ff:ff 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ba:c7ff:fe99:ba87/64 scope link 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T22:10:30.869 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 52:e2:c2:8c:12:3b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:10:30.870 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::50e2:c2ff:fe8c:123b/64 scope link 2024-05-20T22:10:30.870 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:10:30.870 INFO:teuthology.orchestra.run.smithi055.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T22:10:30.870 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether d2:62:32:ab:d0:b9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:10:30.870 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::d062:32ff:feab:d0b9/64 scope link 2024-05-20T22:10:30.870 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-20T22:10:30.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:30.870 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T22:10:30.932 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-20T22:10:30.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:30.934 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-20T22:10:30.992 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-20T22:10:30.992 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T22:10:30.992 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T22:10:30.992 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T22:10:30.992 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T22:10:30.992 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T22:10:30.993 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-20T22:10:30.993 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T22:10:30.993 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T22:10:30.993 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T22:10:30.993 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-20T22:10:31.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T22:10:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:10:31.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:31.123 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-20T22:10:31.139 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T22:10:31.139 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T22:10:31.139 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T22:10:31.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:31.139 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-20T22:10:31.204 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-20T22:10:31.205 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:10:31.205 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T22:10:31.261 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T22:10:31.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T22:10:31.261 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T22:10:31.316 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:10:31.383 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:10:31.440 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:10:31.441 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:10:31.441 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:10:31.441 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:10:31.441 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:10:31.441 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:10:31.441 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:10:31.441 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:10:31.441 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:10:31.442 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:10:31.443 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:10:31.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:31.444 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:10:31.497 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-20T22:10:31.497 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-20T22:10:31.539 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-20T22:10:31.566 DEBUG:teuthology.orchestra.run.smithi055:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T22:10:31.624 INFO:teuthology.orchestra.run.smithi055.stdout:/proc 2024-05-20T22:10:31.624 INFO:teuthology.orchestra.run.smithi055.stdout:/sys 2024-05-20T22:10:31.624 INFO:teuthology.orchestra.run.smithi055.stdout:/dev 2024-05-20T22:10:31.624 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/security 2024-05-20T22:10:31.624 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/shm 2024-05-20T22:10:31.624 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/pts 2024-05-20T22:10:31.624 INFO:teuthology.orchestra.run.smithi055.stdout:/run 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/cgroup 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/pstore 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/bpf 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/ 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/selinux 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/hugepages 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/mqueue 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/debug 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/tracing 2024-05-20T22:10:31.625 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/fs/fuse/connections 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/sys/kernel/config 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/run/user/1000 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T22:10:31.626 INFO:teuthology.orchestra.run.smithi055.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T22:10:31.627 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns 2024-05-20T22:10:31.627 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:10:31.627 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:10:31.627 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:10:31.627 INFO:teuthology.orchestra.run.smithi055.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:10:31.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:31.628 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-20T22:10:31.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:10:31.634+0000 7fef2ac5d480 -1 init, newargv = 0x7fef0c005700 newargc=15 2024-05-20T22:10:31.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:ceph-fuse[58032]: starting ceph client 2024-05-20T22:10:31.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:ceph-fuse[58032]: starting fuse 2024-05-20T22:10:31.652 INFO:teuthology.orchestra.run.smithi055.stdout:40 2024-05-20T22:10:31.652 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-20T22:10:31.652 DEBUG:teuthology.orchestra.run.smithi055:> sudo stdin-killer -- python3 -c ' 2024-05-20T22:10:31.652 DEBUG:teuthology.orchestra.run.smithi055:> import glob 2024-05-20T22:10:31.652 DEBUG:teuthology.orchestra.run.smithi055:> import re 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> import subprocess 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> def _find_admin_socket(client_name): 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> files = glob.glob(asok_path) 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> # Given a non-glob path, it better be there 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> if "*" not in asok_path: 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> assert(len(files) == 1) 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> return files[0] 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> for f in files: 2024-05-20T22:10:31.653 DEBUG:teuthology.orchestra.run.smithi055:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-20T22:10:31.654 DEBUG:teuthology.orchestra.run.smithi055:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-20T22:10:31.654 DEBUG:teuthology.orchestra.run.smithi055:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-20T22:10:31.654 DEBUG:teuthology.orchestra.run.smithi055:> contents = proc_f.read() 2024-05-20T22:10:31.654 DEBUG:teuthology.orchestra.run.smithi055:> if mountpoint in contents: 2024-05-20T22:10:31.654 DEBUG:teuthology.orchestra.run.smithi055:> return f 2024-05-20T22:10:31.654 DEBUG:teuthology.orchestra.run.smithi055:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-20T22:10:31.654 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:10:31.654 DEBUG:teuthology.orchestra.run.smithi055:> print(_find_admin_socket("client.0")) 2024-05-20T22:10:31.654 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:10:31.774 INFO:teuthology.orchestra.run.smithi055.stdout:/var/run/ceph/ceph-client.0.58032.asok 2024-05-20T22:10:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:10:31.784 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58032.asok 2024-05-20T22:10:31.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:31.785 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58032.asok status 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout: "metadata": { 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_sha1": "d2b507c724ef6c8295ee0a35d185cdab167fd61d", 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout: "entity_id": "0", 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": "smithi055", 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout: "pid": "58032", 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout: "root": "/" 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:31.932 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_count": 0, 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_pinned_count": 0, 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 6162, 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: "inst": { 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: "name": { 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "client", 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: "num": 6162 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.1:0", 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 2303448454 2024-05-20T22:10:31.933 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.1:0", 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 2303448454 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: "inst_str": "client.6162 192.168.0.1:0/2303448454", 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: "addr_str": "192.168.0.1:0/2303448454", 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: "inode_count": 1, 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_epoch": 35, 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch": 55, 2024-05-20T22:10:31.934 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch_barrier": 0, 2024-05-20T22:10:31.935 INFO:teuthology.orchestra.run.smithi055.stdout: "blocklisted": false, 2024-05-20T22:10:31.935 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs" 2024-05-20T22:10:31.935 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:10:31.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:31.941 DEBUG:teuthology.orchestra.run.smithi055:> stat --file-system '--printf=%T 2024-05-20T22:10:31.941 DEBUG:teuthology.orchestra.run.smithi055:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-20T22:10:31.957 INFO:teuthology.orchestra.run.smithi055.stdout:fuseblk 2024-05-20T22:10:31.957 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-20T22:10:31.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:10:31.957 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T22:10:32.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T22:10:32.067 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T22:10:32.110 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2024-05-20T22:10:32.110 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2024-05-20T22:10:32.110 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T22:10:32.110 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2024-05-20T22:10:32.111 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2024-05-20T22:10:32.112 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: mon_allow_pool_size_one = true 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2024-05-20T22:10:32.113 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: osd mclock profile = high_recovery_ops 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2024-05-20T22:10:32.114 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug trim objects = true 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2024-05-20T22:10:32.115 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T22:10:32.116 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-20T22:10:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: # 1m isn't quite enough 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: mon_down_mkfs_grace = 2m 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_filestore_osds = false 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: mon down mkfs grace = 300 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:32.119 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = "" 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2024-05-20T22:10:32.120 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds balancer = 20 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-20T22:10:32.121 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-20T22:10:32.122 INFO:teuthology.orchestra.run.smithi055.stdout: osd_deep_scrub_large_omap_object_key_threshold = 1 2024-05-20T22:10:32.122 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-05-20T22:10:32.122 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-05-20T22:10:32.122 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T22:10:32.122 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-05-20T22:10:32.122 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-05-20T22:10:32.123 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T22:10:32.124 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:10:32.125 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-05-20T22:10:32.126 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-05-20T22:10:32.127 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-05-20T22:10:32.128 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-05-20T22:10:32.129 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T22:10:32.130 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: mon down mkfs grace = 300 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-05-20T22:10:32.131 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-05-20T22:10:32.132 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-20T22:10:32.133 INFO:teuthology.orchestra.run.smithi138.stdout: osd_deep_scrub_large_omap_object_key_threshold = 1 2024-05-20T22:10:32.135 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T22:10:32.135 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T22:10:32.135 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T22:10:32.135 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:10:32.136 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T22:10:32.136 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T22:10:32.136 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T22:10:32.136 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:10:32.136 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T22:10:32.136 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T22:10:32.136 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T22:10:32.136 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:10:32.137 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T22:10:32.137 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T22:10:32.137 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T22:10:32.137 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:10:32.148 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T22:10:32.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T22:10:32.149 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T22:10:32.149 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T22:10:32.187 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T22:10:32.187 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T22:10:32.192 INFO:tasks.ceph.mds.b:Started 2024-05-20T22:10:32.192 DEBUG:teuthology.parallel:result is None 2024-05-20T22:10:32.193 INFO:tasks.ceph.mds.d:Started 2024-05-20T22:10:32.193 DEBUG:teuthology.parallel:result is None 2024-05-20T22:10:32.230 INFO:tasks.ceph.mds.c:Started 2024-05-20T22:10:32.231 DEBUG:teuthology.parallel:result is None 2024-05-20T22:10:32.282 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-05-20T22:10:32.282 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-05-20T22:10:32.315 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-20T22:10:32.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:10:32.379+0000 7fef2472c640 -1 --2- 192.168.0.1:0/2303448454 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x5580faa5b4e0 0x5580faa5d900 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] is using msgr V1 protocol 2024-05-20T22:10:32.780 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:10:32.780+0000 7fef2472c640 -1 --2- 192.168.0.1:0/2303448454 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x5580faa5b4e0 0x5580faa5d900 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] is using msgr V1 protocol 2024-05-20T22:10:33.581 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:10:33.581+0000 7fef2472c640 -1 --2- 192.168.0.1:0/2303448454 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x5580faa5b4e0 0x5580faa5d900 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] is using msgr V1 protocol 2024-05-20T22:10:35.183 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:2024-05-20T22:10:35.183+0000 7fef2472c640 -1 --2- 192.168.0.1:0/2303448454 >> [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] conn(0x5580faa5b4e0 0x5580faa5d900 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.55:6835/2445634740,v1:172.21.15.55:6837/2445634740] is using msgr V1 protocol 2024-05-20T22:10:35.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-05-20T22:10:38.187 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T22:10:38.188 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T22:10:38.191 INFO:tasks.ceph.mds.a:Started 2024-05-20T22:10:38.192 DEBUG:teuthology.parallel:result is None 2024-05-20T22:10:38.192 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:38.293 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-20T22:10:38.297 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-20T22:10:38.296+0000 7f0b3ef60640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T22:10:38.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.327+0000 7f5702d45640 1 Processor -- start 2024-05-20T22:10:38.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.327+0000 7f5702d45640 1 -- start start 2024-05-20T22:10:38.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.328+0000 7f5702d45640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56fc072580 0x7f56fc0b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:38.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.328+0000 7f5702d45640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56fc162e20 con 0x7f56fc0d2190 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.328+0000 7f5702d45640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56fc163100 con 0x7f56fc0b8100 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.328+0000 7f5702d45640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56fc1633e0 con 0x7f56fc072580 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f57012e2640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f56fc0d2190 0x7f56fc15fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59722/0 (socket says 172.21.15.55:59722) 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f57012e2640 1 -- 172.21.15.55:0/3061521194 learned_addr learned my addr 172.21.15.55:0/3061521194 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539991453 0 0) 0x7f56fc162e20 con 0x7f56fc0d2190 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56ec003520 con 0x7f56fc0d2190 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fbfff640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56fc072580 0x7f56fc0b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 745215600 0 0) 0x7f56ec003520 con 0x7f56fc0d2190 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56fc072580 msgr2=0x7f56fc0b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56fc072580 0x7f56fc0b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 >> v1:172.21.15.138:6789/0 conn(0x7f56fc0b8100 legacy=0x7f56fc071e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56fc1636c0 con 0x7f56fc0d2190 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f56f40019f0 con 0x7f56fc0d2190 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 >> v1:172.21.15.55:6789/0 conn(0x7f56fc0d2190 legacy=0x7f56fc15fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56ec0039e0 0x7f56ec003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56ec004370 0x7f56ec0048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec0050c0 0x7f56ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56fc1633e0 con 0x7f56ec004370 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec00c010 con 0x7f56ec0050c0 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56ec00c210 con 0x7f56ec0039e0 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.329+0000 7f56fbfff640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56ec004370 0x7f56ec0048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.330+0000 7f5700ae1640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56ec0039e0 0x7f56ec003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.330+0000 7f57012e2640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec0050c0 0x7f56ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.330+0000 7f56fbfff640 1 -- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56ec0039e0 msgr2=0x7f56ec003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.330+0000 7f56fbfff640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56ec0039e0 0x7f56ec003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.330+0000 7f56fbfff640 1 -- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec0050c0 msgr2=0x7f56ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.330+0000 7f56fbfff640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec0050c0 0x7f56ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:38.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.330+0000 7f56fbfff640 1 -- 172.21.15.55:0/3061521194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec003520 con 0x7f56ec004370 2024-05-20T22:10:38.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.331+0000 7f56fbfff640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56ec004370 0x7f56ec0048c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009ac0 tx=0x7f56ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15132e026816eb7e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:38.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.331+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8036070 con 0x7f56ec004370 2024-05-20T22:10:38.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.331+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e8031400 con 0x7f56ec004370 2024-05-20T22:10:38.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.331+0000 7f56fb7fe640 1 -- 172.21.15.55:0/3061521194 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8009690 con 0x7f56ec004370 2024-05-20T22:10:38.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.331+0000 7f5702d45640 1 -- 172.21.15.55:0/3061521194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56ec004370 msgr2=0x7f56ec0048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:38.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.331+0000 7f5702d45640 1 --2- 172.21.15.55:0/3061521194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56ec004370 0x7f56ec0048c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009ac0 tx=0x7f56ec005830 comp rx=0 tx=0).stop 2024-05-20T22:10:38.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.331+0000 7f57012e2640 1 -- 172.21.15.55:0/3061521194 reap_dead start 2024-05-20T22:10:38.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.332+0000 7f5702d45640 1 -- 172.21.15.55:0/3061521194 shutdown_connections 2024-05-20T22:10:38.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.332+0000 7f5702d45640 1 -- 172.21.15.55:0/3061521194 >> 172.21.15.55:0/3061521194 conn(0x7f56fc0d2960 msgr2=0x7f56fc0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:38.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.332+0000 7f5702d45640 1 -- 172.21.15.55:0/3061521194 shutdown_connections 2024-05-20T22:10:38.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.332+0000 7f5702d45640 1 -- 172.21.15.55:0/3061521194 wait complete. 2024-05-20T22:10:38.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.333+0000 7f5702d45640 1 Processor -- start 2024-05-20T22:10:38.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.333+0000 7f5702d45640 1 -- start start 2024-05-20T22:10:38.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f5702d45640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec004370 0x7f56fc0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:38.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f5702d45640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56fc15c770 0x7f56fc161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f5702d45640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc15cb80 0x7f56fc162150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f5702d45640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f56fc0d25a0 con 0x7f56fc15cb80 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f5702d45640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56fc163d10 con 0x7f56ec004370 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f5702d45640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56fc0b5900 con 0x7f56fc15c770 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f57012e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc15cb80 0x7f56fc162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f57012e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc15cb80 0x7f56fc162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57456/0 (socket says 172.21.15.55:57456) 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f57012e2640 1 -- 172.21.15.55:0/3576979546 learned_addr learned my addr 172.21.15.55:0/3576979546 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f56fbfff640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56fc15c770 0x7f56fc161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f5700ae1640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec004370 0x7f56fc0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f57012e2640 1 -- 172.21.15.55:0/3576979546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56fc15c770 msgr2=0x7f56fc161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f57012e2640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56fc15c770 0x7f56fc161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f57012e2640 1 -- 172.21.15.55:0/3576979546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec004370 msgr2=0x7f56fc0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f57012e2640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec004370 0x7f56fc0cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.335+0000 7f57012e2640 1 -- 172.21.15.55:0/3576979546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56fc0c7e10 con 0x7f56fc15cb80 2024-05-20T22:10:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.336+0000 7f57012e2640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc15cb80 0x7f56fc162150 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0d1ab0 tx=0x7f56f4007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.336+0000 7f56f97fa640 1 -- 172.21.15.55:0/3576979546 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f400ab20 con 0x7f56fc15cb80 2024-05-20T22:10:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.336+0000 7f56f97fa640 1 -- 172.21.15.55:0/3576979546 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f4019070 con 0x7f56fc15cb80 2024-05-20T22:10:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.336+0000 7f56f97fa640 1 -- 172.21.15.55:0/3576979546 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f40133f0 con 0x7f56fc15cb80 2024-05-20T22:10:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.336+0000 7f5702d45640 1 -- 172.21.15.55:0/3576979546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56fc0c80f0 con 0x7f56fc15cb80 2024-05-20T22:10:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.336+0000 7f5702d45640 1 -- 172.21.15.55:0/3576979546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56fc0c86b0 con 0x7f56fc15cb80 2024-05-20T22:10:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.337+0000 7f5702d45640 1 -- 172.21.15.55:0/3576979546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56fc0c7b80 con 0x7f56fc15cb80 2024-05-20T22:10:38.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.338+0000 7f56f97fa640 1 -- 172.21.15.55:0/3576979546 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f56f4013590 con 0x7f56fc15cb80 2024-05-20T22:10:38.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.338+0000 7f56f97fa640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f56e006be60 0x7f56e006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:38.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.338+0000 7f56f97fa640 1 -- 172.21.15.55:0/3576979546 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f56f408b3d0 con 0x7f56fc15cb80 2024-05-20T22:10:38.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.338+0000 7f5700ae1640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f56e006be60 0x7f56e006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:38.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.339+0000 7f5700ae1640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f56e006be60 0x7f56e006e370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0d3700 tx=0x7f56f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:38.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.343+0000 7f56f97fa640 1 -- 172.21.15.55:0/3576979546 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f56f4059510 con 0x7f56fc15cb80 2024-05-20T22:10:38.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.563+0000 7f5702d45640 1 -- 172.21.15.55:0/3576979546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56fc0d1f70 con 0x7f56fc15cb80 2024-05-20T22:10:38.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.564+0000 7f56f97fa640 1 -- 172.21.15.55:0/3576979546 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5181 (secure 0 0 0) 0x7f56f405d810 con 0x7f56fc15cb80 2024-05-20T22:10:38.566 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:38.566 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":39,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:38.304548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-05-20T22:10:38.566 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 39 2024-05-20T22:10:38.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 -- 172.21.15.55:0/3576979546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f56e006be60 msgr2=0x7f56e006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:38.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f56e006be60 0x7f56e006e370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0d3700 tx=0x7f56f0009040 comp rx=0 tx=0).stop 2024-05-20T22:10:38.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 -- 172.21.15.55:0/3576979546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc15cb80 msgr2=0x7f56fc162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:38.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc15cb80 0x7f56fc162150 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0d1ab0 tx=0x7f56f4007a90 comp rx=0 tx=0).stop 2024-05-20T22:10:38.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 -- 172.21.15.55:0/3576979546 shutdown_connections 2024-05-20T22:10:38.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f56e006be60 0x7f56e006e370 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc15cb80 0x7f56fc162150 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56fc15c770 0x7f56fc161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 --2- 172.21.15.55:0/3576979546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec004370 0x7f56fc0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 -- 172.21.15.55:0/3576979546 >> 172.21.15.55:0/3576979546 conn(0x7f56fc0d2960 msgr2=0x7f56fc15eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 -- 172.21.15.55:0/3576979546 shutdown_connections 2024-05-20T22:10:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:38.566+0000 7f5702d45640 1 -- 172.21.15.55:0/3576979546 wait complete. 2024-05-20T22:10:38.982 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:10:23.995145+0000', 'modified': '2024-05-20T22:10:38.304548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6189}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6189': {'gid': 6189, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.55:6837/3463078364', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6836', 'nonce': 3463078364}, {'type': 'v1', 'addr': '172.21.15.55:6837', 'nonce': 3463078364}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T22:10:38.982 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6189:up:replay 2024-05-20T22:10:39.983 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:40.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.110+0000 7f8aba46c640 1 Processor -- start 2024-05-20T22:10:40.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.110+0000 7f8aba46c640 1 -- start start 2024-05-20T22:10:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.112+0000 7f8aba46c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ab40d3d80 0x7f8ab40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.112+0000 7f8aba46c640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ab4160550 con 0x7f8ab40d42a0 2024-05-20T22:10:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.112+0000 7f8aba46c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ab415f9d0 con 0x7f8ab415ccc0 2024-05-20T22:10:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.112+0000 7f8aba46c640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f8ab415fcb0 con 0x7f8ab40d3d80 2024-05-20T22:10:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.112+0000 7f8ab37fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ab40d3d80 0x7f8ab40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.112+0000 7f8ab3fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8ab40d42a0 0x7f8ab40d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43082/0 (socket says 172.21.15.55:43082) 2024-05-20T22:10:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.112+0000 7f8ab3fff640 1 -- 172.21.15.55:0/2761814475 learned_addr learned my addr 172.21.15.55:0/2761814475 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.112+0000 7f8ab37fe640 1 -- 172.21.15.55:0/2761814475 >> v1:172.21.15.138:6789/0 conn(0x7f8ab415ccc0 legacy=0x7f8ab40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.112+0000 7f8ab37fe640 1 -- 172.21.15.55:0/2761814475 >> v1:172.21.15.55:6789/0 conn(0x7f8ab40d42a0 legacy=0x7f8ab40d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.112+0000 7f8ab37fe640 1 -- 172.21.15.55:0/2761814475 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab415ff90 con 0x7f8ab40d3d80 2024-05-20T22:10:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.113+0000 7f8ab37fe640 1 --2- 172.21.15.55:0/2761814475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab40d3d80 0x7f8ab40cf710 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80097a0 tx=0x7f8aa802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9987b8a4d4b9f62 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.113+0000 7f8ab2ffd640 1 -- 172.21.15.55:0/2761814475 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8037070 con 0x7f8ab40d3d80 2024-05-20T22:10:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.113+0000 7f8ab2ffd640 1 -- 172.21.15.55:0/2761814475 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa802ae50 con 0x7f8ab40d3d80 2024-05-20T22:10:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.113+0000 7f8ab2ffd640 1 -- 172.21.15.55:0/2761814475 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8032660 con 0x7f8ab40d3d80 2024-05-20T22:10:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.114+0000 7f8aba46c640 1 -- 172.21.15.55:0/2761814475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab40d3d80 msgr2=0x7f8ab40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.114+0000 7f8aba46c640 1 --2- 172.21.15.55:0/2761814475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab40d3d80 0x7f8ab40cf710 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80097a0 tx=0x7f8aa802a140 comp rx=0 tx=0).stop 2024-05-20T22:10:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.114+0000 7f8aba46c640 1 -- 172.21.15.55:0/2761814475 shutdown_connections 2024-05-20T22:10:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.114+0000 7f8aba46c640 1 --2- 172.21.15.55:0/2761814475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab40d3d80 0x7f8ab40cf710 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:40.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.114+0000 7f8aba46c640 1 -- 172.21.15.55:0/2761814475 >> 172.21.15.55:0/2761814475 conn(0x7f8ab40d1c50 msgr2=0x7f8ab40d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:40.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.114+0000 7f8aba46c640 1 -- 172.21.15.55:0/2761814475 shutdown_connections 2024-05-20T22:10:40.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.114+0000 7f8aba46c640 1 -- 172.21.15.55:0/2761814475 wait complete. 2024-05-20T22:10:40.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.116+0000 7f8aba46c640 1 Processor -- start 2024-05-20T22:10:40.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.116+0000 7f8aba46c640 1 -- start start 2024-05-20T22:10:40.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.117+0000 7f8aba46c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab415ccc0 0x7f8ab4153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:40.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.117+0000 7f8aba46c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab4158980 0x7f8ab4153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:40.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.117+0000 7f8aba46c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ab4159310 0x7f8ab414fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:40.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.117+0000 7f8aba46c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab415f7b0 con 0x7f8ab415ccc0 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.117+0000 7f8aba46c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab415f630 con 0x7f8ab4158980 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.117+0000 7f8aba46c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab406c880 con 0x7f8ab4159310 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab415ccc0 0x7f8ab4153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab415ccc0 0x7f8ab4153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50752/0 (socket says 172.21.15.55:50752) 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab3fff640 1 -- 172.21.15.55:0/291191021 learned_addr learned my addr 172.21.15.55:0/291191021 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab37fe640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab4158980 0x7f8ab4153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab8a09640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ab4159310 0x7f8ab414fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab3fff640 1 -- 172.21.15.55:0/291191021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ab4159310 msgr2=0x7f8ab414fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab3fff640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ab4159310 0x7f8ab414fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab3fff640 1 -- 172.21.15.55:0/291191021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab4158980 msgr2=0x7f8ab4153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab3fff640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab4158980 0x7f8ab4153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab3fff640 1 -- 172.21.15.55:0/291191021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab4161710 con 0x7f8ab415ccc0 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab37fe640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab4158980 0x7f8ab4153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab8a09640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ab4159310 0x7f8ab414fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab3fff640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab415ccc0 0x7f8ab4153590 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400c3f0 tx=0x7f8aa400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab0ff9640 1 -- 172.21.15.55:0/291191021 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4017070 con 0x7f8ab415ccc0 2024-05-20T22:10:40.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab0ff9640 1 -- 172.21.15.55:0/291191021 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa4005ce0 con 0x7f8ab415ccc0 2024-05-20T22:10:40.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8aba46c640 1 -- 172.21.15.55:0/291191021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab41619f0 con 0x7f8ab415ccc0 2024-05-20T22:10:40.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8ab0ff9640 1 -- 172.21.15.55:0/291191021 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40072f0 con 0x7f8ab415ccc0 2024-05-20T22:10:40.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.118+0000 7f8aba46c640 1 -- 172.21.15.55:0/291191021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab4161fb0 con 0x7f8ab415ccc0 2024-05-20T22:10:40.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.119+0000 7f8aba46c640 1 -- 172.21.15.55:0/291191021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab404f720 con 0x7f8ab415ccc0 2024-05-20T22:10:40.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.120+0000 7f8ab0ff9640 1 -- 172.21.15.55:0/291191021 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8aa401a070 con 0x7f8ab415ccc0 2024-05-20T22:10:40.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.120+0000 7f8ab0ff9640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8aa006be30 0x7f8aa006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:40.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.121+0000 7f8ab0ff9640 1 -- 172.21.15.55:0/291191021 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f8aa408a480 con 0x7f8ab415ccc0 2024-05-20T22:10:40.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.121+0000 7f8ab37fe640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8aa006be30 0x7f8aa006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:40.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.121+0000 7f8ab37fe640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8aa006be30 0x7f8aa006e340 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8034000 tx=0x7f8aa802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:40.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.125+0000 7f8ab0ff9640 1 -- 172.21.15.55:0/291191021 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8aa4058b70 con 0x7f8ab415ccc0 2024-05-20T22:10:40.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.354+0000 7f8aba46c640 1 -- 172.21.15.55:0/291191021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ab40d2f40 con 0x7f8ab415ccc0 2024-05-20T22:10:40.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.354+0000 7f8ab0ff9640 1 -- 172.21.15.55:0/291191021 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5181 (secure 0 0 0) 0x7f8aa4058470 con 0x7f8ab415ccc0 2024-05-20T22:10:40.356 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:40.357 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:39.315371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-05-20T22:10:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 41 2024-05-20T22:10:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.356+0000 7f8aba46c640 1 -- 172.21.15.55:0/291191021 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8aa006be30 msgr2=0x7f8aa006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.356+0000 7f8aba46c640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8aa006be30 0x7f8aa006e340 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8034000 tx=0x7f8aa802e000 comp rx=0 tx=0).stop 2024-05-20T22:10:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.356+0000 7f8aba46c640 1 -- 172.21.15.55:0/291191021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab415ccc0 msgr2=0x7f8ab4153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.356+0000 7f8aba46c640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab415ccc0 0x7f8ab4153590 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400c3f0 tx=0x7f8aa400c8b0 comp rx=0 tx=0).stop 2024-05-20T22:10:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.357+0000 7f8aba46c640 1 -- 172.21.15.55:0/291191021 shutdown_connections 2024-05-20T22:10:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.357+0000 7f8aba46c640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8aa006be30 0x7f8aa006e340 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.357+0000 7f8aba46c640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ab4159310 0x7f8ab414fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.357+0000 7f8aba46c640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab4158980 0x7f8ab4153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.357+0000 7f8aba46c640 1 --2- 172.21.15.55:0/291191021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ab415ccc0 0x7f8ab4153590 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.357+0000 7f8aba46c640 1 -- 172.21.15.55:0/291191021 >> 172.21.15.55:0/291191021 conn(0x7f8ab40d1c50 msgr2=0x7f8ab415aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:40.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.357+0000 7f8aba46c640 1 -- 172.21.15.55:0/291191021 shutdown_connections 2024-05-20T22:10:40.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:40.357+0000 7f8aba46c640 1 -- 172.21.15.55:0/291191021 wait complete. 2024-05-20T22:10:40.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:10:23.995145+0000', 'modified': '2024-05-20T22:10:39.315371+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6189}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6189': {'gid': 6189, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.55:6837/3463078364', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6836', 'nonce': 3463078364}, {'type': 'v1', 'addr': '172.21.15.55:6837', 'nonce': 3463078364}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T22:10:40.773 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6189:up:rejoin 2024-05-20T22:10:41.774 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:41.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.902+0000 7f2e7e902640 1 Processor -- start 2024-05-20T22:10:41.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.902+0000 7f2e7e902640 1 -- start start 2024-05-20T22:10:41.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e7e902640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e780b5760 0x7f2e780d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:41.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e7e902640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2e78071bc0 0x7f2e78071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:41.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e7e902640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e78072510 con 0x7f2e780ce350 2024-05-20T22:10:41.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e7e902640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2e780b1a70 con 0x7f2e78071bc0 2024-05-20T22:10:41.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e7e902640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e780b1cc0 con 0x7f2e780b5760 2024-05-20T22:10:41.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e6ffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2e780ce350 0x7f2e7815ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43098/0 (socket says 172.21.15.55:43098) 2024-05-20T22:10:41.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e6ffff640 1 -- 172.21.15.55:0/2093651592 learned_addr learned my addr 172.21.15.55:0/2093651592 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e7ce9f640 1 --2- 172.21.15.55:0/2093651592 >> v2:172.21.15.138:3300/0 conn(0x7f2e78071bc0 0x7f2e78071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2754931037 0 0) 0x7f2e60001110 con 0x7f2e780ce350 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e77fff640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e780b5760 0x7f2e780d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e64003520 con 0x7f2e780ce350 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3407789005 0 0) 0x7f2e64003520 con 0x7f2e780ce350 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e780b5760 msgr2=0x7f2e780d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e780b5760 0x7f2e780d0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 >> v2:172.21.15.138:3300/0 conn(0x7f2e78071bc0 msgr2=0x7f2e78071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 --2- 172.21.15.55:0/2093651592 >> v2:172.21.15.138:3300/0 conn(0x7f2e78071bc0 0x7f2e78071fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e780d0680 con 0x7f2e780ce350 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f2e60001840 con 0x7f2e780ce350 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 >> v1:172.21.15.55:6789/0 conn(0x7f2e780ce350 legacy=0x7f2e7815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:41.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e780b5760 0x7f2e640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.904+0000 7f2e777fe640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64003f30 0x7f2e64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e777fe640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e64004d40 0x7f2e64005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780b1a70 con 0x7f2e64003f30 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780b1cc0 con 0x7f2e64004d40 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6400bb50 con 0x7f2e780b5760 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e6ffff640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64003f30 0x7f2e64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e77fff640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e780b5760 0x7f2e640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e7ce9f640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e64004d40 0x7f2e64005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e6ffff640 1 -- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e780b5760 msgr2=0x7f2e640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e6ffff640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e780b5760 0x7f2e640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e6ffff640 1 -- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e64004d40 msgr2=0x7f2e64005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e6ffff640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e64004d40 0x7f2e64005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e6ffff640 1 -- 172.21.15.55:0/2093651592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e64003520 con 0x7f2e64003f30 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.905+0000 7f2e7ce9f640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e64004d40 0x7f2e64005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.906+0000 7f2e6ffff640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64003f30 0x7f2e64004800 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2e60001050 tx=0x7f2e64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a64d8d67c3c39 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.906+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000c0e0 con 0x7f2e64003f30 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.906+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e60034070 con 0x7f2e64003f30 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.906+0000 7f2e777fe640 1 -- 172.21.15.55:0/2093651592 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60005d10 con 0x7f2e64003f30 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.907+0000 7f2e7e902640 1 -- 172.21.15.55:0/2093651592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64003f30 msgr2=0x7f2e64004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.907+0000 7f2e7e902640 1 --2- 172.21.15.55:0/2093651592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64003f30 0x7f2e64004800 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2e60001050 tx=0x7f2e64005830 comp rx=0 tx=0).stop 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.907+0000 7f2e7ce9f640 1 -- 172.21.15.55:0/2093651592 reap_dead start 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.907+0000 7f2e7e902640 1 -- 172.21.15.55:0/2093651592 shutdown_connections 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.907+0000 7f2e7e902640 1 -- 172.21.15.55:0/2093651592 >> 172.21.15.55:0/2093651592 conn(0x7f2e780d2910 msgr2=0x7f2e780d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.907+0000 7f2e7e902640 1 -- 172.21.15.55:0/2093651592 shutdown_connections 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.907+0000 7f2e7e902640 1 -- 172.21.15.55:0/2093651592 wait complete. 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.909+0000 7f2e7e902640 1 Processor -- start 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.909+0000 7f2e7e902640 1 -- start start 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7e902640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e64003f30 0x7f2e78149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:41.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7e902640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e78158e50 0x7f2e7815b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7e902640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e780c9400 0x7f2e781eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7e902640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780d20c0 con 0x7f2e780c9400 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7e902640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780d1f40 con 0x7f2e64003f30 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7e902640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e780d0e80 con 0x7f2e78158e50 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7ce9f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e780c9400 0x7f2e781eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7ce9f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e780c9400 0x7f2e781eaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50778/0 (socket says 172.21.15.55:50778) 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7ce9f640 1 -- 172.21.15.55:0/3667677818 learned_addr learned my addr 172.21.15.55:0/3667677818 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e6ffff640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e78158e50 0x7f2e7815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e77fff640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e64003f30 0x7f2e78149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7ce9f640 1 -- 172.21.15.55:0/3667677818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e78158e50 msgr2=0x7f2e7815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.910+0000 7f2e7ce9f640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e78158e50 0x7f2e7815b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.911+0000 7f2e7ce9f640 1 -- 172.21.15.55:0/3667677818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e64003f30 msgr2=0x7f2e78149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.911+0000 7f2e7ce9f640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e64003f30 0x7f2e78149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:41.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.911+0000 7f2e7ce9f640 1 -- 172.21.15.55:0/3667677818 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e780c8360 con 0x7f2e780c9400 2024-05-20T22:10:41.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.911+0000 7f2e7ce9f640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e780c9400 0x7f2e781eaad0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2e70007c80 tx=0x7f2e700115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:41.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.911+0000 7f2e757fa640 1 -- 172.21.15.55:0/3667677818 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7001a020 con 0x7f2e780c9400 2024-05-20T22:10:41.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.911+0000 7f2e757fa640 1 -- 172.21.15.55:0/3667677818 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e70011bd0 con 0x7f2e780c9400 2024-05-20T22:10:41.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.911+0000 7f2e757fa640 1 -- 172.21.15.55:0/3667677818 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70016470 con 0x7f2e780c9400 2024-05-20T22:10:41.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.911+0000 7f2e7e902640 1 -- 172.21.15.55:0/3667677818 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e780c8640 con 0x7f2e780c9400 2024-05-20T22:10:41.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.911+0000 7f2e7e902640 1 -- 172.21.15.55:0/3667677818 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e780c8c00 con 0x7f2e780c9400 2024-05-20T22:10:41.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.912+0000 7f2e7e902640 1 -- 172.21.15.55:0/3667677818 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7804f720 con 0x7f2e780c9400 2024-05-20T22:10:41.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.913+0000 7f2e757fa640 1 -- 172.21.15.55:0/3667677818 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e70005d10 con 0x7f2e780c9400 2024-05-20T22:10:41.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.913+0000 7f2e757fa640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e5806be60 0x7f2e5806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:41.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.913+0000 7f2e757fa640 1 -- 172.21.15.55:0/3667677818 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f2e7008b9a0 con 0x7f2e780c9400 2024-05-20T22:10:41.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.913+0000 7f2e77fff640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e5806be60 0x7f2e5806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:41.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.914+0000 7f2e77fff640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e5806be60 0x7f2e5806e370 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009b20 tx=0x7f2e68009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:41.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:41.918+0000 7f2e757fa640 1 -- 172.21.15.55:0/3667677818 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2e70024030 con 0x7f2e780c9400 2024-05-20T22:10:42.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.141+0000 7f2e7e902640 1 -- 172.21.15.55:0/3667677818 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e780d1d20 con 0x7f2e780c9400 2024-05-20T22:10:42.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.141+0000 7f2e757fa640 1 -- 172.21.15.55:0/3667677818 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5187 (secure 0 0 0) 0x7f2e7005a1c0 con 0x7f2e780c9400 2024-05-20T22:10:42.144 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:42.144 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:40.317488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:clientreplay","state_seq":4,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-05-20T22:10:42.144 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 42 2024-05-20T22:10:42.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 -- 172.21.15.55:0/3667677818 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e5806be60 msgr2=0x7f2e5806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:42.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e5806be60 0x7f2e5806e370 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009b20 tx=0x7f2e68009190 comp rx=0 tx=0).stop 2024-05-20T22:10:42.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 -- 172.21.15.55:0/3667677818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e780c9400 msgr2=0x7f2e781eaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:42.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e780c9400 0x7f2e781eaad0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2e70007c80 tx=0x7f2e700115a0 comp rx=0 tx=0).stop 2024-05-20T22:10:42.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 -- 172.21.15.55:0/3667677818 shutdown_connections 2024-05-20T22:10:42.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e5806be60 0x7f2e5806e370 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:42.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e780c9400 0x7f2e781eaad0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:42.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e78158e50 0x7f2e7815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:42.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 --2- 172.21.15.55:0/3667677818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e64003f30 0x7f2e78149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:42.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 -- 172.21.15.55:0/3667677818 >> 172.21.15.55:0/3667677818 conn(0x7f2e780d2910 msgr2=0x7f2e7815cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:42.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 -- 172.21.15.55:0/3667677818 shutdown_connections 2024-05-20T22:10:42.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:42.144+0000 7f2e7e902640 1 -- 172.21.15.55:0/3667677818 wait complete. 2024-05-20T22:10:42.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-05-20T22:10:42.561 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:10:23.995145+0000', 'modified': '2024-05-20T22:10:40.317488+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6189}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6189': {'gid': 6189, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:clientreplay', 'state_seq': 4, 'addr': '172.21.15.55:6837/3463078364', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6836', 'nonce': 3463078364}, {'type': 'v1', 'addr': '172.21.15.55:6837', 'nonce': 3463078364}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T22:10:42.561 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6189:up:clientreplay 2024-05-20T22:10:43.310 INFO:tasks.ceph.mds.a.smithi055.stderr:2024-05-20T22:10:43.309+0000 7efd6d0e9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T22:10:43.516 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:10:43.520 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:10:43.562 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:43.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.686+0000 7f8c1f5ea640 1 Processor -- start 2024-05-20T22:10:43.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.687+0000 7f8c1f5ea640 1 -- start start 2024-05-20T22:10:43.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.688+0000 7f8c1f5ea640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8c180d3d80 0x7f8c180cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:43.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.688+0000 7f8c1f5ea640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c18160550 con 0x7f8c1815ccc0 2024-05-20T22:10:43.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.688+0000 7f8c1f5ea640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c1815f9d0 con 0x7f8c180d42a0 2024-05-20T22:10:43.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.688+0000 7f8c1f5ea640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f8c1815fcb0 con 0x7f8c180d3d80 2024-05-20T22:10:43.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.688+0000 7f8c1cb85640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8c180d3d80 0x7f8c180cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:43.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.688+0000 7f8c1db87640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8c1815ccc0 0x7f8c180b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43110/0 (socket says 172.21.15.55:43110) 2024-05-20T22:10:43.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.688+0000 7f8c1db87640 1 -- 172.21.15.55:0/3213629509 learned_addr learned my addr 172.21.15.55:0/3213629509 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:43.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.689+0000 7f8c1cb85640 1 -- 172.21.15.55:0/3213629509 >> v1:172.21.15.138:6789/0 conn(0x7f8c180d42a0 legacy=0x7f8c180d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:43.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.689+0000 7f8c1cb85640 1 -- 172.21.15.55:0/3213629509 >> v1:172.21.15.55:6789/0 conn(0x7f8c1815ccc0 legacy=0x7f8c180b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:43.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.689+0000 7f8c1cb85640 1 -- 172.21.15.55:0/3213629509 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1815ff90 con 0x7f8c180d3d80 2024-05-20T22:10:43.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.689+0000 7f8c1cb85640 1 --2- 172.21.15.55:0/3213629509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c180d3d80 0x7f8c180cf710 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097a0 tx=0x7f8c0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb0d6cd49099efc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:43.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.689+0000 7f8c0ffff640 1 -- 172.21.15.55:0/3213629509 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08037070 con 0x7f8c180d3d80 2024-05-20T22:10:43.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.690+0000 7f8c0ffff640 1 -- 172.21.15.55:0/3213629509 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c08032440 con 0x7f8c180d3d80 2024-05-20T22:10:43.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.690+0000 7f8c0ffff640 1 -- 172.21.15.55:0/3213629509 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08032810 con 0x7f8c180d3d80 2024-05-20T22:10:43.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.690+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/3213629509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c180d3d80 msgr2=0x7f8c180cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:43.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.690+0000 7f8c1f5ea640 1 --2- 172.21.15.55:0/3213629509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c180d3d80 0x7f8c180cf710 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097a0 tx=0x7f8c0802a1a0 comp rx=0 tx=0).stop 2024-05-20T22:10:43.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.690+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/3213629509 shutdown_connections 2024-05-20T22:10:43.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.690+0000 7f8c1f5ea640 1 --2- 172.21.15.55:0/3213629509 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c180d3d80 0x7f8c180cf710 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:43.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.690+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/3213629509 >> 172.21.15.55:0/3213629509 conn(0x7f8c180d1c50 msgr2=0x7f8c180d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:43.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.690+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/3213629509 shutdown_connections 2024-05-20T22:10:43.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.690+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/3213629509 wait complete. 2024-05-20T22:10:43.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.692+0000 7f8c1f5ea640 1 Processor -- start 2024-05-20T22:10:43.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.692+0000 7f8c1f5ea640 1 -- start start 2024-05-20T22:10:43.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.693+0000 7f8c1f5ea640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c1815ccc0 0x7f8c18153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:43.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.693+0000 7f8c1f5ea640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c18153870 0x7f8c181f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:43.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.693+0000 7f8c1f5ea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c181f8e80 0x7f8c181f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:43.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.693+0000 7f8c1f5ea640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1815f7b0 con 0x7f8c1815ccc0 2024-05-20T22:10:43.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.693+0000 7f8c1f5ea640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1815f630 con 0x7f8c181f8e80 2024-05-20T22:10:43.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.693+0000 7f8c1f5ea640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1806c880 con 0x7f8c18153870 2024-05-20T22:10:43.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1d386640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c1815ccc0 0x7f8c18153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:43.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1d386640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c1815ccc0 0x7f8c18153330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50796/0 (socket says 172.21.15.55:50796) 2024-05-20T22:10:43.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1d386640 1 -- 172.21.15.55:0/492485873 learned_addr learned my addr 172.21.15.55:0/492485873 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1db87640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c181f8e80 0x7f8c181f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1cb85640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c18153870 0x7f8c181f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1d386640 1 -- 172.21.15.55:0/492485873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c18153870 msgr2=0x7f8c181f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1d386640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c18153870 0x7f8c181f8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1d386640 1 -- 172.21.15.55:0/492485873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c181f8e80 msgr2=0x7f8c181f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1d386640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c181f8e80 0x7f8c181f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1d386640 1 -- 172.21.15.55:0/492485873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1815a640 con 0x7f8c1815ccc0 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1cb85640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c18153870 0x7f8c181f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1d386640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c1815ccc0 0x7f8c18153330 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400c430 tx=0x7f8c1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c0dffb640 1 -- 172.21.15.55:0/492485873 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14017070 con 0x7f8c1815ccc0 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c0dffb640 1 -- 172.21.15.55:0/492485873 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c140052d0 con 0x7f8c1815ccc0 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/492485873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1815a920 con 0x7f8c1815ccc0 2024-05-20T22:10:43.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.694+0000 7f8c0dffb640 1 -- 172.21.15.55:0/492485873 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14003830 con 0x7f8c1815ccc0 2024-05-20T22:10:43.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.695+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/492485873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c181593f0 con 0x7f8c1815ccc0 2024-05-20T22:10:43.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.696+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/492485873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1804f720 con 0x7f8c1815ccc0 2024-05-20T22:10:43.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.696+0000 7f8c0dffb640 1 -- 172.21.15.55:0/492485873 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c1401a070 con 0x7f8c1815ccc0 2024-05-20T22:10:43.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.696+0000 7f8c0dffb640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8bfc06be60 0x7f8bfc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:43.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.697+0000 7f8c0dffb640 1 -- 172.21.15.55:0/492485873 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f8c1408a3d0 con 0x7f8c1815ccc0 2024-05-20T22:10:43.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.697+0000 7f8c1cb85640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8bfc06be60 0x7f8bfc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:43.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.697+0000 7f8c1cb85640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8bfc06be60 0x7f8bfc06e370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c08034000 tx=0x7f8c0802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:43.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.701+0000 7f8c0dffb640 1 -- 172.21.15.55:0/492485873 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8c14058480 con 0x7f8c1815ccc0 2024-05-20T22:10:43.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.923+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/492485873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c1815aaf0 con 0x7f8c1815ccc0 2024-05-20T22:10:43.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.924+0000 7f8c0dffb640 1 -- 172.21.15.55:0/492485873 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7f8c1405c780 con 0x7f8c1815ccc0 2024-05-20T22:10:43.926 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:43.926 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:10:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:10:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/492485873 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8bfc06be60 msgr2=0x7f8bfc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8bfc06be60 0x7f8bfc06e370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c08034000 tx=0x7f8c0802e000 comp rx=0 tx=0).stop 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/492485873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c1815ccc0 msgr2=0x7f8c18153330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c1815ccc0 0x7f8c18153330 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400c430 tx=0x7f8c1400c8f0 comp rx=0 tx=0).stop 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/492485873 shutdown_connections 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8bfc06be60 0x7f8bfc06e370 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c181f8e80 0x7f8c181f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c18153870 0x7f8c181f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 --2- 172.21.15.55:0/492485873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c1815ccc0 0x7f8c18153330 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/492485873 >> 172.21.15.55:0/492485873 conn(0x7f8c180d1c50 msgr2=0x7f8c180b9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/492485873 shutdown_connections 2024-05-20T22:10:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:43.926+0000 7f8c1f5ea640 1 -- 172.21.15.55:0/492485873 wait complete. 2024-05-20T22:10:44.341 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T22:10:23.995145+0000', 'modified': '2024-05-20T22:10:42.319833+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 61, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6189}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6189': {'gid': 6189, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.55:6837/3463078364', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6836', 'nonce': 3463078364}, {'type': 'v1', 'addr': '172.21.15.55:6837', 'nonce': 3463078364}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6189, 'qdb_cluster': [6189]} 2024-05-20T22:10:44.341 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T22:10:44.341 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-20T22:10:44.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.459+0000 7f221ab2a640 1 Processor -- start 2024-05-20T22:10:44.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.459+0000 7f221ab2a640 1 -- start start 2024-05-20T22:10:44.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f221ab2a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140b2c00 0x7f22140b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f221ab2a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f22140cffd0 0x7f22140d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f221ab2a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22140cb840 con 0x7f221415f420 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f221ab2a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f221415f830 con 0x7f22140cffd0 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f221ab2a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f22140cb5f0 con 0x7f22140b2c00 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22188c6640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f221415f420 0x7f22140ade70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43120/0 (socket says 172.21.15.55:43120) 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22188c6640 1 -- 172.21.15.55:0/3309648935 learned_addr learned my addr 172.21.15.55:0/3309648935 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010571455 0 0) 0x7f22140cb840 con 0x7f221415f420 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f2213fff640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140b2c00 0x7f22140b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21fc003520 con 0x7f221415f420 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22190c7640 1 --2- 172.21.15.55:0/3309648935 >> v2:172.21.15.138:3300/0 conn(0x7f22140cffd0 0x7f22140d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 839589953 0 0) 0x7f21fc003520 con 0x7f221415f420 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140b2c00 msgr2=0x7f22140b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22137fe640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140b2c00 0x7f22140b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 >> v2:172.21.15.138:3300/0 conn(0x7f22140cffd0 msgr2=0x7f22140d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22137fe640 1 --2- 172.21.15.55:0/3309648935 >> v2:172.21.15.138:3300/0 conn(0x7f22140cffd0 0x7f22140d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.461+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22140cbb20 con 0x7f221415f420 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f2204001670 con 0x7f221415f420 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 >> v1:172.21.15.55:6789/0 conn(0x7f221415f420 legacy=0x7f22140ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f22137fe640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140b2c00 0x7f21fc0039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f22137fe640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc003f50 0x7f21fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f22137fe640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21fc004d40 0x7f21fc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f22140cb5f0 con 0x7f21fc003f50 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc00ba50 con 0x7f21fc004d40 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f21fc00bc50 con 0x7f22140b2c00 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f2213fff640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc003f50 0x7f21fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f22188c6640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140b2c00 0x7f21fc0039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f22190c7640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21fc004d40 0x7f21fc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f2213fff640 1 -- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140b2c00 msgr2=0x7f21fc0039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f2213fff640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140b2c00 0x7f21fc0039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f2213fff640 1 -- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21fc004d40 msgr2=0x7f21fc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f2213fff640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21fc004d40 0x7f21fc005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.462+0000 7f2213fff640 1 -- 172.21.15.55:0/3309648935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc003520 con 0x7f21fc003f50 2024-05-20T22:10:44.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.463+0000 7f2213fff640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc003f50 0x7f21fc004800 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2208009a90 tx=0x7f21fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c75c16cdc2dc51 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:44.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.463+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208036070 con 0x7f21fc003f50 2024-05-20T22:10:44.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.463+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2208031400 con 0x7f21fc003f50 2024-05-20T22:10:44.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.463+0000 7f22137fe640 1 -- 172.21.15.55:0/3309648935 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208009690 con 0x7f21fc003f50 2024-05-20T22:10:44.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.464+0000 7f221ab2a640 1 -- 172.21.15.55:0/3309648935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc003f50 msgr2=0x7f21fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.464+0000 7f221ab2a640 1 --2- 172.21.15.55:0/3309648935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc003f50 0x7f21fc004800 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2208009a90 tx=0x7f21fc005830 comp rx=0 tx=0).stop 2024-05-20T22:10:44.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.464+0000 7f22190c7640 1 -- 172.21.15.55:0/3309648935 reap_dead start 2024-05-20T22:10:44.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.464+0000 7f221ab2a640 1 -- 172.21.15.55:0/3309648935 shutdown_connections 2024-05-20T22:10:44.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.464+0000 7f221ab2a640 1 -- 172.21.15.55:0/3309648935 >> 172.21.15.55:0/3309648935 conn(0x7f221409a400 msgr2=0x7f22140b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:44.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.464+0000 7f221ab2a640 1 -- 172.21.15.55:0/3309648935 shutdown_connections 2024-05-20T22:10:44.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.464+0000 7f221ab2a640 1 -- 172.21.15.55:0/3309648935 wait complete. 2024-05-20T22:10:44.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.465+0000 7f221ab2a640 1 Processor -- start 2024-05-20T22:10:44.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.466+0000 7f221ab2a640 1 -- start start 2024-05-20T22:10:44.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f221ab2a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21fc003f50 0x7f2214151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f221ab2a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22140c4ad0 0x7f22140ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f221ab2a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140ae950 0x7f2214160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f221ab2a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f22140d0920 con 0x7f22140c4ad0 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f221ab2a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22140cc0e0 con 0x7f21fc003f50 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f221ab2a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f22140cd5d0 con 0x7f22140ae950 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f2213fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22140c4ad0 0x7f22140ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f2213fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22140c4ad0 0x7f22140ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50800/0 (socket says 172.21.15.55:50800) 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f2213fff640 1 -- 172.21.15.55:0/2522664838 learned_addr learned my addr 172.21.15.55:0/2522664838 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f22188c6640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21fc003f50 0x7f2214151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f22190c7640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140ae950 0x7f2214160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f2213fff640 1 -- 172.21.15.55:0/2522664838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140ae950 msgr2=0x7f2214160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f2213fff640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140ae950 0x7f2214160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f2213fff640 1 -- 172.21.15.55:0/2522664838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21fc003f50 msgr2=0x7f2214151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.467+0000 7f2213fff640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21fc003f50 0x7f2214151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.468+0000 7f2213fff640 1 -- 172.21.15.55:0/2522664838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22140aed60 con 0x7f22140c4ad0 2024-05-20T22:10:44.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.468+0000 7f2213fff640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22140c4ad0 0x7f22140ae410 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2208034040 tx=0x7f2208004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:44.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.468+0000 7f22117fa640 1 -- 172.21.15.55:0/2522664838 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208040020 con 0x7f22140c4ad0 2024-05-20T22:10:44.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.468+0000 7f22117fa640 1 -- 172.21.15.55:0/2522664838 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2208003bc0 con 0x7f22140c4ad0 2024-05-20T22:10:44.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.468+0000 7f221ab2a640 1 -- 172.21.15.55:0/2522664838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2214151960 con 0x7f22140c4ad0 2024-05-20T22:10:44.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.468+0000 7f22117fa640 1 -- 172.21.15.55:0/2522664838 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208036040 con 0x7f22140c4ad0 2024-05-20T22:10:44.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.468+0000 7f221ab2a640 1 -- 172.21.15.55:0/2522664838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22140c5870 con 0x7f22140c4ad0 2024-05-20T22:10:44.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.470+0000 7f22117fa640 1 -- 172.21.15.55:0/2522664838 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2208043020 con 0x7f22140c4ad0 2024-05-20T22:10:44.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.470+0000 7f22117fa640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f220006be30 0x7f220006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.470+0000 7f22117fa640 1 -- 172.21.15.55:0/2522664838 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f22080ab320 con 0x7f22140c4ad0 2024-05-20T22:10:44.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.470+0000 7f22188c6640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f220006be30 0x7f220006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.471+0000 7f22188c6640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f220006be30 0x7f220006e340 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2204007fd0 tx=0x7f2204009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:44.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.471+0000 7f22219cc740 1 Processor -- start 2024-05-20T22:10:44.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.471+0000 7f22219cc740 1 -- start start 2024-05-20T22:10:44.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.472+0000 7f22219cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c265270 0x55bf0c630700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.472+0000 7f22219cc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c62d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.472+0000 7f22219cc740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c244590 0x55bf0c606360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.472+0000 7f22219cc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c5694d0 con 0x55bf0c23c3b0 2024-05-20T22:10:44.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.472+0000 7f22219cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c5113e0 con 0x55bf0c265270 2024-05-20T22:10:44.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.472+0000 7f22219cc740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bf0c565490 con 0x55bf0c244590 2024-05-20T22:10:44.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f2213fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c62d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f2213fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c62d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50814/0 (socket says 172.21.15.55:50814) 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f2213fff640 1 -- 172.21.15.55:0/591288700 learned_addr learned my addr 172.21.15.55:0/591288700 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f22188c6640 1 --2- 172.21.15.55:0/591288700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c265270 0x55bf0c630700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f22190c7640 1 --2- 172.21.15.55:0/591288700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c244590 0x55bf0c606360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f2213fff640 1 -- 172.21.15.55:0/591288700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c244590 msgr2=0x55bf0c606360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f2213fff640 1 --2- 172.21.15.55:0/591288700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c244590 0x55bf0c606360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f2213fff640 1 -- 172.21.15.55:0/591288700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c265270 msgr2=0x55bf0c630700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f2213fff640 1 --2- 172.21.15.55:0/591288700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c265270 0x55bf0c630700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f2213fff640 1 -- 172.21.15.55:0/591288700 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0c27d2d0 con 0x55bf0c23c3b0 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f22190c7640 1 --2- 172.21.15.55:0/591288700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c244590 0x55bf0c606360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f2213fff640 1 --2- 172.21.15.55:0/591288700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c62d0a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f22080a95d0 tx=0x7f2208039390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528f8874d052ac1b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f21fa7fc640 1 -- 172.21.15.55:0/591288700 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22080793b0 con 0x55bf0c23c3b0 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f21fa7fc640 1 -- 172.21.15.55:0/591288700 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220807d760 con 0x55bf0c23c3b0 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.473+0000 7f21fa7fc640 1 -- 172.21.15.55:0/591288700 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208081f90 con 0x55bf0c23c3b0 2024-05-20T22:10:44.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.474+0000 7f22219cc740 1 -- 172.21.15.55:0/591288700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 msgr2=0x55bf0c62d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.474+0000 7f22219cc740 1 --2- 172.21.15.55:0/591288700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c62d0a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f22080a95d0 tx=0x7f2208039390 comp rx=0 tx=0).stop 2024-05-20T22:10:44.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.474+0000 7f22219cc740 1 -- 172.21.15.55:0/591288700 shutdown_connections 2024-05-20T22:10:44.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.474+0000 7f22219cc740 1 --2- 172.21.15.55:0/591288700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c244590 0x55bf0c606360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.474+0000 7f22219cc740 1 --2- 172.21.15.55:0/591288700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c62d0a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.474+0000 7f22219cc740 1 --2- 172.21.15.55:0/591288700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c265270 0x55bf0c630700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.474+0000 7f22219cc740 1 -- 172.21.15.55:0/591288700 >> 172.21.15.55:0/591288700 conn(0x55bf0c2593c0 msgr2=0x55bf0c605af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:44.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.474+0000 7f22219cc740 1 -- 172.21.15.55:0/591288700 shutdown_connections 2024-05-20T22:10:44.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.474+0000 7f22219cc740 1 -- 172.21.15.55:0/591288700 wait complete. 2024-05-20T22:10:44.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.475+0000 7f22219cc740 1 -- start start 2024-05-20T22:10:44.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.475+0000 7f22219cc740 1 Processor -- start 2024-05-20T22:10:44.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.476+0000 7f22219cc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c633ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22219cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c6313a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22219cc740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c649ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22219cc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c565490 con 0x55bf0c23c3b0 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22219cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c5248e0 con 0x55bf0c244590 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22219cc740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bf0c5694d0 con 0x55bf0c265270 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22219cc740 10 client.0 _ll_register_callbacks cb 0x55bf0c4f19c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22188c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c633ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22188c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c633ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50826/0 (socket says 172.21.15.55:50826) 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22188c6640 1 -- 172.21.15.55:0/4119253792 learned_addr learned my addr 172.21.15.55:0/4119253792 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f2213fff640 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c6313a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22190c7640 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c649ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22188c6640 1 -- 172.21.15.55:0/4119253792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 msgr2=0x55bf0c649ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22188c6640 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c649ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22188c6640 1 -- 172.21.15.55:0/4119253792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 msgr2=0x55bf0c6313a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22188c6640 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c6313a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22188c6640 1 -- 172.21.15.55:0/4119253792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0c3476b0 con 0x55bf0c23c3b0 2024-05-20T22:10:44.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22188c6640 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c633ff0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2204015960 tx=0x7f2204000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d23c28038de5a30 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f21f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f21f37fe640 1 -- 172.21.15.55:0/4119253792 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204026070 con 0x55bf0c23c3b0 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f21f37fe640 1 -- 172.21.15.55:0/4119253792 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f2204027ae0 con 0x55bf0c23c3b0 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.477+0000 7f22219cc740 1 -- 172.21.15.55:0/4119253792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f220807d760 con 0x55bf0c23c3b0 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f21f37fe640 1 -- 172.21.15.55:0/4119253792 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2204032050 con 0x55bf0c23c3b0 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f22219cc740 10 client.6231 fetch_fsmap learned FSMap version 43 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f22219cc740 1 -- 172.21.15.55:0/4119253792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x55bf0c6318e0 con 0x55bf0c23c3b0 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f21f37fe640 1 -- 172.21.15.55:0/4119253792 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220807d760 con 0x55bf0c23c3b0 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f21f37fe640 1 -- 172.21.15.55:0/4119253792 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f2204035020 con 0x55bf0c23c3b0 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f22219cc740 10 client.6231 fetch_fsmap finished waiting for FSMap version 43 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f22219cc740 10 client.6231 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f22219cc740 20 client.6231 populate_metadata read hostname 'smithi055' 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f22219cc740 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.55:6836/3463078364,v1:172.21.15.55:6837/3463078364] conn(0x55bf0c223e40 0x55bf0c65db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f22219cc740 4 client.6231 mds_command: new command op to 6189 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f22219cc740 1 -- 172.21.15.55:0/4119253792 --> [v2:172.21.15.55:6836/3463078364,v1:172.21.15.55:6837/3463078364] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf0c576c00 con 0x55bf0c223e40 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f2213fff640 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.55:6836/3463078364,v1:172.21.15.55:6837/3463078364] conn(0x55bf0c223e40 0x55bf0c65db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f21f37fe640 10 client.6231 ms_handle_connect on v2:172.21.15.55:6836/3463078364 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.478+0000 7f2213fff640 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.55:6836/3463078364,v1:172.21.15.55:6837/3463078364] conn(0x55bf0c223e40 0x55bf0c65db30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c335c4a3f12dda7c server_cookie=d44fb4a0a87bc229 in_seq=0 out_seq=0 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f21f37fe640 1 -- 172.21.15.55:0/4119253792 <== mds.0 v2:172.21.15.55:6836/3463078364 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf0c576c00 con 0x55bf0c223e40 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f21f37fe640 10 client.6231 handle_command_reply: tid=0 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f22219cc740 1 client.6231 shutdown 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f22219cc740 1 -- 172.21.15.55:0/4119253792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 msgr2=0x55bf0c633ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f22219cc740 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c633ff0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2204015960 tx=0x7f2204000fe0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f22219cc740 1 -- 172.21.15.55:0/4119253792 shutdown_connections 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f22219cc740 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c649ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f22219cc740 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.55:6836/3463078364,v1:172.21.15.55:6837/3463078364] conn(0x55bf0c223e40 0x55bf0c65db30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f22219cc740 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c6313a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f22219cc740 1 --2- 172.21.15.55:0/4119253792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c633ff0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f21f37fe640 0 client.6231 ms_handle_reset on v2:172.21.15.55:6836/3463078364 2024-05-20T22:10:44.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.479+0000 7f22219cc740 1 -- 172.21.15.55:0/4119253792 >> 172.21.15.55:0/4119253792 conn(0x55bf0c2593c0 msgr2=0x55bf0c605af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:44.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.480+0000 7f22219cc740 1 -- 172.21.15.55:0/4119253792 shutdown_connections 2024-05-20T22:10:44.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.480+0000 7f22219cc740 1 -- 172.21.15.55:0/4119253792 wait complete. 2024-05-20T22:10:44.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.480+0000 7f22219cc740 20 client.6231 trim_cache size 0 max 16384 2024-05-20T22:10:44.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.499+0000 7f22219cc740 1 Processor -- start 2024-05-20T22:10:44.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.499+0000 7f22219cc740 1 -- start start 2024-05-20T22:10:44.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22219cc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c630700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22219cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c630c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22219cc740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c64d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22219cc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2214201a60 con 0x55bf0c23c3b0 2024-05-20T22:10:44.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22219cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2214200e40 con 0x55bf0c244590 2024-05-20T22:10:44.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22219cc740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2214201410 con 0x55bf0c265270 2024-05-20T22:10:44.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22188c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c630700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22188c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c630700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50842/0 (socket says 172.21.15.55:50842) 2024-05-20T22:10:44.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22188c6640 1 -- 172.21.15.55:0/897723606 learned_addr learned my addr 172.21.15.55:0/897723606 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:44.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f2213fff640 1 --2- 172.21.15.55:0/897723606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c630c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22188c6640 1 -- 172.21.15.55:0/897723606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 msgr2=0x55bf0c64d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22188c6640 1 --2- 172.21.15.55:0/897723606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c64d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22188c6640 1 -- 172.21.15.55:0/897723606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 msgr2=0x55bf0c630c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22188c6640 1 --2- 172.21.15.55:0/897723606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c630c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f22188c6640 1 -- 172.21.15.55:0/897723606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0c631710 con 0x55bf0c23c3b0 2024-05-20T22:10:44.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.500+0000 7f2213fff640 1 --2- 172.21.15.55:0/897723606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c630c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:44.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.501+0000 7f22188c6640 1 --2- 172.21.15.55:0/897723606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c630700 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2204014dc0 tx=0x7f220401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a463f72bac636ae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:44.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.501+0000 7f21f9ffb640 1 -- 172.21.15.55:0/897723606 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204026070 con 0x55bf0c23c3b0 2024-05-20T22:10:44.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.501+0000 7f21f9ffb640 1 -- 172.21.15.55:0/897723606 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220402c650 con 0x55bf0c23c3b0 2024-05-20T22:10:44.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.501+0000 7f21f9ffb640 1 -- 172.21.15.55:0/897723606 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220402ca00 con 0x55bf0c23c3b0 2024-05-20T22:10:44.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.501+0000 7f22219cc740 1 -- 172.21.15.55:0/897723606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 msgr2=0x55bf0c630700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.501+0000 7f22219cc740 1 --2- 172.21.15.55:0/897723606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c630700 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2204014dc0 tx=0x7f220401d000 comp rx=0 tx=0).stop 2024-05-20T22:10:44.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.501+0000 7f22219cc740 1 -- 172.21.15.55:0/897723606 shutdown_connections 2024-05-20T22:10:44.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.501+0000 7f22219cc740 1 --2- 172.21.15.55:0/897723606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c64d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.501+0000 7f22219cc740 1 --2- 172.21.15.55:0/897723606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c630c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.501+0000 7f22219cc740 1 --2- 172.21.15.55:0/897723606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c630700 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.502+0000 7f22219cc740 1 -- 172.21.15.55:0/897723606 >> 172.21.15.55:0/897723606 conn(0x55bf0c2593c0 msgr2=0x55bf0c64ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:44.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.502+0000 7f22219cc740 1 -- 172.21.15.55:0/897723606 shutdown_connections 2024-05-20T22:10:44.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.502+0000 7f22219cc740 1 -- 172.21.15.55:0/897723606 wait complete. 2024-05-20T22:10:44.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.502+0000 7f22219cc740 1 -- start start 2024-05-20T22:10:44.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.503+0000 7f22219cc740 1 Processor -- start 2024-05-20T22:10:44.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22219cc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c6c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22219cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c62e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22219cc740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c6cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22219cc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c64a160 con 0x55bf0c23c3b0 2024-05-20T22:10:44.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22219cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c5113e0 con 0x55bf0c244590 2024-05-20T22:10:44.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22219cc740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bf0c64ab50 con 0x55bf0c265270 2024-05-20T22:10:44.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22219cc740 10 client.0 _ll_register_callbacks cb 0x7f22040076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T22:10:44.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22188c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c6c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22188c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c6c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50850/0 (socket says 172.21.15.55:50850) 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22188c6640 1 -- 172.21.15.55:0/91281855 learned_addr learned my addr 172.21.15.55:0/91281855 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f2213fff640 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c62e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22188c6640 1 -- 172.21.15.55:0/91281855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 msgr2=0x55bf0c6cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22190c7640 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c6cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22188c6640 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c6cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22188c6640 1 -- 172.21.15.55:0/91281855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 msgr2=0x55bf0c62e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22188c6640 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c62e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f22188c6640 1 -- 172.21.15.55:0/91281855 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0c633dc0 con 0x55bf0c23c3b0 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.504+0000 7f2213fff640 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c62e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f22188c6640 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c6c7ff0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2204000ef0 tx=0x7f2204016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f6be5e4abfd378 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f21f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-20T22:10:44.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f21f37fe640 1 -- 172.21.15.55:0/91281855 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204043020 con 0x55bf0c23c3b0 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f21f37fe640 1 -- 172.21.15.55:0/91281855 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f220402c450 con 0x55bf0c23c3b0 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f22219cc740 1 -- 172.21.15.55:0/91281855 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f220402c650 con 0x55bf0c23c3b0 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f21f37fe640 1 -- 172.21.15.55:0/91281855 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f220404d050 con 0x55bf0c23c3b0 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f22219cc740 10 client.6237 fetch_fsmap learned FSMap version 43 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f22219cc740 1 -- 172.21.15.55:0/91281855 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x55bf0c6c8530 con 0x55bf0c23c3b0 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f21f37fe640 1 -- 172.21.15.55:0/91281855 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220402c650 con 0x55bf0c23c3b0 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f21f37fe640 1 -- 172.21.15.55:0/91281855 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f2204050020 con 0x55bf0c23c3b0 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f22219cc740 10 client.6237 fetch_fsmap finished waiting for FSMap version 43 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f22219cc740 10 client.6237 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f22219cc740 20 client.6237 populate_metadata read hostname 'smithi055' 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f22219cc740 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.55:6836/3463078364,v1:172.21.15.55:6837/3463078364] conn(0x55bf0c6cb4e0 0x55bf0c646d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f22219cc740 4 client.6237 mds_command: new command op to 6189 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.505+0000 7f22219cc740 1 -- 172.21.15.55:0/91281855 --> [v2:172.21.15.55:6836/3463078364,v1:172.21.15.55:6837/3463078364] -- command(tid 0: {"prefix": "status"}) -- 0x55bf0c570f50 con 0x55bf0c6cb4e0 2024-05-20T22:10:44.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f2213fff640 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.55:6836/3463078364,v1:172.21.15.55:6837/3463078364] conn(0x55bf0c6cb4e0 0x55bf0c646d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f21f37fe640 10 client.6237 ms_handle_connect on v2:172.21.15.55:6836/3463078364 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f2213fff640 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.55:6836/3463078364,v1:172.21.15.55:6837/3463078364] conn(0x55bf0c6cb4e0 0x55bf0c646d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f2333f883c7dfc2 server_cookie=97aa42697735961c in_seq=0 out_seq=0 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f21f37fe640 1 -- 172.21.15.55:0/91281855 <== mds.0 v2:172.21.15.55:6836/3463078364 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55bf0c570f50 con 0x55bf0c6cb4e0 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f21f37fe640 10 client.6237 handle_command_reply: tid=0 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f22219cc740 1 client.6237 shutdown 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f22219cc740 1 -- 172.21.15.55:0/91281855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 msgr2=0x55bf0c6c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f22219cc740 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c6c7ff0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2204000ef0 tx=0x7f2204016000 comp rx=0 tx=0).stop 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f22219cc740 1 -- 172.21.15.55:0/91281855 shutdown_connections 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f22219cc740 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf0c265270 0x55bf0c6cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f22219cc740 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.55:6836/3463078364,v1:172.21.15.55:6837/3463078364] conn(0x55bf0c6cb4e0 0x55bf0c646d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f22219cc740 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf0c244590 0x55bf0c62e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f22219cc740 1 --2- 172.21.15.55:0/91281855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bf0c23c3b0 0x55bf0c6c7ff0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f22219cc740 1 -- 172.21.15.55:0/91281855 >> 172.21.15.55:0/91281855 conn(0x55bf0c2593c0 msgr2=0x55bf0c66ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:44.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.506+0000 7f21f37fe640 0 client.6237 ms_handle_reset on v2:172.21.15.55:6836/3463078364 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f22219cc740 1 -- 172.21.15.55:0/91281855 shutdown_connections 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f22219cc740 1 -- 172.21.15.55:0/91281855 wait complete. 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f22219cc740 20 client.6237 trim_cache size 0 max 16384 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "de8ca569-3fd1-4cb2-8bdc-99f367745927", 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 6189, 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-20T22:10:44.513 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 6.1988138460000002, 2024-05-20T22:10:44.514 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 43, 2024-05-20T22:10:44.514 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 61, 2024-05-20T22:10:44.514 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 61, 2024-05-20T22:10:44.514 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 6.2123127120000001 2024-05-20T22:10:44.514 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 -- 172.21.15.55:0/2522664838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f220006be30 msgr2=0x7f220006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f220006be30 0x7f220006e340 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2204007fd0 tx=0x7f2204009040 comp rx=0 tx=0).stop 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 -- 172.21.15.55:0/2522664838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22140c4ad0 msgr2=0x7f22140ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22140c4ad0 0x7f22140ae410 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2208034040 tx=0x7f2208004730 comp rx=0 tx=0).stop 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 -- 172.21.15.55:0/2522664838 shutdown_connections 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f220006be30 0x7f220006e340 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22140ae950 0x7f2214160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22140c4ad0 0x7f22140ae410 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 --2- 172.21.15.55:0/2522664838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21fc003f50 0x7f2214151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 -- 172.21.15.55:0/2522664838 >> 172.21.15.55:0/2522664838 conn(0x7f221409a400 msgr2=0x7f221406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 -- 172.21.15.55:0/2522664838 shutdown_connections 2024-05-20T22:10:44.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.507+0000 7f221ab2a640 1 -- 172.21.15.55:0/2522664838 wait complete. 2024-05-20T22:10:44.519 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:44.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.633+0000 7ffbf01e3640 1 Processor -- start 2024-05-20T22:10:44.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.633+0000 7ffbf01e3640 1 -- start start 2024-05-20T22:10:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbf01e3640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffbe80d3d80 0x7ffbe80cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbf01e3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe815ccc0 0x7ffbe80b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbf01e3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbe8160550 con 0x7ffbe80d42a0 2024-05-20T22:10:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbf01e3640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7ffbe815f9d0 con 0x7ffbe80d3d80 2024-05-20T22:10:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbf01e3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe815fcb0 con 0x7ffbe815ccc0 2024-05-20T22:10:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbed77e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffbe80d3d80 0x7ffbe80cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbed77e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffbe80d3d80 0x7ffbe80cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50858/0 (socket says 172.21.15.55:50858) 2024-05-20T22:10:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbed77e640 1 -- 172.21.15.55:0/2712095152 learned_addr learned my addr 172.21.15.55:0/2712095152 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbee780640 1 --2- 172.21.15.55:0/2712095152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe815ccc0 0x7ffbe80b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbed77e640 1 -- 172.21.15.55:0/2712095152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe815ccc0 msgr2=0x7ffbe80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbed77e640 1 --2- 172.21.15.55:0/2712095152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe815ccc0 0x7ffbe80b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbed77e640 1 -- 172.21.15.55:0/2712095152 >> v1:172.21.15.138:6789/0 conn(0x7ffbe80d42a0 legacy=0x7ffbe80d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.635+0000 7ffbed77e640 1 -- 172.21.15.55:0/2712095152 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe815ff90 con 0x7ffbe80d3d80 2024-05-20T22:10:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.636+0000 7ffbed77e640 1 --2- 172.21.15.55:0/2712095152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe80d3d80 0x7ffbe80cf710 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009870 tx=0x7ffbd802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bb36a09f14b485 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.636+0000 7ffbecf7d640 1 -- 172.21.15.55:0/2712095152 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8037070 con 0x7ffbe80d3d80 2024-05-20T22:10:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.636+0000 7ffbecf7d640 1 -- 172.21.15.55:0/2712095152 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd8032440 con 0x7ffbe80d3d80 2024-05-20T22:10:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.636+0000 7ffbecf7d640 1 -- 172.21.15.55:0/2712095152 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8032810 con 0x7ffbe80d3d80 2024-05-20T22:10:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.637+0000 7ffbf01e3640 1 -- 172.21.15.55:0/2712095152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe80d3d80 msgr2=0x7ffbe80cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.637+0000 7ffbf01e3640 1 --2- 172.21.15.55:0/2712095152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe80d3d80 0x7ffbe80cf710 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009870 tx=0x7ffbd802a1c0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.637+0000 7ffbf01e3640 1 -- 172.21.15.55:0/2712095152 shutdown_connections 2024-05-20T22:10:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.637+0000 7ffbf01e3640 1 --2- 172.21.15.55:0/2712095152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe815ccc0 0x7ffbe80b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.637+0000 7ffbf01e3640 1 --2- 172.21.15.55:0/2712095152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe80d3d80 0x7ffbe80cf710 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.637+0000 7ffbf01e3640 1 -- 172.21.15.55:0/2712095152 >> 172.21.15.55:0/2712095152 conn(0x7ffbe80d1c50 msgr2=0x7ffbe80d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:44.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.637+0000 7ffbf01e3640 1 -- 172.21.15.55:0/2712095152 shutdown_connections 2024-05-20T22:10:44.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.637+0000 7ffbf01e3640 1 -- 172.21.15.55:0/2712095152 wait complete. 2024-05-20T22:10:44.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.639+0000 7ffbf01e3640 1 Processor -- start 2024-05-20T22:10:44.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.639+0000 7ffbf01e3640 1 -- start start 2024-05-20T22:10:44.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbf01e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe815ccc0 0x7ffbe80cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbf01e3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe8150b10 0x7ffbe81f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbf01e3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe8162930 0x7ffbe8162d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbf01e3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe815f7b0 con 0x7ffbe8150b10 2024-05-20T22:10:44.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbf01e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe815f630 con 0x7ffbe815ccc0 2024-05-20T22:10:44.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbf01e3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe80b3aa0 con 0x7ffbe8162930 2024-05-20T22:10:44.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbed77e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe8150b10 0x7ffbe81f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbed77e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe8150b10 0x7ffbe81f42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50872/0 (socket says 172.21.15.55:50872) 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbed77e640 1 -- 172.21.15.55:0/1849649913 learned_addr learned my addr 172.21.15.55:0/1849649913 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbedf7f640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe815ccc0 0x7ffbe80cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbee780640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe8162930 0x7ffbe8162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbed77e640 1 -- 172.21.15.55:0/1849649913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe8162930 msgr2=0x7ffbe8162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbed77e640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe8162930 0x7ffbe8162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbed77e640 1 -- 172.21.15.55:0/1849649913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe815ccc0 msgr2=0x7ffbe80cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbed77e640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe815ccc0 0x7ffbe80cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbed77e640 1 -- 172.21.15.55:0/1849649913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe8163280 con 0x7ffbe8150b10 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbee780640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe8162930 0x7ffbe8162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.640+0000 7ffbedf7f640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe815ccc0 0x7ffbe80cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.641+0000 7ffbed77e640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe8150b10 0x7ffbe81f42b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000c00 tx=0x7ffbd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.641+0000 7ffbde7fc640 1 -- 172.21.15.55:0/1849649913 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8037040 con 0x7ffbe8150b10 2024-05-20T22:10:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.641+0000 7ffbde7fc640 1 -- 172.21.15.55:0/1849649913 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd8042070 con 0x7ffbe8150b10 2024-05-20T22:10:44.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.641+0000 7ffbf01e3640 1 -- 172.21.15.55:0/1849649913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe80cebd0 con 0x7ffbe8150b10 2024-05-20T22:10:44.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.641+0000 7ffbde7fc640 1 -- 172.21.15.55:0/1849649913 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8030880 con 0x7ffbe8150b10 2024-05-20T22:10:44.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.641+0000 7ffbf01e3640 1 -- 172.21.15.55:0/1849649913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe80cf190 con 0x7ffbe8150b10 2024-05-20T22:10:44.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.642+0000 7ffbf01e3640 1 -- 172.21.15.55:0/1849649913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe804f720 con 0x7ffbe8150b10 2024-05-20T22:10:44.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.642+0000 7ffbde7fc640 1 -- 172.21.15.55:0/1849649913 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbd802ad10 con 0x7ffbe8150b10 2024-05-20T22:10:44.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.643+0000 7ffbde7fc640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ffbcc06be30 0x7ffbcc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:44.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.643+0000 7ffbde7fc640 1 -- 172.21.15.55:0/1849649913 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7ffbd80ab6f0 con 0x7ffbe8150b10 2024-05-20T22:10:44.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.643+0000 7ffbedf7f640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ffbcc06be30 0x7ffbcc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:44.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.643+0000 7ffbedf7f640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ffbcc06be30 0x7ffbcc06e340 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40015e0 tx=0x7ffbe4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.647+0000 7ffbde7fc640 1 -- 172.21.15.55:0/1849649913 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffbe8163280 con 0x7ffbe8150b10 2024-05-20T22:10:44.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.870+0000 7ffbf01e3640 1 -- 172.21.15.55:0/1849649913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbe80d2ee0 con 0x7ffbe8150b10 2024-05-20T22:10:44.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.870+0000 7ffbde7fc640 1 -- 172.21.15.55:0/1849649913 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7ffbd80a9e20 con 0x7ffbe8150b10 2024-05-20T22:10:44.871 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:44.871 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:10:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.872+0000 7ffbf01e3640 1 -- 172.21.15.55:0/1849649913 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ffbcc06be30 msgr2=0x7ffbcc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.872+0000 7ffbf01e3640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ffbcc06be30 0x7ffbcc06e340 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40015e0 tx=0x7ffbe4009040 comp rx=0 tx=0).stop 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.872+0000 7ffbf01e3640 1 -- 172.21.15.55:0/1849649913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe8150b10 msgr2=0x7ffbe81f42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.872+0000 7ffbf01e3640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe8150b10 0x7ffbe81f42b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000c00 tx=0x7ffbd80046c0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.873+0000 7ffbf01e3640 1 -- 172.21.15.55:0/1849649913 shutdown_connections 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.873+0000 7ffbf01e3640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ffbcc06be30 0x7ffbcc06e340 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.873+0000 7ffbf01e3640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbe8162930 0x7ffbe8162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.873+0000 7ffbf01e3640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbe8150b10 0x7ffbe81f42b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.873+0000 7ffbf01e3640 1 --2- 172.21.15.55:0/1849649913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe815ccc0 0x7ffbe80cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.873+0000 7ffbf01e3640 1 -- 172.21.15.55:0/1849649913 >> 172.21.15.55:0/1849649913 conn(0x7ffbe80d1c50 msgr2=0x7ffbe80d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:44.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.873+0000 7ffbf01e3640 1 -- 172.21.15.55:0/1849649913 shutdown_connections 2024-05-20T22:10:44.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:44.873+0000 7ffbf01e3640 1 -- 172.21.15.55:0/1849649913 wait complete. 2024-05-20T22:10:44.885 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-20T22:10:45.016 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":23,"msgr_send_messages":22,"msgr_recv_bytes":13418,"msgr_send_bytes":30281,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.005120961,"msgr_running_send_time":0.000663172,"msgr_running_recv_time":0.002616327,"msgr_running_fast_dispatch_time":0.000476521,"msgr_send_messages_queue_lat":{"avgcount":21,"sum":0.003743879,"avgtime":0.000178279},"msgr_handle_ack_lat":{"avgcount":12,"sum":0.000002503,"avgtime":0.000000208},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":10,"msgr_send_messages":14,"msgr_recv_bytes":4136,"msgr_send_bytes":94119,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004813924,"msgr_running_send_time":0.000635399,"msgr_running_recv_time":0.003799807,"msgr_running_fast_dispatch_time":0.000224956,"msgr_send_messages_queue_lat":{"avgcount":12,"sum":0.003959921,"avgtime":0.000329993},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":90832},"AsyncMessenger::Worker-2":{"msgr_recv_messages":47,"msgr_send_messages":40,"msgr_recv_bytes":123748,"msgr_send_bytes":15579,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.007983242,"msgr_running_send_time":0.001267927,"msgr_running_recv_time":0.007148390,"msgr_running_fast_dispatch_time":0.000726834,"msgr_send_messages_queue_lat":{"avgcount":40,"sum":0.007858365,"avgtime":0.000196459},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":109984,"msgr_send_encrypted_bytes":5952},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":47,"sum":0.003288908,"avgtime":0.000069976}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000101017,"avgtime":0.000025254}},"mds":{"request":3,"reply":1,"reply_latency":{"avgcount":1,"sum":0.000538898,"avgtime":0.000538898},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":7,"inodes_bottom":3,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":3,"traverse_hit":3,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":0},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":2,"evex":0,"evtrm":0,"ev":5,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4196616,"rdpos":4195244,"jlat":{"avgcount":2,"sum":0.003776592,"avgtime":0.001888296},"replayed":3},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":1,"cap+":1,"cap-":0,"rss":109844,"heap":248188},"mds_server":{"dispatch_client_request":2,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":1,"sum":0.000538898,"avgtime":0.000538898},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":667,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":237239,"buffer_anon_items":68,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5848,"osdmap_items":95,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52320,"mds_co_items":189,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":46,"op_send_bytes":1469,"op_resend":0,"op_reply":46,"op_latency":{"avgcount":46,"sum":0.066998721,"avgtime":0.001456493},"op_inflight":0,"oplen_avg":{"avgcount":46,"sum":92},"op":46,"op_r":36,"op_w":10,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":2,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":1,"omap_rd":48,"omap_del":0},"oft":{"omap_total_objs":0,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":75,"get_sum":131914,"get_or_fail_fail":0,"get_or_fail_success":75,"take":0,"take_sum":0,"put":75,"put_sum":131914,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":2586,"put":46,"put_sum":2586,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":46,"put":46,"put_sum":46,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ca9a045ba0":{"val":0,"max":3758096384,"get_started":0,"get":2,"get_sum":931,"get_or_fail_fail":0,"get_or_fail_success":2,"take":0,"take_sum":0,"put":2,"put_sum":931,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-20T22:10:45.018 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 12, "avgtime": 2.08e-07, "sum": 2.503e-06 }, "msgr_recv_bytes": 13418, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 23, "msgr_running_fast_dispatch_time": 0.000476521, "msgr_running_recv_time": 0.002616327, "msgr_running_send_time": 0.000663172, "msgr_running_total_time": 0.005120961, "msgr_send_bytes": 30281, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 22, "msgr_send_messages_queue_lat": { "avgcount": 21, "avgtime": 0.000178279, "sum": 0.003743879 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 4136, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 10, "msgr_running_fast_dispatch_time": 0.000224956, "msgr_running_recv_time": 0.003799807, "msgr_running_send_time": 0.000635399, "msgr_running_total_time": 0.004813924, "msgr_send_bytes": 94119, "msgr_send_encrypted_bytes": 90832, "msgr_send_messages": 14, "msgr_send_messages_queue_lat": { "avgcount": 12, "avgtime": 0.000329993, "sum": 0.003959921 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 123748, "msgr_recv_encrypted_bytes": 109984, "msgr_recv_messages": 47, "msgr_running_fast_dispatch_time": 0.000726834, "msgr_running_recv_time": 0.00714839, "msgr_running_send_time": 0.001267927, "msgr_running_total_time": 0.007983242, "msgr_send_bytes": 15579, "msgr_send_encrypted_bytes": 5952, "msgr_send_messages": 40, "msgr_send_messages_queue_lat": { "avgcount": 40, "avgtime": 0.000196459, "sum": 0.007858365 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 47, "avgtime": 6.9976e-05, "sum": 0.003288908 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.5254e-05, "sum": 0.000101017 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 3, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 7, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 0, "q": 0, "reply": 1, "reply_latency": { "avgcount": 1, "avgtime": 0.000538898, "sum": 0.000538898 }, "request": 3, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 3, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 3, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 5, "evadd": 2, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 2, "avgtime": 0.001888296, "sum": 0.003776592 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4196616 }, "mds_mem": { "cap": 1, "cap+": 1, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 248188, "ino": 13, "ino+": 13, "ino-": 0, "rss": 109844 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 2, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000538898, "sum": 0.000538898 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 667, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 237239, "buffer_anon_items": 68, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52320, "mds_co_items": 189, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5848, "osdmap_items": 95, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 0, "omap_rd": 48, "omap_wr": 1, "op": 46, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 46, "avgtime": 0.001456493, "sum": 0.066998721 }, "op_pg": 0, "op_r": 36, "op_reply": 46, "op_resend": 0, "op_rmw": 0, "op_send": 46, "op_send_bytes": 1469, "op_w": 10, "oplen_avg": { "avgcount": 46, "sum": 92 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 0, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 75, "get_or_fail_fail": 0, "get_or_fail_success": 75, "get_started": 0, "get_sum": 131914, "max": 104857600, "put": 75, "put_sum": 131914, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 46, "put_sum": 2586, "take": 46, "take_sum": 2586, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 46, "put_sum": 46, "take": 46, "take_sum": 46, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ca9a045ba0": { "get": 2, "get_or_fail_fail": 0, "get_or_fail_success": 2, "get_started": 0, "get_sum": 931, "max": 3758096384, "put": 2, "put_sum": 931, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-20T22:10:45.018 INFO:tasks.cephfs.test_openfiletable:omap_total_updates_0:0 2024-05-20T22:10:45.018 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:10:45.018 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:10:45.035 DEBUG:teuthology.orchestra.run.smithi055:> stdin-killer -- python3 -c ' 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> import fcntl 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> import sys 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> import time 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> with open("/home/ubuntu/cephtest/mnt.0/omap_counter_test_file", '"'"'w'"'"') as f: 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> f.write("content") 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> f.flush() 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> while True: 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> print("open_background: keeping file open", file=sys.stderr) 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> try: 2024-05-20T22:10:45.036 DEBUG:teuthology.orchestra.run.smithi055:> if os.read(0, 4096) == b"": 2024-05-20T22:10:45.037 DEBUG:teuthology.orchestra.run.smithi055:> break 2024-05-20T22:10:45.037 DEBUG:teuthology.orchestra.run.smithi055:> except BlockingIOError: 2024-05-20T22:10:45.037 DEBUG:teuthology.orchestra.run.smithi055:> pass 2024-05-20T22:10:45.037 DEBUG:teuthology.orchestra.run.smithi055:> time.sleep(2) 2024-05-20T22:10:45.037 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-20T22:10:45.079 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/omap_counter_test_file 2024-05-20T22:10:45.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:10:45.095 INFO:teuthology.orchestra.run.smithi055.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/omap_counter_test_file': No such file or directory 2024-05-20T22:10:45.150 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:46.096 DEBUG:teuthology.orchestra.run.smithi055:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/omap_counter_test_file 2024-05-20T22:10:46.116 INFO:teuthology.orchestra.run.smithi055.stdout:7 2024-05-20T22:10:46.117 INFO:tasks.cephfs.mount:File omap_counter_test_file became visible with size 7 from 0 after 1s 2024-05-20T22:10:46.117 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:46.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.282+0000 7ff756fb7640 1 Processor -- start 2024-05-20T22:10:46.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.282+0000 7ff756fb7640 1 -- start start 2024-05-20T22:10:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff756fb7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7500d42a0 0x7ff7500cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff756fb7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7500d3d80 0x7ff7500b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff756fb7640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff750160550 con 0x7ff75015ccc0 2024-05-20T22:10:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff756fb7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff75015f9d0 con 0x7ff7500d42a0 2024-05-20T22:10:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff756fb7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff75015fcb0 con 0x7ff7500d3d80 2024-05-20T22:10:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff755554640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7500d3d80 0x7ff7500b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff754d53640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7500d42a0 0x7ff7500cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff755554640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7500d3d80 0x7ff7500b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.55:55946/0 (socket says 172.21.15.55:55946) 2024-05-20T22:10:46.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff74ffff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff75015ccc0 0x7ff7500d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:40588/0 (socket says 172.21.15.55:40588) 2024-05-20T22:10:46.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff755554640 1 -- 172.21.15.55:0/705111075 learned_addr learned my addr 172.21.15.55:0/705111075 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:46.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff755554640 1 -- 172.21.15.55:0/705111075 >> v2:172.21.15.138:3300/0 conn(0x7ff7500d42a0 msgr2=0x7ff7500cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:46.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff755554640 1 --2- 172.21.15.55:0/705111075 >> v2:172.21.15.138:3300/0 conn(0x7ff7500d42a0 0x7ff7500cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff755554640 1 -- 172.21.15.55:0/705111075 >> v1:172.21.15.138:6789/0 conn(0x7ff75015ccc0 legacy=0x7ff7500d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:46.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.284+0000 7ff755554640 1 -- 172.21.15.55:0/705111075 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75015ff90 con 0x7ff7500d3d80 2024-05-20T22:10:46.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.285+0000 7ff755554640 1 --2- 172.21.15.55:0/705111075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7500d3d80 0x7ff7500b3560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff748009e50 tx=0x7ff74802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d007d57b24850c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:46.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.285+0000 7ff74f7fe640 1 -- 172.21.15.55:0/705111075 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748036070 con 0x7ff7500d3d80 2024-05-20T22:10:46.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.285+0000 7ff74f7fe640 1 -- 172.21.15.55:0/705111075 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff748031600 con 0x7ff7500d3d80 2024-05-20T22:10:46.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.285+0000 7ff74f7fe640 1 -- 172.21.15.55:0/705111075 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7480319d0 con 0x7ff7500d3d80 2024-05-20T22:10:46.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.286+0000 7ff756fb7640 1 -- 172.21.15.55:0/705111075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7500d3d80 msgr2=0x7ff7500b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:46.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.286+0000 7ff756fb7640 1 --2- 172.21.15.55:0/705111075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7500d3d80 0x7ff7500b3560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff748009e50 tx=0x7ff74802a680 comp rx=0 tx=0).stop 2024-05-20T22:10:46.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.286+0000 7ff756fb7640 1 -- 172.21.15.55:0/705111075 shutdown_connections 2024-05-20T22:10:46.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.286+0000 7ff756fb7640 1 --2- 172.21.15.55:0/705111075 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7500d3d80 0x7ff7500b3560 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.286+0000 7ff756fb7640 1 --2- 172.21.15.55:0/705111075 >> v2:172.21.15.138:3300/0 conn(0x7ff7500d42a0 0x7ff7500cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.286+0000 7ff756fb7640 1 -- 172.21.15.55:0/705111075 >> 172.21.15.55:0/705111075 conn(0x7ff7500d1c50 msgr2=0x7ff7500d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:46.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.286+0000 7ff756fb7640 1 -- 172.21.15.55:0/705111075 shutdown_connections 2024-05-20T22:10:46.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.286+0000 7ff756fb7640 1 -- 172.21.15.55:0/705111075 wait complete. 2024-05-20T22:10:46.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.288+0000 7ff756fb7640 1 Processor -- start 2024-05-20T22:10:46.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.288+0000 7ff756fb7640 1 -- start start 2024-05-20T22:10:46.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.289+0000 7ff756fb7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff75015ccc0 0x7ff750155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.289+0000 7ff756fb7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff750156fd0 0x7ff7501573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.289+0000 7ff756fb7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff750158f20 0x7ff750159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.289+0000 7ff756fb7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff75015f7b0 con 0x7ff750156fd0 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.289+0000 7ff756fb7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff75015f630 con 0x7ff75015ccc0 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.289+0000 7ff756fb7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff7500b3aa0 con 0x7ff750158f20 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff750156fd0 0x7ff7501573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff750156fd0 0x7ff7501573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50878/0 (socket says 172.21.15.55:50878) 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74ffff640 1 -- 172.21.15.55:0/3119451975 learned_addr learned my addr 172.21.15.55:0/3119451975 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff755554640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff750158f20 0x7ff750159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff754d53640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff75015ccc0 0x7ff750155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74ffff640 1 -- 172.21.15.55:0/3119451975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff750158f20 msgr2=0x7ff750159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74ffff640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff750158f20 0x7ff750159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74ffff640 1 -- 172.21.15.55:0/3119451975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff75015ccc0 msgr2=0x7ff750155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:46.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74ffff640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff75015ccc0 0x7ff750155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74ffff640 1 -- 172.21.15.55:0/3119451975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7500cb650 con 0x7ff750156fd0 2024-05-20T22:10:46.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74ffff640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff750156fd0 0x7ff7501573e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff74400c3f0 tx=0x7ff74400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:46.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74d7fa640 1 -- 172.21.15.55:0/3119451975 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7440074e0 con 0x7ff750156fd0 2024-05-20T22:10:46.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff74d7fa640 1 -- 172.21.15.55:0/3119451975 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff744007680 con 0x7ff750156fd0 2024-05-20T22:10:46.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.290+0000 7ff756fb7640 1 -- 172.21.15.55:0/3119451975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7500c86c0 con 0x7ff750156fd0 2024-05-20T22:10:46.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.291+0000 7ff74d7fa640 1 -- 172.21.15.55:0/3119451975 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744004a00 con 0x7ff750156fd0 2024-05-20T22:10:46.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.291+0000 7ff756fb7640 1 -- 172.21.15.55:0/3119451975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff750159870 con 0x7ff750156fd0 2024-05-20T22:10:46.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.291+0000 7ff756fb7640 1 -- 172.21.15.55:0/3119451975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75004f720 con 0x7ff750156fd0 2024-05-20T22:10:46.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.292+0000 7ff74d7fa640 1 -- 172.21.15.55:0/3119451975 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff74401a020 con 0x7ff750156fd0 2024-05-20T22:10:46.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.293+0000 7ff74d7fa640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff73c06be30 0x7ff73c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.293+0000 7ff74d7fa640 1 -- 172.21.15.55:0/3119451975 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7ff74408a730 con 0x7ff750156fd0 2024-05-20T22:10:46.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.293+0000 7ff754d53640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff73c06be30 0x7ff73c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.293+0000 7ff754d53640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff73c06be30 0x7ff73c06e340 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff740009aa0 tx=0x7ff740009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:46.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.297+0000 7ff74d7fa640 1 -- 172.21.15.55:0/3119451975 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff7500cb650 con 0x7ff750156fd0 2024-05-20T22:10:46.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.523+0000 7ff756fb7640 1 -- 172.21.15.55:0/3119451975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7501579e0 con 0x7ff750156fd0 2024-05-20T22:10:46.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.523+0000 7ff74d7fa640 1 -- 172.21.15.55:0/3119451975 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7ff7440587d0 con 0x7ff750156fd0 2024-05-20T22:10:46.524 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:46.524 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:10:46.525 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:10:46.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 -- 172.21.15.55:0/3119451975 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff73c06be30 msgr2=0x7ff73c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:46.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff73c06be30 0x7ff73c06e340 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff740009aa0 tx=0x7ff740009190 comp rx=0 tx=0).stop 2024-05-20T22:10:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 -- 172.21.15.55:0/3119451975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff750156fd0 msgr2=0x7ff7501573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff750156fd0 0x7ff7501573e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff74400c3f0 tx=0x7ff74400c8b0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 -- 172.21.15.55:0/3119451975 shutdown_connections 2024-05-20T22:10:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7ff73c06be30 0x7ff73c06e340 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff750158f20 0x7ff750159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff750156fd0 0x7ff7501573e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 --2- 172.21.15.55:0/3119451975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff75015ccc0 0x7ff750155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 -- 172.21.15.55:0/3119451975 >> 172.21.15.55:0/3119451975 conn(0x7ff7500d1c50 msgr2=0x7ff75015b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 -- 172.21.15.55:0/3119451975 shutdown_connections 2024-05-20T22:10:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.525+0000 7ff756fb7640 1 -- 172.21.15.55:0/3119451975 wait complete. 2024-05-20T22:10:46.538 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-20T22:10:46.669 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":30,"msgr_send_messages":27,"msgr_recv_bytes":15904,"msgr_send_bytes":33304,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.005699930,"msgr_running_send_time":0.000855460,"msgr_running_recv_time":0.002829143,"msgr_running_fast_dispatch_time":0.000651718,"msgr_send_messages_queue_lat":{"avgcount":26,"sum":0.003844234,"avgtime":0.000147855},"msgr_handle_ack_lat":{"avgcount":16,"sum":0.000003504,"avgtime":0.000000219},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":10,"msgr_send_messages":14,"msgr_recv_bytes":4136,"msgr_send_bytes":94119,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004813924,"msgr_running_send_time":0.000635399,"msgr_running_recv_time":0.003799807,"msgr_running_fast_dispatch_time":0.000224956,"msgr_send_messages_queue_lat":{"avgcount":12,"sum":0.003959921,"avgtime":0.000329993},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":90832},"AsyncMessenger::Worker-2":{"msgr_recv_messages":49,"msgr_send_messages":42,"msgr_recv_bytes":124266,"msgr_send_bytes":18265,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.008205163,"msgr_running_send_time":0.001382070,"msgr_running_recv_time":0.007187597,"msgr_running_fast_dispatch_time":0.000774550,"msgr_send_messages_queue_lat":{"avgcount":42,"sum":0.007940860,"avgtime":0.000189068},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":110256,"msgr_send_encrypted_bytes":6592},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":51,"sum":0.003683535,"avgtime":0.000072226}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000101017,"avgtime":0.000025254}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.002757320,"avgtime":0.001378660},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":3,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4198344,"rdpos":4195244,"jlat":{"avgcount":3,"sum":0.005332399,"avgtime":0.001777466},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":111124,"heap":248188},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002218422,"avgtime":0.002218422},"req_getattr_latency":{"avgcount":1,"sum":0.000538898,"avgtime":0.000538898},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":667,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":237239,"buffer_anon_items":68,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5848,"osdmap_items":95,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52320,"mds_co_items":189,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":51,"op_send_bytes":3733,"op_resend":0,"op_reply":51,"op_latency":{"avgcount":51,"sum":0.077998511,"avgtime":0.001529382},"op_inflight":0,"oplen_avg":{"avgcount":51,"sum":104},"op":51,"op_r":36,"op_w":15,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":84,"get_sum":134056,"get_or_fail_fail":0,"get_or_fail_success":84,"take":0,"take_sum":0,"put":84,"put_sum":134056,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":4850,"put":51,"put_sum":4850,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":51,"put":51,"put_sum":51,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ca9a045ba0":{"val":0,"max":3758096384,"get_started":0,"get":3,"get_sum":2659,"get_or_fail_fail":0,"get_or_fail_success":3,"take":0,"take_sum":0,"put":3,"put_sum":2659,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-20T22:10:46.670 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 16, "avgtime": 2.19e-07, "sum": 3.504e-06 }, "msgr_recv_bytes": 15904, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 30, "msgr_running_fast_dispatch_time": 0.000651718, "msgr_running_recv_time": 0.002829143, "msgr_running_send_time": 0.00085546, "msgr_running_total_time": 0.00569993, "msgr_send_bytes": 33304, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 27, "msgr_send_messages_queue_lat": { "avgcount": 26, "avgtime": 0.000147855, "sum": 0.003844234 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 4136, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 10, "msgr_running_fast_dispatch_time": 0.000224956, "msgr_running_recv_time": 0.003799807, "msgr_running_send_time": 0.000635399, "msgr_running_total_time": 0.004813924, "msgr_send_bytes": 94119, "msgr_send_encrypted_bytes": 90832, "msgr_send_messages": 14, "msgr_send_messages_queue_lat": { "avgcount": 12, "avgtime": 0.000329993, "sum": 0.003959921 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 124266, "msgr_recv_encrypted_bytes": 110256, "msgr_recv_messages": 49, "msgr_running_fast_dispatch_time": 0.00077455, "msgr_running_recv_time": 0.007187597, "msgr_running_send_time": 0.00138207, "msgr_running_total_time": 0.008205163, "msgr_send_bytes": 18265, "msgr_send_encrypted_bytes": 6592, "msgr_send_messages": 42, "msgr_send_messages_queue_lat": { "avgcount": 42, "avgtime": 0.000189068, "sum": 0.00794086 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 51, "avgtime": 7.2226e-05, "sum": 0.003683535 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.5254e-05, "sum": 0.000101017 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00137866, "sum": 0.00275732 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 3, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.001777466, "sum": 0.005332399 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4198344 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 248188, "ino": 14, "ino+": 14, "ino-": 0, "rss": 111124 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002218422, "sum": 0.002218422 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000538898, "sum": 0.000538898 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 667, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 237239, "buffer_anon_items": 68, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52320, "mds_co_items": 189, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5848, "osdmap_items": 95, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 51, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 51, "avgtime": 0.001529382, "sum": 0.077998511 }, "op_pg": 0, "op_r": 36, "op_reply": 51, "op_resend": 0, "op_rmw": 0, "op_send": 51, "op_send_bytes": 3733, "op_w": 15, "oplen_avg": { "avgcount": 51, "sum": 104 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 84, "get_or_fail_fail": 0, "get_or_fail_success": 84, "get_started": 0, "get_sum": 134056, "max": 104857600, "put": 84, "put_sum": 134056, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 51, "put_sum": 4850, "take": 51, "take_sum": 4850, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 51, "put_sum": 51, "take": 51, "take_sum": 51, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ca9a045ba0": { "get": 3, "get_or_fail_fail": 0, "get_or_fail_success": 3, "get_started": 0, "get_sum": 2659, "max": 3758096384, "put": 3, "put_sum": 2659, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-20T22:10:46.670 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-20T22:10:46.671 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:46.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.784+0000 7f2f71a3d640 1 Processor -- start 2024-05-20T22:10:46.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.784+0000 7f2f71a3d640 1 -- start start 2024-05-20T22:10:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f71a3d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b57b0 0x7f2f6c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f71a3d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f6c071e70 0x7f2f6c072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f71a3d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6c0727c0 con 0x7f2f6c0d0860 2024-05-20T22:10:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f71a3d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c0b7060 con 0x7f2f6c071e70 2024-05-20T22:10:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f71a3d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c0b84a0 con 0x7f2f6c0b57b0 2024-05-20T22:10:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f6a7fc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f6c071e70 0x7f2f6c072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f6a7fc640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f6c071e70 0x7f2f6c072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50890/0 (socket says 172.21.15.55:50890) 2024-05-20T22:10:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f6a7fc640 1 -- 172.21.15.55:0/3229143189 learned_addr learned my addr 172.21.15.55:0/3229143189 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f6affd640 1 --2- 172.21.15.55:0/3229143189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b57b0 0x7f2f6c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f6a7fc640 1 -- 172.21.15.55:0/3229143189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b57b0 msgr2=0x7f2f6c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f6a7fc640 1 --2- 172.21.15.55:0/3229143189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b57b0 0x7f2f6c15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f6a7fc640 1 -- 172.21.15.55:0/3229143189 >> v1:172.21.15.138:6789/0 conn(0x7f2f6c0d0860 legacy=0x7f2f6c0d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f6a7fc640 1 -- 172.21.15.55:0/3229143189 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c163d00 con 0x7f2f6c071e70 2024-05-20T22:10:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.786+0000 7f2f6affd640 1 --2- 172.21.15.55:0/3229143189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b57b0 0x7f2f6c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:10:46.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.787+0000 7f2f6a7fc640 1 --2- 172.21.15.55:0/3229143189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c071e70 0x7f2f6c072280 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009870 tx=0x7f2f5402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1efc143df37117d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:46.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.787+0000 7f2f69ffb640 1 -- 172.21.15.55:0/3229143189 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54037070 con 0x7f2f6c071e70 2024-05-20T22:10:46.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.787+0000 7f2f69ffb640 1 -- 172.21.15.55:0/3229143189 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f54032440 con 0x7f2f6c071e70 2024-05-20T22:10:46.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.787+0000 7f2f69ffb640 1 -- 172.21.15.55:0/3229143189 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54032810 con 0x7f2f6c071e70 2024-05-20T22:10:46.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.788+0000 7f2f71a3d640 1 -- 172.21.15.55:0/3229143189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c071e70 msgr2=0x7f2f6c072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:46.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.788+0000 7f2f71a3d640 1 --2- 172.21.15.55:0/3229143189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c071e70 0x7f2f6c072280 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009870 tx=0x7f2f5402a180 comp rx=0 tx=0).stop 2024-05-20T22:10:46.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.788+0000 7f2f71a3d640 1 -- 172.21.15.55:0/3229143189 shutdown_connections 2024-05-20T22:10:46.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.788+0000 7f2f71a3d640 1 --2- 172.21.15.55:0/3229143189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c071e70 0x7f2f6c072280 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.788+0000 7f2f71a3d640 1 --2- 172.21.15.55:0/3229143189 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b57b0 0x7f2f6c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.788+0000 7f2f71a3d640 1 -- 172.21.15.55:0/3229143189 >> 172.21.15.55:0/3229143189 conn(0x7f2f6c0d2960 msgr2=0x7f2f6c0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:46.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.788+0000 7f2f71a3d640 1 -- 172.21.15.55:0/3229143189 shutdown_connections 2024-05-20T22:10:46.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.788+0000 7f2f71a3d640 1 -- 172.21.15.55:0/3229143189 wait complete. 2024-05-20T22:10:46.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.790+0000 7f2f71a3d640 1 Processor -- start 2024-05-20T22:10:46.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.790+0000 7f2f71a3d640 1 -- start start 2024-05-20T22:10:46.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f71a3d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c071e70 0x7f2f6c149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f71a3d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b57b0 0x7f2f6c162440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f71a3d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c0d0860 0x7f2f6c15d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f71a3d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c163740 con 0x7f2f6c0d0860 2024-05-20T22:10:46.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f71a3d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c163460 con 0x7f2f6c0b57b0 2024-05-20T22:10:46.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f71a3d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c163a20 con 0x7f2f6c071e70 2024-05-20T22:10:46.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f6b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c0d0860 0x7f2f6c15d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f6b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c0d0860 0x7f2f6c15d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50896/0 (socket says 172.21.15.55:50896) 2024-05-20T22:10:46.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f6b7fe640 1 -- 172.21.15.55:0/2191171342 learned_addr learned my addr 172.21.15.55:0/2191171342 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:46.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f6affd640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c071e70 0x7f2f6c149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.791+0000 7f2f6a7fc640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b57b0 0x7f2f6c162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f6b7fe640 1 -- 172.21.15.55:0/2191171342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c071e70 msgr2=0x7f2f6c149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:46.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f6b7fe640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c071e70 0x7f2f6c149ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f6b7fe640 1 -- 172.21.15.55:0/2191171342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b57b0 msgr2=0x7f2f6c162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:46.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f6b7fe640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b57b0 0x7f2f6c162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:46.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f6b7fe640 1 -- 172.21.15.55:0/2191171342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c15c0c0 con 0x7f2f6c0d0860 2024-05-20T22:10:46.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f6b7fe640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c0d0860 0x7f2f6c15d160 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00be90 tx=0x7f2f5c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:46.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f63fff640 1 -- 172.21.15.55:0/2191171342 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c019070 con 0x7f2f6c0d0860 2024-05-20T22:10:46.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f63fff640 1 -- 172.21.15.55:0/2191171342 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c014440 con 0x7f2f6c0d0860 2024-05-20T22:10:46.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f63fff640 1 -- 172.21.15.55:0/2191171342 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c014870 con 0x7f2f6c0d0860 2024-05-20T22:10:46.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f71a3d640 1 -- 172.21.15.55:0/2191171342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c15c3a0 con 0x7f2f6c0d0860 2024-05-20T22:10:46.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.792+0000 7f2f71a3d640 1 -- 172.21.15.55:0/2191171342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c15c960 con 0x7f2f6c0d0860 2024-05-20T22:10:46.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.793+0000 7f2f71a3d640 1 -- 172.21.15.55:0/2191171342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c04f720 con 0x7f2f6c0d0860 2024-05-20T22:10:46.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.794+0000 7f2f63fff640 1 -- 172.21.15.55:0/2191171342 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f5c01c070 con 0x7f2f6c0d0860 2024-05-20T22:10:46.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.794+0000 7f2f63fff640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f4c06be30 0x7f2f4c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:46.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.794+0000 7f2f63fff640 1 -- 172.21.15.55:0/2191171342 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f2f5c08b030 con 0x7f2f6c0d0860 2024-05-20T22:10:46.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.794+0000 7f2f6affd640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f4c06be30 0x7f2f4c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:46.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.795+0000 7f2f6affd640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f4c06be30 0x7f2f4c06e340 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400aac0 tx=0x7f2f64008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:46.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:46.799+0000 7f2f63fff640 1 -- 172.21.15.55:0/2191171342 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2f5c0590e0 con 0x7f2f6c0d0860 2024-05-20T22:10:47.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.021+0000 7f2f71a3d640 1 -- 172.21.15.55:0/2191171342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f6c15b510 con 0x7f2f6c0d0860 2024-05-20T22:10:47.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.022+0000 7f2f63fff640 1 -- 172.21.15.55:0/2191171342 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7f2f5c05d3e0 con 0x7f2f6c0d0860 2024-05-20T22:10:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:47.023 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:10:47.023 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:10:47.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 -- 172.21.15.55:0/2191171342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f4c06be30 msgr2=0x7f2f4c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:47.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f4c06be30 0x7f2f4c06e340 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400aac0 tx=0x7f2f64008040 comp rx=0 tx=0).stop 2024-05-20T22:10:47.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 -- 172.21.15.55:0/2191171342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c0d0860 msgr2=0x7f2f6c15d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:47.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c0d0860 0x7f2f6c15d160 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00be90 tx=0x7f2f5c00bf80 comp rx=0 tx=0).stop 2024-05-20T22:10:47.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 -- 172.21.15.55:0/2191171342 shutdown_connections 2024-05-20T22:10:47.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2f4c06be30 0x7f2f4c06e340 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:47.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6c0d0860 0x7f2f6c15d160 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:47.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b57b0 0x7f2f6c162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:47.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 --2- 172.21.15.55:0/2191171342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c071e70 0x7f2f6c149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:47.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 -- 172.21.15.55:0/2191171342 >> 172.21.15.55:0/2191171342 conn(0x7f2f6c0d2960 msgr2=0x7f2f6c1572a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:47.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 -- 172.21.15.55:0/2191171342 shutdown_connections 2024-05-20T22:10:47.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47.024+0000 7f2f71a3d640 1 -- 172.21.15.55:0/2191171342 wait complete. 2024-05-20T22:10:47.036 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-20T22:10:47.153 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:47.167 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":30,"msgr_send_messages":27,"msgr_recv_bytes":15904,"msgr_send_bytes":33304,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.005699930,"msgr_running_send_time":0.000855460,"msgr_running_recv_time":0.002829143,"msgr_running_fast_dispatch_time":0.000651718,"msgr_send_messages_queue_lat":{"avgcount":26,"sum":0.003844234,"avgtime":0.000147855},"msgr_handle_ack_lat":{"avgcount":16,"sum":0.000003504,"avgtime":0.000000219},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":10,"msgr_send_messages":14,"msgr_recv_bytes":4136,"msgr_send_bytes":94119,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004813924,"msgr_running_send_time":0.000635399,"msgr_running_recv_time":0.003799807,"msgr_running_fast_dispatch_time":0.000224956,"msgr_send_messages_queue_lat":{"avgcount":12,"sum":0.003959921,"avgtime":0.000329993},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":90832},"AsyncMessenger::Worker-2":{"msgr_recv_messages":49,"msgr_send_messages":42,"msgr_recv_bytes":124266,"msgr_send_bytes":18265,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.008205163,"msgr_running_send_time":0.001382070,"msgr_running_recv_time":0.007187597,"msgr_running_fast_dispatch_time":0.000774550,"msgr_send_messages_queue_lat":{"avgcount":42,"sum":0.007940860,"avgtime":0.000189068},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":110256,"msgr_send_encrypted_bytes":6592},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":51,"sum":0.003683535,"avgtime":0.000072226}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000101017,"avgtime":0.000025254}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.002757320,"avgtime":0.001378660},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":3,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4198344,"rdpos":4195244,"jlat":{"avgcount":3,"sum":0.005332399,"avgtime":0.001777466},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":111124,"heap":248188},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002218422,"avgtime":0.002218422},"req_getattr_latency":{"avgcount":1,"sum":0.000538898,"avgtime":0.000538898},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":667,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":237239,"buffer_anon_items":68,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5848,"osdmap_items":95,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52320,"mds_co_items":189,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":51,"op_send_bytes":3733,"op_resend":0,"op_reply":51,"op_latency":{"avgcount":51,"sum":0.077998511,"avgtime":0.001529382},"op_inflight":0,"oplen_avg":{"avgcount":51,"sum":104},"op":51,"op_r":36,"op_w":15,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":84,"get_sum":134056,"get_or_fail_fail":0,"get_or_fail_success":84,"take":0,"take_sum":0,"put":84,"put_sum":134056,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":4850,"put":51,"put_sum":4850,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":51,"put":51,"put_sum":51,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ca9a045ba0":{"val":0,"max":3758096384,"get_started":0,"get":3,"get_sum":2659,"get_or_fail_fail":0,"get_or_fail_success":3,"take":0,"take_sum":0,"put":3,"put_sum":2659,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-20T22:10:47.168 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 16, "avgtime": 2.19e-07, "sum": 3.504e-06 }, "msgr_recv_bytes": 15904, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 30, "msgr_running_fast_dispatch_time": 0.000651718, "msgr_running_recv_time": 0.002829143, "msgr_running_send_time": 0.00085546, "msgr_running_total_time": 0.00569993, "msgr_send_bytes": 33304, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 27, "msgr_send_messages_queue_lat": { "avgcount": 26, "avgtime": 0.000147855, "sum": 0.003844234 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 4136, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 10, "msgr_running_fast_dispatch_time": 0.000224956, "msgr_running_recv_time": 0.003799807, "msgr_running_send_time": 0.000635399, "msgr_running_total_time": 0.004813924, "msgr_send_bytes": 94119, "msgr_send_encrypted_bytes": 90832, "msgr_send_messages": 14, "msgr_send_messages_queue_lat": { "avgcount": 12, "avgtime": 0.000329993, "sum": 0.003959921 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 124266, "msgr_recv_encrypted_bytes": 110256, "msgr_recv_messages": 49, "msgr_running_fast_dispatch_time": 0.00077455, "msgr_running_recv_time": 0.007187597, "msgr_running_send_time": 0.00138207, "msgr_running_total_time": 0.008205163, "msgr_send_bytes": 18265, "msgr_send_encrypted_bytes": 6592, "msgr_send_messages": 42, "msgr_send_messages_queue_lat": { "avgcount": 42, "avgtime": 0.000189068, "sum": 0.00794086 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 51, "avgtime": 7.2226e-05, "sum": 0.003683535 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.5254e-05, "sum": 0.000101017 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00137866, "sum": 0.00275732 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 3, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.001777466, "sum": 0.005332399 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4198344 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 248188, "ino": 14, "ino+": 14, "ino-": 0, "rss": 111124 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002218422, "sum": 0.002218422 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000538898, "sum": 0.000538898 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 667, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 237239, "buffer_anon_items": 68, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52320, "mds_co_items": 189, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5848, "osdmap_items": 95, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 51, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 51, "avgtime": 0.001529382, "sum": 0.077998511 }, "op_pg": 0, "op_r": 36, "op_reply": 51, "op_resend": 0, "op_rmw": 0, "op_send": 51, "op_send_bytes": 3733, "op_w": 15, "oplen_avg": { "avgcount": 51, "sum": 104 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 84, "get_or_fail_fail": 0, "get_or_fail_success": 84, "get_started": 0, "get_sum": 134056, "max": 104857600, "put": 84, "put_sum": 134056, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 51, "put_sum": 4850, "take": 51, "take_sum": 4850, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 51, "put_sum": 51, "take": 51, "take_sum": 51, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ca9a045ba0": { "get": 3, "get_or_fail_fail": 0, "get_or_fail_success": 3, "get_started": 0, "get_sum": 2659, "max": 3758096384, "put": 3, "put_sum": 2659, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-20T22:10:47.168 INFO:tasks.cephfs.test_openfiletable:omap_total_updates_1:2 2024-05-20T22:10:47.168 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/omap_counter_test_file", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi055') 2024-05-20T22:10:47.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-05-20T22:10:49.155 INFO:teuthology.orchestra.run.smithi055.stderr:open_background: keeping file open 2024-05-20T22:10:49.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T22:10:49.168 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:49.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.337+0000 7f55e73c3640 1 Processor -- start 2024-05-20T22:10:49.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.337+0000 7f55e73c3640 1 -- start start 2024-05-20T22:10:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.338+0000 7f55e73c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e00cd290 0x7f55e015cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.338+0000 7f55e73c3640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f55e006f0f0 0x7f55e006f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.338+0000 7f55e73c3640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e006fa40 con 0x7f55e00b5b80 2024-05-20T22:10:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.338+0000 7f55e73c3640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f55e00d1090 con 0x7f55e006f0f0 2024-05-20T22:10:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.338+0000 7f55e73c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f55e015d4a0 con 0x7f55e00cd290 2024-05-20T22:10:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.338+0000 7f55e515f640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f55e00b5b80 0x7f55e0160bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52734/0 (socket says 172.21.15.55:52734) 2024-05-20T22:10:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.339+0000 7f55e515f640 1 -- 172.21.15.55:0/3189951084 learned_addr learned my addr 172.21.15.55:0/3189951084 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.339+0000 7f55e5960640 1 --2- 172.21.15.55:0/3189951084 >> v2:172.21.15.55:3300/0 conn(0x7f55e006f0f0 0x7f55e006f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.339+0000 7f55d7fff640 1 -- 172.21.15.55:0/3189951084 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 631361788 0 0) 0x7f55dc001110 con 0x7f55e00b5b80 2024-05-20T22:10:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.339+0000 7f55e495e640 1 --2- 172.21.15.55:0/3189951084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e00cd290 0x7f55e015cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:49.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.339+0000 7f55d7fff640 1 -- 172.21.15.55:0/3189951084 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55cc003520 con 0x7f55e00b5b80 2024-05-20T22:10:49.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.339+0000 7f55e5960640 1 -- 172.21.15.55:0/3189951084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e00cd290 msgr2=0x7f55e015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:49.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.339+0000 7f55e5960640 1 --2- 172.21.15.55:0/3189951084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e00cd290 0x7f55e015cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:49.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.339+0000 7f55e5960640 1 -- 172.21.15.55:0/3189951084 >> v1:172.21.15.55:6789/0 conn(0x7f55e00b5b80 legacy=0x7f55e0160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:49.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.339+0000 7f55e5960640 1 -- 172.21.15.55:0/3189951084 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e0161380 con 0x7f55e006f0f0 2024-05-20T22:10:49.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.340+0000 7f55e5960640 1 --2- 172.21.15.55:0/3189951084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 0x7f55e006f500 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009d90 tx=0x7f55d802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c809335207ef2961 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:49.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.340+0000 7f55d7fff640 1 -- 172.21.15.55:0/3189951084 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8031600 con 0x7f55e006f0f0 2024-05-20T22:10:49.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.340+0000 7f55d7fff640 1 -- 172.21.15.55:0/3189951084 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d80317a0 con 0x7f55e006f0f0 2024-05-20T22:10:49.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.340+0000 7f55d7fff640 1 -- 172.21.15.55:0/3189951084 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8031b50 con 0x7f55e006f0f0 2024-05-20T22:10:49.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.341+0000 7f55e73c3640 1 -- 172.21.15.55:0/3189951084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 msgr2=0x7f55e006f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:49.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.341+0000 7f55e73c3640 1 --2- 172.21.15.55:0/3189951084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 0x7f55e006f500 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009d90 tx=0x7f55d802a5d0 comp rx=0 tx=0).stop 2024-05-20T22:10:49.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.341+0000 7f55e73c3640 1 -- 172.21.15.55:0/3189951084 shutdown_connections 2024-05-20T22:10:49.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.341+0000 7f55e73c3640 1 --2- 172.21.15.55:0/3189951084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e00cd290 0x7f55e015cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:49.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.341+0000 7f55e73c3640 1 --2- 172.21.15.55:0/3189951084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 0x7f55e006f500 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:49.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.341+0000 7f55e73c3640 1 -- 172.21.15.55:0/3189951084 >> 172.21.15.55:0/3189951084 conn(0x7f55e00d2910 msgr2=0x7f55e00d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:49.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.341+0000 7f55e73c3640 1 -- 172.21.15.55:0/3189951084 shutdown_connections 2024-05-20T22:10:49.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.341+0000 7f55e73c3640 1 -- 172.21.15.55:0/3189951084 wait complete. 2024-05-20T22:10:49.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.343+0000 7f55e73c3640 1 Processor -- start 2024-05-20T22:10:49.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.343+0000 7f55e73c3640 1 -- start start 2024-05-20T22:10:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e73c3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 0x7f55e00c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e73c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e00b5b80 0x7f55e014b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e73c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e00cd290 0x7f55e00b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e73c3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f55e00d12e0 con 0x7f55e006f0f0 2024-05-20T22:10:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e73c3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0161940 con 0x7f55e00cd290 2024-05-20T22:10:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e73c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f55e00d1800 con 0x7f55e00b5b80 2024-05-20T22:10:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e515f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 0x7f55e00c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e515f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 0x7f55e00c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51182/0 (socket says 172.21.15.55:51182) 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e515f640 1 -- 172.21.15.55:0/2744775460 learned_addr learned my addr 172.21.15.55:0/2744775460 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e495e640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e00b5b80 0x7f55e014b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e5960640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e00cd290 0x7f55e00b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e515f640 1 -- 172.21.15.55:0/2744775460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e00b5b80 msgr2=0x7f55e014b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e515f640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e00b5b80 0x7f55e014b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e515f640 1 -- 172.21.15.55:0/2744775460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e00cd290 msgr2=0x7f55e00b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e515f640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e00cd290 0x7f55e00b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.344+0000 7f55e515f640 1 -- 172.21.15.55:0/2744775460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e00c92d0 con 0x7f55e006f0f0 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.345+0000 7f55e515f640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 0x7f55e00c8d90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00c3a0 tx=0x7f55dc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.345+0000 7f55d5ffb640 1 -- 172.21.15.55:0/2744775460 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc017070 con 0x7f55e006f0f0 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.345+0000 7f55d5ffb640 1 -- 172.21.15.55:0/2744775460 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55dc005ce0 con 0x7f55e006f0f0 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.345+0000 7f55e73c3640 1 -- 172.21.15.55:0/2744775460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e014beb0 con 0x7f55e006f0f0 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.345+0000 7f55d5ffb640 1 -- 172.21.15.55:0/2744775460 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc007310 con 0x7f55e006f0f0 2024-05-20T22:10:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.345+0000 7f55e73c3640 1 -- 172.21.15.55:0/2744775460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e0149450 con 0x7f55e006f0f0 2024-05-20T22:10:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.346+0000 7f55e73c3640 1 -- 172.21.15.55:0/2744775460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e004f720 con 0x7f55e006f0f0 2024-05-20T22:10:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.346+0000 7f55d5ffb640 1 -- 172.21.15.55:0/2744775460 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f55dc01a070 con 0x7f55e006f0f0 2024-05-20T22:10:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.347+0000 7f55d5ffb640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c406be60 0x7f55c406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.347+0000 7f55d5ffb640 1 -- 172.21.15.55:0/2744775460 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f55dc08a460 con 0x7f55e006f0f0 2024-05-20T22:10:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.347+0000 7f55e495e640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c406be60 0x7f55c406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.348+0000 7f55e495e640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c406be60 0x7f55c406e370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f55d0006fd0 tx=0x7f55d0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:49.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.352+0000 7f55d5ffb640 1 -- 172.21.15.55:0/2744775460 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f55e014beb0 con 0x7f55e006f0f0 2024-05-20T22:10:49.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.575+0000 7f55e73c3640 1 -- 172.21.15.55:0/2744775460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55e00b2050 con 0x7f55e006f0f0 2024-05-20T22:10:49.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.576+0000 7f55d5ffb640 1 -- 172.21.15.55:0/2744775460 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7f55e006fa40 con 0x7f55e006f0f0 2024-05-20T22:10:49.577 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:49.577 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:10:49.577 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:10:49.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 -- 172.21.15.55:0/2744775460 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c406be60 msgr2=0x7f55c406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:49.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c406be60 0x7f55c406e370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f55d0006fd0 tx=0x7f55d0008040 comp rx=0 tx=0).stop 2024-05-20T22:10:49.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 -- 172.21.15.55:0/2744775460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 msgr2=0x7f55e00c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:49.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 0x7f55e00c8d90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00c3a0 tx=0x7f55dc00c860 comp rx=0 tx=0).stop 2024-05-20T22:10:49.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 -- 172.21.15.55:0/2744775460 shutdown_connections 2024-05-20T22:10:49.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f55c406be60 0x7f55c406e370 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:49.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e00cd290 0x7f55e00b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:49.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55e00b5b80 0x7f55e014b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:49.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 --2- 172.21.15.55:0/2744775460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e006f0f0 0x7f55e00c8d90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:49.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 -- 172.21.15.55:0/2744775460 >> 172.21.15.55:0/2744775460 conn(0x7f55e00d2910 msgr2=0x7f55e00d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:49.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 -- 172.21.15.55:0/2744775460 shutdown_connections 2024-05-20T22:10:49.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:49.578+0000 7f55e73c3640 1 -- 172.21.15.55:0/2744775460 wait complete. 2024-05-20T22:10:49.591 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-20T22:10:49.722 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":33,"msgr_send_messages":27,"msgr_recv_bytes":17158,"msgr_send_bytes":33304,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.006015876,"msgr_running_send_time":0.000949096,"msgr_running_recv_time":0.002922097,"msgr_running_fast_dispatch_time":0.000773143,"msgr_send_messages_queue_lat":{"avgcount":26,"sum":0.003844234,"avgtime":0.000147855},"msgr_handle_ack_lat":{"avgcount":19,"sum":0.000004046,"avgtime":0.000000212},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":10,"msgr_send_messages":15,"msgr_recv_bytes":4136,"msgr_send_bytes":95927,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004877683,"msgr_running_send_time":0.000689196,"msgr_running_recv_time":0.003799807,"msgr_running_fast_dispatch_time":0.000224956,"msgr_send_messages_queue_lat":{"avgcount":13,"sum":0.004011839,"avgtime":0.000308603},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":92640},"AsyncMessenger::Worker-2":{"msgr_recv_messages":50,"msgr_send_messages":43,"msgr_recv_bytes":124538,"msgr_send_bytes":18905,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.008317194,"msgr_running_send_time":0.001445092,"msgr_running_recv_time":0.007208664,"msgr_running_fast_dispatch_time":0.000795062,"msgr_send_messages_queue_lat":{"avgcount":43,"sum":0.007964677,"avgtime":0.000185225},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":110528,"msgr_send_encrypted_bytes":7232},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":51,"sum":0.003683535,"avgtime":0.000072226}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000101017,"avgtime":0.000025254}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.002757320,"avgtime":0.001378660},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":3,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4198344,"rdpos":4195244,"jlat":{"avgcount":3,"sum":0.005332399,"avgtime":0.001777466},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":111252,"heap":248188},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002218422,"avgtime":0.002218422},"req_getattr_latency":{"avgcount":1,"sum":0.000538898,"avgtime":0.000538898},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":672,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":237793,"buffer_anon_items":73,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5848,"osdmap_items":95,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55623,"mds_co_items":229,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":51,"op_send_bytes":3733,"op_resend":0,"op_reply":51,"op_latency":{"avgcount":51,"sum":0.077998511,"avgtime":0.001529382},"op_inflight":0,"oplen_avg":{"avgcount":51,"sum":104},"op":51,"op_r":36,"op_w":15,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":88,"get_sum":135170,"get_or_fail_fail":0,"get_or_fail_success":88,"take":0,"take_sum":0,"put":88,"put_sum":135170,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":4850,"put":51,"put_sum":4850,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":51,"put":51,"put_sum":51,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ca9a045ba0":{"val":0,"max":3758096384,"get_started":0,"get":3,"get_sum":2659,"get_or_fail_fail":0,"get_or_fail_success":3,"take":0,"take_sum":0,"put":3,"put_sum":2659,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-20T22:10:49.723 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 19, "avgtime": 2.12e-07, "sum": 4.046e-06 }, "msgr_recv_bytes": 17158, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 33, "msgr_running_fast_dispatch_time": 0.000773143, "msgr_running_recv_time": 0.002922097, "msgr_running_send_time": 0.000949096, "msgr_running_total_time": 0.006015876, "msgr_send_bytes": 33304, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 27, "msgr_send_messages_queue_lat": { "avgcount": 26, "avgtime": 0.000147855, "sum": 0.003844234 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 4136, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 10, "msgr_running_fast_dispatch_time": 0.000224956, "msgr_running_recv_time": 0.003799807, "msgr_running_send_time": 0.000689196, "msgr_running_total_time": 0.004877683, "msgr_send_bytes": 95927, "msgr_send_encrypted_bytes": 92640, "msgr_send_messages": 15, "msgr_send_messages_queue_lat": { "avgcount": 13, "avgtime": 0.000308603, "sum": 0.004011839 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 124538, "msgr_recv_encrypted_bytes": 110528, "msgr_recv_messages": 50, "msgr_running_fast_dispatch_time": 0.000795062, "msgr_running_recv_time": 0.007208664, "msgr_running_send_time": 0.001445092, "msgr_running_total_time": 0.008317194, "msgr_send_bytes": 18905, "msgr_send_encrypted_bytes": 7232, "msgr_send_messages": 43, "msgr_send_messages_queue_lat": { "avgcount": 43, "avgtime": 0.000185225, "sum": 0.007964677 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 51, "avgtime": 7.2226e-05, "sum": 0.003683535 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.5254e-05, "sum": 0.000101017 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00137866, "sum": 0.00275732 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 3, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.001777466, "sum": 0.005332399 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4198344 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 248188, "ino": 14, "ino+": 14, "ino-": 0, "rss": 111252 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002218422, "sum": 0.002218422 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000538898, "sum": 0.000538898 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 672, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 237793, "buffer_anon_items": 73, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55623, "mds_co_items": 229, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5848, "osdmap_items": 95, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 51, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 51, "avgtime": 0.001529382, "sum": 0.077998511 }, "op_pg": 0, "op_r": 36, "op_reply": 51, "op_resend": 0, "op_rmw": 0, "op_send": 51, "op_send_bytes": 3733, "op_w": 15, "oplen_avg": { "avgcount": 51, "sum": 104 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 88, "get_or_fail_fail": 0, "get_or_fail_success": 88, "get_started": 0, "get_sum": 135170, "max": 104857600, "put": 88, "put_sum": 135170, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 51, "put_sum": 4850, "take": 51, "take_sum": 4850, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 51, "put_sum": 51, "take": 51, "take_sum": 51, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ca9a045ba0": { "get": 3, "get_or_fail_fail": 0, "get_or_fail_success": 3, "get_started": 0, "get_sum": 2659, "max": 3758096384, "put": 3, "put_sum": 2659, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-20T22:10:49.723 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-05-20T22:10:54.724 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:10:54.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.853+0000 7f41fca22640 1 Processor -- start 2024-05-20T22:10:54.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.853+0000 7f41fca22640 1 -- start start 2024-05-20T22:10:54.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.854+0000 7f41fca22640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80cebe0 0x7f41f80d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:54.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.854+0000 7f41fca22640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f41f80cd000 0x7f41f80b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:54.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.854+0000 7f41fca22640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f80b3da0 con 0x7f41f80d1040 2024-05-20T22:10:54.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.854+0000 7f41fca22640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f41f815f6a0 con 0x7f41f80cd000 2024-05-20T22:10:54.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.854+0000 7f41fca22640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41f80b4040 con 0x7f41f80cebe0 2024-05-20T22:10:54.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.854+0000 7f41f6d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f41f80cd000 0x7f41f80b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:54.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.855+0000 7f41effff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f41f80d1040 0x7f41f80b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52738/0 (socket says 172.21.15.55:52738) 2024-05-20T22:10:54.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.855+0000 7f41effff640 1 -- 172.21.15.55:0/3635675320 learned_addr learned my addr 172.21.15.55:0/3635675320 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:10:54.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.855+0000 7f41f6d9d640 1 --2- 172.21.15.55:0/3635675320 >> v2:172.21.15.55:3300/0 conn(0x7f41f80cd000 0x7f41f80b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51196/0 (socket says 172.21.15.55:51196) 2024-05-20T22:10:54.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.855+0000 7f41f659c640 1 --2- 172.21.15.55:0/3635675320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80cebe0 0x7f41f80d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:54.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.855+0000 7f41f6d9d640 1 -- 172.21.15.55:0/3635675320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80cebe0 msgr2=0x7f41f80d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:10:54.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.855+0000 7f41f6d9d640 1 --2- 172.21.15.55:0/3635675320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80cebe0 0x7f41f80d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:54.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.855+0000 7f41f6d9d640 1 -- 172.21.15.55:0/3635675320 >> v1:172.21.15.55:6789/0 conn(0x7f41f80d1040 legacy=0x7f41f80b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:54.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.855+0000 7f41f6d9d640 1 -- 172.21.15.55:0/3635675320 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f80cdae0 con 0x7f41f80cd000 2024-05-20T22:10:54.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.856+0000 7f41f6d9d640 1 --2- 172.21.15.55:0/3635675320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 0x7f41f80b3190 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009d90 tx=0x7f41f002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ed121d04ce7bca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:54.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.856+0000 7f41f5d9b640 1 -- 172.21.15.55:0/3635675320 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0007740 con 0x7f41f80cd000 2024-05-20T22:10:54.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.856+0000 7f41f5d9b640 1 -- 172.21.15.55:0/3635675320 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f00078e0 con 0x7f41f80cd000 2024-05-20T22:10:54.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.856+0000 7f41f5d9b640 1 -- 172.21.15.55:0/3635675320 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0007cb0 con 0x7f41f80cd000 2024-05-20T22:10:54.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.856+0000 7f41fca22640 1 -- 172.21.15.55:0/3635675320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 msgr2=0x7f41f80b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:54.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.856+0000 7f41fca22640 1 --2- 172.21.15.55:0/3635675320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 0x7f41f80b3190 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009d90 tx=0x7f41f002a660 comp rx=0 tx=0).stop 2024-05-20T22:10:54.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.857+0000 7f41fca22640 1 -- 172.21.15.55:0/3635675320 shutdown_connections 2024-05-20T22:10:54.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.857+0000 7f41fca22640 1 --2- 172.21.15.55:0/3635675320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 0x7f41f80b3190 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:54.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.857+0000 7f41fca22640 1 --2- 172.21.15.55:0/3635675320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80cebe0 0x7f41f80d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:54.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.857+0000 7f41fca22640 1 -- 172.21.15.55:0/3635675320 >> 172.21.15.55:0/3635675320 conn(0x7f41f809a400 msgr2=0x7f41f809a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:54.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.857+0000 7f41fca22640 1 -- 172.21.15.55:0/3635675320 shutdown_connections 2024-05-20T22:10:54.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.857+0000 7f41fca22640 1 -- 172.21.15.55:0/3635675320 wait complete. 2024-05-20T22:10:54.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.859+0000 7f41fca22640 1 Processor -- start 2024-05-20T22:10:54.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.859+0000 7f41fca22640 1 -- start start 2024-05-20T22:10:54.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41fca22640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 0x7f41f8156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:54.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41fca22640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80cebe0 0x7f41f8149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:54.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41fca22640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f80d1040 0x7f41f80b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:54.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41fca22640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f41f80ceff0 con 0x7f41f80cd000 2024-05-20T22:10:54.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41fca22640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41f80ce0a0 con 0x7f41f80d1040 2024-05-20T22:10:54.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41fca22640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41f80d1450 con 0x7f41f80cebe0 2024-05-20T22:10:54.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41f659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 0x7f41f8156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:54.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41f659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 0x7f41f8156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51208/0 (socket says 172.21.15.55:51208) 2024-05-20T22:10:54.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41f659c640 1 -- 172.21.15.55:0/2643661249 learned_addr learned my addr 172.21.15.55:0/2643661249 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41effff640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80cebe0 0x7f41f8149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41f6d9d640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f80d1040 0x7f41f80b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41f659c640 1 -- 172.21.15.55:0/2643661249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80cebe0 msgr2=0x7f41f8149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41f659c640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80cebe0 0x7f41f8149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41f659c640 1 -- 172.21.15.55:0/2643661249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f80d1040 msgr2=0x7f41f80b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41f659c640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f80d1040 0x7f41f80b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41f659c640 1 -- 172.21.15.55:0/2643661249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f8155670 con 0x7f41f80cd000 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.860+0000 7f41f6d9d640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f80d1040 0x7f41f80b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.861+0000 7f41f659c640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 0x7f41f8156b80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f41e800d800 tx=0x7f41e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.861+0000 7f41eeffd640 1 -- 172.21.15.55:0/2643661249 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8018020 con 0x7f41f80cd000 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.861+0000 7f41eeffd640 1 -- 172.21.15.55:0/2643661249 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e80096c0 con 0x7f41f80cd000 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.861+0000 7f41eeffd640 1 -- 172.21.15.55:0/2643661249 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8009ab0 con 0x7f41f80cd000 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.861+0000 7f41fca22640 1 -- 172.21.15.55:0/2643661249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f8155ae0 con 0x7f41f80cd000 2024-05-20T22:10:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.861+0000 7f41fca22640 1 -- 172.21.15.55:0/2643661249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f81560a0 con 0x7f41f80cd000 2024-05-20T22:10:54.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.862+0000 7f41fca22640 1 -- 172.21.15.55:0/2643661249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f804f720 con 0x7f41f80cd000 2024-05-20T22:10:54.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.863+0000 7f41eeffd640 1 -- 172.21.15.55:0/2643661249 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41e801b020 con 0x7f41f80cd000 2024-05-20T22:10:54.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.863+0000 7f41eeffd640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41d806be60 0x7f41d806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:10:54.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.863+0000 7f41eeffd640 1 -- 172.21.15.55:0/2643661249 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f41e808b650 con 0x7f41f80cd000 2024-05-20T22:10:54.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.863+0000 7f41effff640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41d806be60 0x7f41d806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:10:54.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.864+0000 7f41effff640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41d806be60 0x7f41d806e370 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f41f81554b0 tx=0x7f41e0007e80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:10:54.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:54.868+0000 7f41eeffd640 1 -- 172.21.15.55:0/2643661249 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f41e8022020 con 0x7f41f80cd000 2024-05-20T22:10:55.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.089+0000 7f41fca22640 1 -- 172.21.15.55:0/2643661249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41f80b5c20 con 0x7f41f80cd000 2024-05-20T22:10:55.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.089+0000 7f41eeffd640 1 -- 172.21.15.55:0/2643661249 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7f41e8059d50 con 0x7f41f80cd000 2024-05-20T22:10:55.090 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:10:55.091 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:10:55.091 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:10:55.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 -- 172.21.15.55:0/2643661249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41d806be60 msgr2=0x7f41d806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:55.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41d806be60 0x7f41d806e370 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f41f81554b0 tx=0x7f41e0007e80 comp rx=0 tx=0).stop 2024-05-20T22:10:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 -- 172.21.15.55:0/2643661249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 msgr2=0x7f41f8156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:10:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 0x7f41f8156b80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f41e800d800 tx=0x7f41e800dcc0 comp rx=0 tx=0).stop 2024-05-20T22:10:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 -- 172.21.15.55:0/2643661249 shutdown_connections 2024-05-20T22:10:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f41d806be60 0x7f41d806e370 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f80d1040 0x7f41f80b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f80cebe0 0x7f41f8149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 --2- 172.21.15.55:0/2643661249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41f80cd000 0x7f41f8156b80 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:10:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 -- 172.21.15.55:0/2643661249 >> 172.21.15.55:0/2643661249 conn(0x7f41f809a400 msgr2=0x7f41f809a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:10:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 -- 172.21.15.55:0/2643661249 shutdown_connections 2024-05-20T22:10:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:10:55.091+0000 7f41fca22640 1 -- 172.21.15.55:0/2643661249 wait complete. 2024-05-20T22:10:55.104 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-20T22:10:55.235 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":40,"msgr_send_messages":28,"msgr_recv_bytes":19716,"msgr_send_bytes":33422,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.006784536,"msgr_running_send_time":0.001147581,"msgr_running_recv_time":0.003289715,"msgr_running_fast_dispatch_time":0.001040325,"msgr_send_messages_queue_lat":{"avgcount":27,"sum":0.003856164,"avgtime":0.000142820},"msgr_handle_ack_lat":{"avgcount":27,"sum":0.000005321,"avgtime":0.000000197},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":10,"msgr_send_messages":16,"msgr_recv_bytes":4136,"msgr_send_bytes":97735,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004945974,"msgr_running_send_time":0.000747805,"msgr_running_recv_time":0.003799807,"msgr_running_fast_dispatch_time":0.000224956,"msgr_send_messages_queue_lat":{"avgcount":14,"sum":0.004047507,"avgtime":0.000289107},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":94448},"AsyncMessenger::Worker-2":{"msgr_recv_messages":51,"msgr_send_messages":44,"msgr_recv_bytes":124810,"msgr_send_bytes":19545,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.008519479,"msgr_running_send_time":0.001531045,"msgr_running_recv_time":0.007250167,"msgr_running_fast_dispatch_time":0.000820175,"msgr_send_messages_queue_lat":{"avgcount":44,"sum":0.008013178,"avgtime":0.000182117},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":110800,"msgr_send_encrypted_bytes":7872},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":51,"sum":0.003683535,"avgtime":0.000072226}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000101017,"avgtime":0.000025254}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.002757320,"avgtime":0.001378660},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":4,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4200023,"rdpos":4195244,"jlat":{"avgcount":3,"sum":0.005332399,"avgtime":0.001777466},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":111636,"heap":248188},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":1,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002218422,"avgtime":0.002218422},"req_getattr_latency":{"avgcount":1,"sum":0.000538898,"avgtime":0.000538898},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":677,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":237793,"buffer_anon_items":73,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5848,"osdmap_items":95,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55623,"mds_co_items":229,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":51,"op_send_bytes":3733,"op_resend":0,"op_reply":51,"op_latency":{"avgcount":51,"sum":0.077998511,"avgtime":0.001529382},"op_inflight":0,"oplen_avg":{"avgcount":51,"sum":104},"op":51,"op_r":36,"op_w":15,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":96,"get_sum":137228,"get_or_fail_fail":0,"get_or_fail_success":96,"take":0,"take_sum":0,"put":96,"put_sum":137228,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":4850,"put":51,"put_sum":4850,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":51,"put":51,"put_sum":51,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ca9a045ba0":{"val":1679,"max":3758096384,"get_started":0,"get":4,"get_sum":4338,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":3,"put_sum":2659,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-20T22:10:55.236 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 27, "avgtime": 1.97e-07, "sum": 5.321e-06 }, "msgr_recv_bytes": 19716, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 40, "msgr_running_fast_dispatch_time": 0.001040325, "msgr_running_recv_time": 0.003289715, "msgr_running_send_time": 0.001147581, "msgr_running_total_time": 0.006784536, "msgr_send_bytes": 33422, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 28, "msgr_send_messages_queue_lat": { "avgcount": 27, "avgtime": 0.00014282, "sum": 0.003856164 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 4136, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 10, "msgr_running_fast_dispatch_time": 0.000224956, "msgr_running_recv_time": 0.003799807, "msgr_running_send_time": 0.000747805, "msgr_running_total_time": 0.004945974, "msgr_send_bytes": 97735, "msgr_send_encrypted_bytes": 94448, "msgr_send_messages": 16, "msgr_send_messages_queue_lat": { "avgcount": 14, "avgtime": 0.000289107, "sum": 0.004047507 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 124810, "msgr_recv_encrypted_bytes": 110800, "msgr_recv_messages": 51, "msgr_running_fast_dispatch_time": 0.000820175, "msgr_running_recv_time": 0.007250167, "msgr_running_send_time": 0.001531045, "msgr_running_total_time": 0.008519479, "msgr_send_bytes": 19545, "msgr_send_encrypted_bytes": 7872, "msgr_send_messages": 44, "msgr_send_messages_queue_lat": { "avgcount": 44, "avgtime": 0.000182117, "sum": 0.008013178 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 51, "avgtime": 7.2226e-05, "sum": 0.003683535 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.5254e-05, "sum": 0.000101017 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00137866, "sum": 0.00275732 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.001777466, "sum": 0.005332399 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4200023 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 248188, "ino": 14, "ino+": 14, "ino-": 0, "rss": 111636 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 1, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002218422, "sum": 0.002218422 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000538898, "sum": 0.000538898 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 677, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 237793, "buffer_anon_items": 73, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55623, "mds_co_items": 229, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5848, "osdmap_items": 95, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 51, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 51, "avgtime": 0.001529382, "sum": 0.077998511 }, "op_pg": 0, "op_r": 36, "op_reply": 51, "op_resend": 0, "op_rmw": 0, "op_send": 51, "op_send_bytes": 3733, "op_w": 15, "oplen_avg": { "avgcount": 51, "sum": 104 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 96, "get_or_fail_fail": 0, "get_or_fail_success": 96, "get_started": 0, "get_sum": 137228, "max": 104857600, "put": 96, "put_sum": 137228, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 51, "put_sum": 4850, "take": 51, "take_sum": 4850, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 51, "put_sum": 51, "take": 51, "take_sum": 51, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ca9a045ba0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4338, "max": 3758096384, "put": 3, "put_sum": 2659, "take": 0, "take_sum": 0, "val": 1679, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-20T22:10:55.237 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-05-20T22:11:00.238 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:11:00.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.367+0000 7efeaca92640 1 Processor -- start 2024-05-20T22:11:00.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.367+0000 7efeaca92640 1 -- start start 2024-05-20T22:11:00.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efeaca92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efea815f440 0x7efea80d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efeaca92640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efea80b3210 0x7efea80d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efeaca92640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efea80b09a0 0x7efea80b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efeaca92640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7efea806bfd0 con 0x7efea80b3210 2024-05-20T22:11:00.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efeaca92640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efea80d0bc0 con 0x7efea80b09a0 2024-05-20T22:11:00.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efeaca92640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efea806b6c0 con 0x7efea815f440 2024-05-20T22:11:00.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efea5d9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efea80b3210 0x7efea80d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efea5d9b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efea80b3210 0x7efea80d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48608/0 (socket says 172.21.15.55:48608) 2024-05-20T22:11:00.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efea5d9b640 1 -- 172.21.15.55:0/3333032425 learned_addr learned my addr 172.21.15.55:0/3333032425 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:00.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efea5d9b640 1 -- 172.21.15.55:0/3333032425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efea815f440 msgr2=0x7efea80d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:00.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efea5d9b640 1 --2- 172.21.15.55:0/3333032425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efea815f440 0x7efea80d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efea6d9d640 1 --2- 172.21.15.55:0/3333032425 >> v2:172.21.15.138:3300/0 conn(0x7efea80b09a0 0x7efea80b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efea5d9b640 1 -- 172.21.15.55:0/3333032425 >> v2:172.21.15.138:3300/0 conn(0x7efea80b09a0 msgr2=0x7efea80b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:00.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efea5d9b640 1 --2- 172.21.15.55:0/3333032425 >> v2:172.21.15.138:3300/0 conn(0x7efea80b09a0 0x7efea80b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.369+0000 7efea5d9b640 1 -- 172.21.15.55:0/3333032425 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea806b980 con 0x7efea80b3210 2024-05-20T22:11:00.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.370+0000 7efea5d9b640 1 --2- 172.21.15.55:0/3333032425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b3210 0x7efea80d0680 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efea80736b0 tx=0x7efe9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe797fa93bad19d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:00.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.370+0000 7efea559a640 1 -- 172.21.15.55:0/3333032425 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98037070 con 0x7efea80b3210 2024-05-20T22:11:00.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.370+0000 7efea559a640 1 -- 172.21.15.55:0/3333032425 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe98032440 con 0x7efea80b3210 2024-05-20T22:11:00.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.370+0000 7efea559a640 1 -- 172.21.15.55:0/3333032425 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98032810 con 0x7efea80b3210 2024-05-20T22:11:00.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.371+0000 7efeaca92640 1 -- 172.21.15.55:0/3333032425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b3210 msgr2=0x7efea80d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:00.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.371+0000 7efeaca92640 1 --2- 172.21.15.55:0/3333032425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b3210 0x7efea80d0680 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efea80736b0 tx=0x7efe9802a1e0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.371+0000 7efeaca92640 1 -- 172.21.15.55:0/3333032425 shutdown_connections 2024-05-20T22:11:00.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.371+0000 7efeaca92640 1 --2- 172.21.15.55:0/3333032425 >> v2:172.21.15.138:3300/0 conn(0x7efea80b09a0 0x7efea80b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.371+0000 7efeaca92640 1 --2- 172.21.15.55:0/3333032425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b3210 0x7efea80d0680 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.371+0000 7efeaca92640 1 --2- 172.21.15.55:0/3333032425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efea815f440 0x7efea80d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.371+0000 7efeaca92640 1 -- 172.21.15.55:0/3333032425 >> 172.21.15.55:0/3333032425 conn(0x7efea809a400 msgr2=0x7efea80b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:00.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.371+0000 7efeaca92640 1 -- 172.21.15.55:0/3333032425 shutdown_connections 2024-05-20T22:11:00.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.371+0000 7efeaca92640 1 -- 172.21.15.55:0/3333032425 wait complete. 2024-05-20T22:11:00.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.373+0000 7efeaca92640 1 Processor -- start 2024-05-20T22:11:00.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.373+0000 7efeaca92640 1 -- start start 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.374+0000 7efeaca92640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b09a0 0x7efea80c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.374+0000 7efeaca92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efea80b3210 0x7efea814e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.374+0000 7efeaca92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efea815f440 0x7efea815d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.374+0000 7efeaca92640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efea80b2460 con 0x7efea80b09a0 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.374+0000 7efeaca92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efea80b22e0 con 0x7efea815f440 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efeaca92640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efea80b6510 con 0x7efea80b3210 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b09a0 0x7efea80c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b09a0 0x7efea80c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48610/0 (socket says 172.21.15.55:48610) 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea659c640 1 -- 172.21.15.55:0/1862742946 learned_addr learned my addr 172.21.15.55:0/1862742946 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea5d9b640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efea80b3210 0x7efea814e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea6d9d640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efea815f440 0x7efea815d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea659c640 1 -- 172.21.15.55:0/1862742946 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efea80b3210 msgr2=0x7efea814e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea659c640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efea80b3210 0x7efea814e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea659c640 1 -- 172.21.15.55:0/1862742946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efea815f440 msgr2=0x7efea815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea659c640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efea815f440 0x7efea815d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea659c640 1 -- 172.21.15.55:0/1862742946 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea814ec20 con 0x7efea80b09a0 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea5d9b640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efea80b3210 0x7efea814e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea6d9d640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efea815f440 0x7efea815d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efea659c640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b09a0 0x7efea80c8550 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efe9000b440 tx=0x7efe9000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efe9effd640 1 -- 172.21.15.55:0/1862742946 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90015020 con 0x7efea80b09a0 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efe9effd640 1 -- 172.21.15.55:0/1862742946 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe900118b0 con 0x7efea80b09a0 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efeaca92640 1 -- 172.21.15.55:0/1862742946 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea80cd0c0 con 0x7efea80b09a0 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efe9effd640 1 -- 172.21.15.55:0/1862742946 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90011ca0 con 0x7efea80b09a0 2024-05-20T22:11:00.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.375+0000 7efeaca92640 1 -- 172.21.15.55:0/1862742946 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea80cd680 con 0x7efea80b09a0 2024-05-20T22:11:00.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.377+0000 7efeaca92640 1 -- 172.21.15.55:0/1862742946 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea804f720 con 0x7efea80b09a0 2024-05-20T22:11:00.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.377+0000 7efe9effd640 1 -- 172.21.15.55:0/1862742946 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe90018020 con 0x7efea80b09a0 2024-05-20T22:11:00.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.378+0000 7efe9effd640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe8c06be60 0x7efe8c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.378+0000 7efe9effd640 1 -- 172.21.15.55:0/1862742946 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7efe9008b260 con 0x7efea80b09a0 2024-05-20T22:11:00.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.378+0000 7efea5d9b640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe8c06be60 0x7efe8c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.378+0000 7efea5d9b640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe8c06be60 0x7efe8c06e370 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efe98034000 tx=0x7efe9802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:00.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.382+0000 7efe9effd640 1 -- 172.21.15.55:0/1862742946 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efe9001f020 con 0x7efea80b09a0 2024-05-20T22:11:00.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.604+0000 7efeaca92640 1 -- 172.21.15.55:0/1862742946 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efea806d090 con 0x7efea80b09a0 2024-05-20T22:11:00.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.604+0000 7efe9effd640 1 -- 172.21.15.55:0/1862742946 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7efe9000d040 con 0x7efea80b09a0 2024-05-20T22:11:00.605 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:00.605 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:11:00.606 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:11:00.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.606+0000 7efeaca92640 1 -- 172.21.15.55:0/1862742946 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe8c06be60 msgr2=0x7efe8c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.606+0000 7efeaca92640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe8c06be60 0x7efe8c06e370 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efe98034000 tx=0x7efe9802e000 comp rx=0 tx=0).stop 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.606+0000 7efeaca92640 1 -- 172.21.15.55:0/1862742946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b09a0 msgr2=0x7efea80c8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.606+0000 7efeaca92640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b09a0 0x7efea80c8550 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efe9000b440 tx=0x7efe9000b900 comp rx=0 tx=0).stop 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.606+0000 7efeaca92640 1 -- 172.21.15.55:0/1862742946 shutdown_connections 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.606+0000 7efeaca92640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efe8c06be60 0x7efe8c06e370 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.606+0000 7efeaca92640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efea815f440 0x7efea815d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.606+0000 7efeaca92640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efea80b3210 0x7efea814e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.606+0000 7efeaca92640 1 --2- 172.21.15.55:0/1862742946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea80b09a0 0x7efea80c8550 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.606+0000 7efeaca92640 1 -- 172.21.15.55:0/1862742946 >> 172.21.15.55:0/1862742946 conn(0x7efea809a400 msgr2=0x7efea806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.607+0000 7efeaca92640 1 -- 172.21.15.55:0/1862742946 shutdown_connections 2024-05-20T22:11:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.607+0000 7efeaca92640 1 -- 172.21.15.55:0/1862742946 wait complete. 2024-05-20T22:11:00.619 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-20T22:11:00.751 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":49,"msgr_send_messages":32,"msgr_recv_bytes":23058,"msgr_send_bytes":35026,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.007698761,"msgr_running_send_time":0.001452953,"msgr_running_recv_time":0.003545463,"msgr_running_fast_dispatch_time":0.001354796,"msgr_send_messages_queue_lat":{"avgcount":31,"sum":0.003950031,"avgtime":0.000127420},"msgr_handle_ack_lat":{"avgcount":34,"sum":0.000006903,"avgtime":0.000000203},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":10,"msgr_send_messages":17,"msgr_recv_bytes":4136,"msgr_send_bytes":99543,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.005092721,"msgr_running_send_time":0.000886766,"msgr_running_recv_time":0.003799807,"msgr_running_fast_dispatch_time":0.000224956,"msgr_send_messages_queue_lat":{"avgcount":15,"sum":0.004088350,"avgtime":0.000272556},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":96256},"AsyncMessenger::Worker-2":{"msgr_recv_messages":54,"msgr_send_messages":47,"msgr_recv_bytes":125574,"msgr_send_bytes":22590,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.008906927,"msgr_running_send_time":0.001722288,"msgr_running_recv_time":0.007322758,"msgr_running_fast_dispatch_time":0.000912989,"msgr_send_messages_queue_lat":{"avgcount":47,"sum":0.008205214,"avgtime":0.000174579},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":111072,"msgr_send_encrypted_bytes":8512},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":56,"sum":0.004055922,"avgtime":0.000072427}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000101017,"avgtime":0.000025254}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.002757320,"avgtime":0.001378660},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":4,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4200023,"rdpos":4195244,"jlat":{"avgcount":4,"sum":0.007794180,"avgtime":0.001948545},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":111892,"heap":248188},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":1,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002218422,"avgtime":0.002218422},"req_getattr_latency":{"avgcount":1,"sum":0.000538898,"avgtime":0.000538898},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":682,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":253825,"buffer_anon_items":77,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5848,"osdmap_items":95,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":57399,"mds_co_items":240,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":56,"op_send_bytes":5714,"op_resend":0,"op_reply":56,"op_latency":{"avgcount":56,"sum":0.091998242,"avgtime":0.001642825},"op_inflight":0,"oplen_avg":{"avgcount":56,"sum":111},"op":56,"op_r":36,"op_w":20,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":13,"omap_rd":48,"omap_del":4},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":2,"omap_total_removes":1},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":108,"get_sum":140202,"get_or_fail_fail":0,"get_or_fail_success":108,"take":0,"take_sum":0,"put":108,"put_sum":140202,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":56,"take_sum":6831,"put":56,"put_sum":6831,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":56,"take_sum":56,"put":56,"put_sum":56,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ca9a045ba0":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4338,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4338,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-20T22:11:00.752 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 34, "avgtime": 2.03e-07, "sum": 6.903e-06 }, "msgr_recv_bytes": 23058, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 49, "msgr_running_fast_dispatch_time": 0.001354796, "msgr_running_recv_time": 0.003545463, "msgr_running_send_time": 0.001452953, "msgr_running_total_time": 0.007698761, "msgr_send_bytes": 35026, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 32, "msgr_send_messages_queue_lat": { "avgcount": 31, "avgtime": 0.00012742, "sum": 0.003950031 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 4136, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 10, "msgr_running_fast_dispatch_time": 0.000224956, "msgr_running_recv_time": 0.003799807, "msgr_running_send_time": 0.000886766, "msgr_running_total_time": 0.005092721, "msgr_send_bytes": 99543, "msgr_send_encrypted_bytes": 96256, "msgr_send_messages": 17, "msgr_send_messages_queue_lat": { "avgcount": 15, "avgtime": 0.000272556, "sum": 0.00408835 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 125574, "msgr_recv_encrypted_bytes": 111072, "msgr_recv_messages": 54, "msgr_running_fast_dispatch_time": 0.000912989, "msgr_running_recv_time": 0.007322758, "msgr_running_send_time": 0.001722288, "msgr_running_total_time": 0.008906927, "msgr_send_bytes": 22590, "msgr_send_encrypted_bytes": 8512, "msgr_send_messages": 47, "msgr_send_messages_queue_lat": { "avgcount": 47, "avgtime": 0.000174579, "sum": 0.008205214 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 56, "avgtime": 7.2427e-05, "sum": 0.004055922 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.5254e-05, "sum": 0.000101017 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00137866, "sum": 0.00275732 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.001948545, "sum": 0.00779418 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4200023 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 248188, "ino": 14, "ino+": 14, "ino-": 0, "rss": 111892 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 1, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002218422, "sum": 0.002218422 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000538898, "sum": 0.000538898 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 682, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 253825, "buffer_anon_items": 77, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 57399, "mds_co_items": 240, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5848, "osdmap_items": 95, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 4, "omap_rd": 48, "omap_wr": 13, "op": 56, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 56, "avgtime": 0.001642825, "sum": 0.091998242 }, "op_pg": 0, "op_r": 36, "op_reply": 56, "op_resend": 0, "op_rmw": 0, "op_send": 56, "op_send_bytes": 5714, "op_w": 20, "oplen_avg": { "avgcount": 56, "sum": 111 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 108, "get_or_fail_fail": 0, "get_or_fail_success": 108, "get_started": 0, "get_sum": 140202, "max": 104857600, "put": 108, "put_sum": 140202, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 56, "put_sum": 6831, "take": 56, "take_sum": 6831, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 56, "put_sum": 56, "take": 56, "take_sum": 56, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ca9a045ba0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4338, "max": 3758096384, "put": 4, "put_sum": 4338, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-20T22:11:00.752 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-05-20T22:11:00.753 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:11:00.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.866+0000 7fd9aaf38640 1 Processor -- start 2024-05-20T22:11:00.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.866+0000 7fd9aaf38640 1 -- start start 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.867+0000 7fd9aaf38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d42a0 0x7fd9a40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.867+0000 7fd9aaf38640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9a415ccc0 0x7fd9a40d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.867+0000 7fd9aaf38640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a40d3fc0 con 0x7fd9a40d3bb0 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.867+0000 7fd9aaf38640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a4160550 con 0x7fd9a415ccc0 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.867+0000 7fd9aaf38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a415f9d0 con 0x7fd9a40d42a0 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.867+0000 7fd9a3fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9a415ccc0 0x7fd9a40d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.867+0000 7fd9a94d5640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd9a40d3bb0 0x7fd9a40b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35904/0 (socket says 172.21.15.55:35904) 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.867+0000 7fd9a94d5640 1 -- 172.21.15.55:0/3021139497 learned_addr learned my addr 172.21.15.55:0/3021139497 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.868+0000 7fd9a8cd4640 1 --2- 172.21.15.55:0/3021139497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d42a0 0x7fd9a40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.868+0000 7fd9a3fff640 1 -- 172.21.15.55:0/3021139497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d42a0 msgr2=0x7fd9a40cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.868+0000 7fd9a3fff640 1 --2- 172.21.15.55:0/3021139497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d42a0 0x7fd9a40cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.868+0000 7fd9a3fff640 1 -- 172.21.15.55:0/3021139497 >> v1:172.21.15.55:6789/0 conn(0x7fd9a40d3bb0 legacy=0x7fd9a40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:00.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.868+0000 7fd9a37fe640 1 -- 172.21.15.55:0/3021139497 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3981787741 0 0) 0x7fd98c001110 con 0x7fd9a40d3bb0 2024-05-20T22:11:00.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.868+0000 7fd9a3fff640 1 -- 172.21.15.55:0/3021139497 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a415fcb0 con 0x7fd9a415ccc0 2024-05-20T22:11:00.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.868+0000 7fd9a8cd4640 1 --2- 172.21.15.55:0/3021139497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d42a0 0x7fd9a40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.869+0000 7fd9a3fff640 1 --2- 172.21.15.55:0/3021139497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 0x7fd9a40d3670 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd994009d90 tx=0x7fd99402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dee52721b3fa04 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.869+0000 7fd9a37fe640 1 -- 172.21.15.55:0/3021139497 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994031600 con 0x7fd9a415ccc0 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.869+0000 7fd9a37fe640 1 -- 172.21.15.55:0/3021139497 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9940317a0 con 0x7fd9a415ccc0 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.869+0000 7fd9a37fe640 1 -- 172.21.15.55:0/3021139497 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994031b70 con 0x7fd9a415ccc0 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.869+0000 7fd9aaf38640 1 -- 172.21.15.55:0/3021139497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 msgr2=0x7fd9a40d3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.869+0000 7fd9aaf38640 1 --2- 172.21.15.55:0/3021139497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 0x7fd9a40d3670 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd994009d90 tx=0x7fd99402a5d0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.869+0000 7fd9aaf38640 1 -- 172.21.15.55:0/3021139497 shutdown_connections 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.869+0000 7fd9aaf38640 1 --2- 172.21.15.55:0/3021139497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 0x7fd9a40d3670 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.869+0000 7fd9aaf38640 1 --2- 172.21.15.55:0/3021139497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d42a0 0x7fd9a40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.869+0000 7fd9aaf38640 1 -- 172.21.15.55:0/3021139497 >> 172.21.15.55:0/3021139497 conn(0x7fd9a40d1c50 msgr2=0x7fd9a40d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:00.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.870+0000 7fd9aaf38640 1 -- 172.21.15.55:0/3021139497 shutdown_connections 2024-05-20T22:11:00.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.870+0000 7fd9aaf38640 1 -- 172.21.15.55:0/3021139497 wait complete. 2024-05-20T22:11:00.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.871+0000 7fd9aaf38640 1 Processor -- start 2024-05-20T22:11:00.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.871+0000 7fd9aaf38640 1 -- start start 2024-05-20T22:11:00.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9aaf38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d3bb0 0x7fd9a41541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9aaf38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 0x7fd9a41566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9aaf38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9a4160f60 0x7fd9a4161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9aaf38640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a415f630 con 0x7fd9a415ccc0 2024-05-20T22:11:00.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9aaf38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4160270 con 0x7fd9a4160f60 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9aaf38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a415f7b0 con 0x7fd9a40d3bb0 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 0x7fd9a41566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 0x7fd9a41566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48638/0 (socket says 172.21.15.55:48638) 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a3fff640 1 -- 172.21.15.55:0/2156941879 learned_addr learned my addr 172.21.15.55:0/2156941879 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a8cd4640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d3bb0 0x7fd9a41541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a94d5640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9a4160f60 0x7fd9a4161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a3fff640 1 -- 172.21.15.55:0/2156941879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d3bb0 msgr2=0x7fd9a41541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a3fff640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d3bb0 0x7fd9a41541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a3fff640 1 -- 172.21.15.55:0/2156941879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9a4160f60 msgr2=0x7fd9a4161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a3fff640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9a4160f60 0x7fd9a4161370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a3fff640 1 -- 172.21.15.55:0/2156941879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a41572e0 con 0x7fd9a415ccc0 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.873+0000 7fd9a3fff640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 0x7fd9a41566d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd994000f80 tx=0x7fd994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.874+0000 7fd9a17fa640 1 -- 172.21.15.55:0/2156941879 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99403f070 con 0x7fd9a415ccc0 2024-05-20T22:11:00.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.874+0000 7fd9a17fa640 1 -- 172.21.15.55:0/2156941879 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd994030be0 con 0x7fd9a415ccc0 2024-05-20T22:11:00.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.874+0000 7fd9aaf38640 1 -- 172.21.15.55:0/2156941879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a41575c0 con 0x7fd9a415ccc0 2024-05-20T22:11:00.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.874+0000 7fd9a17fa640 1 -- 172.21.15.55:0/2156941879 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994009450 con 0x7fd9a415ccc0 2024-05-20T22:11:00.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.874+0000 7fd9aaf38640 1 -- 172.21.15.55:0/2156941879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a4160760 con 0x7fd9a415ccc0 2024-05-20T22:11:00.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.875+0000 7fd9aaf38640 1 -- 172.21.15.55:0/2156941879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a41546f0 con 0x7fd9a415ccc0 2024-05-20T22:11:00.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.875+0000 7fd9a17fa640 1 -- 172.21.15.55:0/2156941879 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd994031650 con 0x7fd9a415ccc0 2024-05-20T22:11:00.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.876+0000 7fd9a17fa640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd98406be30 0x7fd98406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:00.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.876+0000 7fd9a8cd4640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd98406be30 0x7fd98406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:00.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.876+0000 7fd9a17fa640 1 -- 172.21.15.55:0/2156941879 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7fd9940ac220 con 0x7fd9a415ccc0 2024-05-20T22:11:00.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.877+0000 7fd9a8cd4640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd98406be30 0x7fd98406e340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00aac0 tx=0x7fd99c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:00.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:00.881+0000 7fd9a17fa640 1 -- 172.21.15.55:0/2156941879 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd99407a230 con 0x7fd9a415ccc0 2024-05-20T22:11:01.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.102+0000 7fd9aaf38640 1 -- 172.21.15.55:0/2156941879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9a404f720 con 0x7fd9a415ccc0 2024-05-20T22:11:01.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.103+0000 7fd9a17fa640 1 -- 172.21.15.55:0/2156941879 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7fd99407e530 con 0x7fd9a415ccc0 2024-05-20T22:11:01.143 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:01.143 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:11:01.145 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:11:01.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 -- 172.21.15.55:0/2156941879 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd98406be30 msgr2=0x7fd98406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:01.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd98406be30 0x7fd98406e340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00aac0 tx=0x7fd99c008040 comp rx=0 tx=0).stop 2024-05-20T22:11:01.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 -- 172.21.15.55:0/2156941879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 msgr2=0x7fd9a41566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:01.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 0x7fd9a41566d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd994000f80 tx=0x7fd994004a40 comp rx=0 tx=0).stop 2024-05-20T22:11:01.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 -- 172.21.15.55:0/2156941879 shutdown_connections 2024-05-20T22:11:01.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd98406be30 0x7fd98406e340 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9a4160f60 0x7fd9a4161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a415ccc0 0x7fd9a41566d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 --2- 172.21.15.55:0/2156941879 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9a40d3bb0 0x7fd9a41541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 -- 172.21.15.55:0/2156941879 >> 172.21.15.55:0/2156941879 conn(0x7fd9a40d1c50 msgr2=0x7fd9a415e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:01.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 -- 172.21.15.55:0/2156941879 shutdown_connections 2024-05-20T22:11:01.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.105+0000 7fd9aaf38640 1 -- 172.21.15.55:0/2156941879 wait complete. 2024-05-20T22:11:01.146 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-20T22:11:01.278 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":49,"msgr_send_messages":32,"msgr_recv_bytes":23058,"msgr_send_bytes":35026,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.007698761,"msgr_running_send_time":0.001452953,"msgr_running_recv_time":0.003545463,"msgr_running_fast_dispatch_time":0.001354796,"msgr_send_messages_queue_lat":{"avgcount":31,"sum":0.003950031,"avgtime":0.000127420},"msgr_handle_ack_lat":{"avgcount":34,"sum":0.000006903,"avgtime":0.000000203},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":10,"msgr_send_messages":17,"msgr_recv_bytes":4136,"msgr_send_bytes":99543,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.005092721,"msgr_running_send_time":0.000886766,"msgr_running_recv_time":0.003799807,"msgr_running_fast_dispatch_time":0.000224956,"msgr_send_messages_queue_lat":{"avgcount":15,"sum":0.004088350,"avgtime":0.000272556},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":96256},"AsyncMessenger::Worker-2":{"msgr_recv_messages":54,"msgr_send_messages":47,"msgr_recv_bytes":125574,"msgr_send_bytes":22590,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.008906927,"msgr_running_send_time":0.001722288,"msgr_running_recv_time":0.007322758,"msgr_running_fast_dispatch_time":0.000912989,"msgr_send_messages_queue_lat":{"avgcount":47,"sum":0.008205214,"avgtime":0.000174579},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":111072,"msgr_send_encrypted_bytes":8512},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":56,"sum":0.004055922,"avgtime":0.000072427}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000101017,"avgtime":0.000025254}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.002757320,"avgtime":0.001378660},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":4,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4200023,"rdpos":4195244,"jlat":{"avgcount":4,"sum":0.007794180,"avgtime":0.001948545},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":111892,"heap":248188},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":1,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002218422,"avgtime":0.002218422},"req_getattr_latency":{"avgcount":1,"sum":0.000538898,"avgtime":0.000538898},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":682,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":253825,"buffer_anon_items":77,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5848,"osdmap_items":95,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":57399,"mds_co_items":240,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":56,"op_send_bytes":5714,"op_resend":0,"op_reply":56,"op_latency":{"avgcount":56,"sum":0.091998242,"avgtime":0.001642825},"op_inflight":0,"oplen_avg":{"avgcount":56,"sum":111},"op":56,"op_r":36,"op_w":20,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":13,"omap_rd":48,"omap_del":4},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":2,"omap_total_removes":1},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":108,"get_sum":140202,"get_or_fail_fail":0,"get_or_fail_success":108,"take":0,"take_sum":0,"put":108,"put_sum":140202,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":56,"take_sum":6831,"put":56,"put_sum":6831,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":56,"take_sum":56,"put":56,"put_sum":56,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ca9a045ba0":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4338,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4338,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-20T22:11:01.279 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 34, "avgtime": 2.03e-07, "sum": 6.903e-06 }, "msgr_recv_bytes": 23058, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 49, "msgr_running_fast_dispatch_time": 0.001354796, "msgr_running_recv_time": 0.003545463, "msgr_running_send_time": 0.001452953, "msgr_running_total_time": 0.007698761, "msgr_send_bytes": 35026, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 32, "msgr_send_messages_queue_lat": { "avgcount": 31, "avgtime": 0.00012742, "sum": 0.003950031 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 4136, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 10, "msgr_running_fast_dispatch_time": 0.000224956, "msgr_running_recv_time": 0.003799807, "msgr_running_send_time": 0.000886766, "msgr_running_total_time": 0.005092721, "msgr_send_bytes": 99543, "msgr_send_encrypted_bytes": 96256, "msgr_send_messages": 17, "msgr_send_messages_queue_lat": { "avgcount": 15, "avgtime": 0.000272556, "sum": 0.00408835 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 125574, "msgr_recv_encrypted_bytes": 111072, "msgr_recv_messages": 54, "msgr_running_fast_dispatch_time": 0.000912989, "msgr_running_recv_time": 0.007322758, "msgr_running_send_time": 0.001722288, "msgr_running_total_time": 0.008906927, "msgr_send_bytes": 22590, "msgr_send_encrypted_bytes": 8512, "msgr_send_messages": 47, "msgr_send_messages_queue_lat": { "avgcount": 47, "avgtime": 0.000174579, "sum": 0.008205214 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 56, "avgtime": 7.2427e-05, "sum": 0.004055922 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.5254e-05, "sum": 0.000101017 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00137866, "sum": 0.00275732 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.001948545, "sum": 0.00779418 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4200023 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 248188, "ino": 14, "ino+": 14, "ino-": 0, "rss": 111892 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 1, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002218422, "sum": 0.002218422 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000538898, "sum": 0.000538898 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 682, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 253825, "buffer_anon_items": 77, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 57399, "mds_co_items": 240, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5848, "osdmap_items": 95, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 4, "omap_rd": 48, "omap_wr": 13, "op": 56, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 56, "avgtime": 0.001642825, "sum": 0.091998242 }, "op_pg": 0, "op_r": 36, "op_reply": 56, "op_resend": 0, "op_rmw": 0, "op_send": 56, "op_send_bytes": 5714, "op_w": 20, "oplen_avg": { "avgcount": 56, "sum": 111 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 108, "get_or_fail_fail": 0, "get_or_fail_success": 108, "get_started": 0, "get_sum": 140202, "max": 104857600, "put": 108, "put_sum": 140202, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 56, "put_sum": 6831, "take": 56, "take_sum": 6831, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 56, "put_sum": 56, "take": 56, "take_sum": 56, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ca9a045ba0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4338, "max": 3758096384, "put": 4, "put_sum": 4338, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-20T22:11:01.279 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-20T22:11:01.279 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:11:01.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.394+0000 7f9b92baa640 1 Processor -- start 2024-05-20T22:11:01.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.394+0000 7f9b92baa640 1 -- start start 2024-05-20T22:11:01.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.395+0000 7f9b92baa640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9b8c15f250 0x7f9b8c0b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:01.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.395+0000 7f9b92baa640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8c163d00 con 0x7f9b8c072580 2024-05-20T22:11:01.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.395+0000 7f9b92baa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8c163180 con 0x7f9b8c0b57b0 2024-05-20T22:11:01.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.395+0000 7f9b92baa640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c0b7060 con 0x7f9b8c15f250 2024-05-20T22:11:01.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.395+0000 7f9b8bfff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9b8c072580 0x7f9b8c0b7490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35914/0 (socket says 172.21.15.55:35914) 2024-05-20T22:11:01.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.395+0000 7f9b91147640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9b8c15f250 0x7f9b8c0b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:01.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.395+0000 7f9b8bfff640 1 -- 172.21.15.55:0/1120708215 learned_addr learned my addr 172.21.15.55:0/1120708215 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:01.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.396+0000 7f9b91147640 1 -- 172.21.15.55:0/1120708215 >> v1:172.21.15.138:6789/0 conn(0x7f9b8c0b57b0 legacy=0x7f9b8c071e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:01.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.396+0000 7f9b91147640 1 -- 172.21.15.55:0/1120708215 >> v1:172.21.15.55:6789/0 conn(0x7f9b8c072580 legacy=0x7f9b8c0b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:01.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.396+0000 7f9b91147640 1 -- 172.21.15.55:0/1120708215 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c163460 con 0x7f9b8c15f250 2024-05-20T22:11:01.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.396+0000 7f9b8b7fe640 1 -- 172.21.15.55:0/1120708215 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1387440915 0 0) 0x7f9b8c163d00 con 0x7f9b8c072580 2024-05-20T22:11:01.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.396+0000 7f9b91147640 1 --2- 172.21.15.55:0/1120708215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c15f250 0x7f9b8c0b7ba0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009d90 tx=0x7f9b8402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9face7dba2d4872 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:01.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.397+0000 7f9b8b7fe640 1 -- 172.21.15.55:0/1120708215 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84031600 con 0x7f9b8c15f250 2024-05-20T22:11:01.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.397+0000 7f9b8b7fe640 1 -- 172.21.15.55:0/1120708215 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b840317a0 con 0x7f9b8c15f250 2024-05-20T22:11:01.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.397+0000 7f9b8b7fe640 1 -- 172.21.15.55:0/1120708215 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84031b70 con 0x7f9b8c15f250 2024-05-20T22:11:01.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.397+0000 7f9b92baa640 1 -- 172.21.15.55:0/1120708215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c15f250 msgr2=0x7f9b8c0b7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:01.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.397+0000 7f9b92baa640 1 --2- 172.21.15.55:0/1120708215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c15f250 0x7f9b8c0b7ba0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009d90 tx=0x7f9b8402a610 comp rx=0 tx=0).stop 2024-05-20T22:11:01.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.397+0000 7f9b92baa640 1 -- 172.21.15.55:0/1120708215 shutdown_connections 2024-05-20T22:11:01.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.397+0000 7f9b92baa640 1 --2- 172.21.15.55:0/1120708215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c15f250 0x7f9b8c0b7ba0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.397+0000 7f9b92baa640 1 -- 172.21.15.55:0/1120708215 >> 172.21.15.55:0/1120708215 conn(0x7f9b8c0d2960 msgr2=0x7f9b8c0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:01.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.397+0000 7f9b92baa640 1 -- 172.21.15.55:0/1120708215 shutdown_connections 2024-05-20T22:11:01.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.397+0000 7f9b92baa640 1 -- 172.21.15.55:0/1120708215 wait complete. 2024-05-20T22:11:01.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.399+0000 7f9b92baa640 1 Processor -- start 2024-05-20T22:11:01.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.399+0000 7f9b92baa640 1 -- start start 2024-05-20T22:11:01.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b92baa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c0b57b0 0x7f9b8c162310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:01.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b92baa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b8c15f250 0x7f9b8c14f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:01.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b92baa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c160b90 0x7f9b8c150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:01.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b92baa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c15f660 con 0x7f9b8c160b90 2024-05-20T22:11:01.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b92baa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c163a20 con 0x7f9b8c0b57b0 2024-05-20T22:11:01.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b92baa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c162dc0 con 0x7f9b8c15f250 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b91147640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c160b90 0x7f9b8c150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b91147640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c160b90 0x7f9b8c150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48646/0 (socket says 172.21.15.55:48646) 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b91147640 1 -- 172.21.15.55:0/2418260395 learned_addr learned my addr 172.21.15.55:0/2418260395 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b90946640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c0b57b0 0x7f9b8c162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b8bfff640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b8c15f250 0x7f9b8c14f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b91147640 1 -- 172.21.15.55:0/2418260395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b8c15f250 msgr2=0x7f9b8c14f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b91147640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b8c15f250 0x7f9b8c14f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b91147640 1 -- 172.21.15.55:0/2418260395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c0b57b0 msgr2=0x7f9b8c162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b91147640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c0b57b0 0x7f9b8c162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b91147640 1 -- 172.21.15.55:0/2418260395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c161270 con 0x7f9b8c160b90 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b8bfff640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b8c15f250 0x7f9b8c14f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b91147640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c160b90 0x7f9b8c150780 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009eb0 tx=0x7f9b84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b897fa640 1 -- 172.21.15.55:0/2418260395 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84040070 con 0x7f9b8c160b90 2024-05-20T22:11:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b897fa640 1 -- 172.21.15.55:0/2418260395 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b84030be0 con 0x7f9b8c160b90 2024-05-20T22:11:01.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.401+0000 7f9b92baa640 1 -- 172.21.15.55:0/2418260395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c161550 con 0x7f9b8c160b90 2024-05-20T22:11:01.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.402+0000 7f9b897fa640 1 -- 172.21.15.55:0/2418260395 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84009450 con 0x7f9b8c160b90 2024-05-20T22:11:01.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.402+0000 7f9b92baa640 1 -- 172.21.15.55:0/2418260395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c161b10 con 0x7f9b8c160b90 2024-05-20T22:11:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.402+0000 7f9b92baa640 1 -- 172.21.15.55:0/2418260395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c04f720 con 0x7f9b8c160b90 2024-05-20T22:11:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.403+0000 7f9b897fa640 1 -- 172.21.15.55:0/2418260395 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b84039040 con 0x7f9b8c160b90 2024-05-20T22:11:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.404+0000 7f9b897fa640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b7006be30 0x7f9b7006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.404+0000 7f9b897fa640 1 -- 172.21.15.55:0/2418260395 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f9b840abdf0 con 0x7f9b8c160b90 2024-05-20T22:11:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.404+0000 7f9b90946640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b7006be30 0x7f9b7006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.404+0000 7f9b90946640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b7006be30 0x7f9b7006e340 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9b80006fd0 tx=0x7f9b80009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:01.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.408+0000 7f9b897fa640 1 -- 172.21.15.55:0/2418260395 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9b84079ea0 con 0x7f9b8c160b90 2024-05-20T22:11:01.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.632+0000 7f9b92baa640 1 -- 172.21.15.55:0/2418260395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b8c0b2440 con 0x7f9b8c160b90 2024-05-20T22:11:01.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.632+0000 7f9b897fa640 1 -- 172.21.15.55:0/2418260395 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7f9b8407e1a0 con 0x7f9b8c160b90 2024-05-20T22:11:01.633 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:01.634 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:11:01.634 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 -- 172.21.15.55:0/2418260395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b7006be30 msgr2=0x7f9b7006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b7006be30 0x7f9b7006e340 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9b80006fd0 tx=0x7f9b80009060 comp rx=0 tx=0).stop 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 -- 172.21.15.55:0/2418260395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c160b90 msgr2=0x7f9b8c150780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c160b90 0x7f9b8c150780 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009eb0 tx=0x7f9b84004a40 comp rx=0 tx=0).stop 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 -- 172.21.15.55:0/2418260395 shutdown_connections 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b7006be30 0x7f9b7006e340 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b8c160b90 0x7f9b8c150780 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b8c15f250 0x7f9b8c14f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 --2- 172.21.15.55:0/2418260395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c0b57b0 0x7f9b8c162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 -- 172.21.15.55:0/2418260395 >> 172.21.15.55:0/2418260395 conn(0x7f9b8c0d2960 msgr2=0x7f9b8c0cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:01.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 -- 172.21.15.55:0/2418260395 shutdown_connections 2024-05-20T22:11:01.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:01.635+0000 7f9b92baa640 1 -- 172.21.15.55:0/2418260395 wait complete. 2024-05-20T22:11:01.647 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-20T22:11:01.777 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":50,"msgr_send_messages":32,"msgr_recv_bytes":23476,"msgr_send_bytes":35026,"msgr_created_connections":9,"msgr_active_connections":5,"msgr_running_total_time":0.007798952,"msgr_running_send_time":0.001481544,"msgr_running_recv_time":0.003575525,"msgr_running_fast_dispatch_time":0.001394080,"msgr_send_messages_queue_lat":{"avgcount":31,"sum":0.003950031,"avgtime":0.000127420},"msgr_handle_ack_lat":{"avgcount":35,"sum":0.000007095,"avgtime":0.000000202},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":10,"msgr_send_messages":17,"msgr_recv_bytes":4136,"msgr_send_bytes":99543,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.005092721,"msgr_running_send_time":0.000886766,"msgr_running_recv_time":0.003799807,"msgr_running_fast_dispatch_time":0.000224956,"msgr_send_messages_queue_lat":{"avgcount":15,"sum":0.004088350,"avgtime":0.000272556},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":96256},"AsyncMessenger::Worker-2":{"msgr_recv_messages":55,"msgr_send_messages":48,"msgr_recv_bytes":125846,"msgr_send_bytes":23230,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.009087290,"msgr_running_send_time":0.001808922,"msgr_running_recv_time":0.007359116,"msgr_running_fast_dispatch_time":0.000938574,"msgr_send_messages_queue_lat":{"avgcount":48,"sum":0.008231058,"avgtime":0.000171480},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":111344,"msgr_send_encrypted_bytes":9152},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":56,"sum":0.004055922,"avgtime":0.000072427}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000101017,"avgtime":0.000025254}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.002757320,"avgtime":0.001378660},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":4,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4200023,"rdpos":4195244,"jlat":{"avgcount":4,"sum":0.007794180,"avgtime":0.001948545},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":112020,"heap":248188},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":1,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002218422,"avgtime":0.002218422},"req_getattr_latency":{"avgcount":1,"sum":0.000538898,"avgtime":0.000538898},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":682,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":253825,"buffer_anon_items":77,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5848,"osdmap_items":95,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":57399,"mds_co_items":240,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":56,"op_send_bytes":5714,"op_resend":0,"op_reply":56,"op_latency":{"avgcount":56,"sum":0.091998242,"avgtime":0.001642825},"op_inflight":0,"oplen_avg":{"avgcount":56,"sum":111},"op":56,"op_r":36,"op_w":20,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":13,"omap_rd":48,"omap_del":4},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":2,"omap_total_removes":1},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":110,"get_sum":140660,"get_or_fail_fail":0,"get_or_fail_success":110,"take":0,"take_sum":0,"put":110,"put_sum":140660,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":56,"take_sum":6831,"put":56,"put_sum":6831,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":56,"take_sum":56,"put":56,"put_sum":56,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55ca9a045ba0":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4338,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4338,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-20T22:11:01.779 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 5, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 35, "avgtime": 2.02e-07, "sum": 7.095e-06 }, "msgr_recv_bytes": 23476, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 50, "msgr_running_fast_dispatch_time": 0.00139408, "msgr_running_recv_time": 0.003575525, "msgr_running_send_time": 0.001481544, "msgr_running_total_time": 0.007798952, "msgr_send_bytes": 35026, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 32, "msgr_send_messages_queue_lat": { "avgcount": 31, "avgtime": 0.00012742, "sum": 0.003950031 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 4136, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 10, "msgr_running_fast_dispatch_time": 0.000224956, "msgr_running_recv_time": 0.003799807, "msgr_running_send_time": 0.000886766, "msgr_running_total_time": 0.005092721, "msgr_send_bytes": 99543, "msgr_send_encrypted_bytes": 96256, "msgr_send_messages": 17, "msgr_send_messages_queue_lat": { "avgcount": 15, "avgtime": 0.000272556, "sum": 0.00408835 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 125846, "msgr_recv_encrypted_bytes": 111344, "msgr_recv_messages": 55, "msgr_running_fast_dispatch_time": 0.000938574, "msgr_running_recv_time": 0.007359116, "msgr_running_send_time": 0.001808922, "msgr_running_total_time": 0.00908729, "msgr_send_bytes": 23230, "msgr_send_encrypted_bytes": 9152, "msgr_send_messages": 48, "msgr_send_messages_queue_lat": { "avgcount": 48, "avgtime": 0.00017148, "sum": 0.008231058 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 56, "avgtime": 7.2427e-05, "sum": 0.004055922 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.5254e-05, "sum": 0.000101017 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00137866, "sum": 0.00275732 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.001948545, "sum": 0.00779418 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4200023 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 248188, "ino": 14, "ino+": 14, "ino-": 0, "rss": 112020 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 1, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002218422, "sum": 0.002218422 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.000538898, "sum": 0.000538898 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 682, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 253825, "buffer_anon_items": 77, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 57399, "mds_co_items": 240, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5848, "osdmap_items": 95, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 4, "omap_rd": 48, "omap_wr": 13, "op": 56, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 56, "avgtime": 0.001642825, "sum": 0.091998242 }, "op_pg": 0, "op_r": 36, "op_reply": 56, "op_resend": 0, "op_rmw": 0, "op_send": 56, "op_send_bytes": 5714, "op_w": 20, "oplen_avg": { "avgcount": 56, "sum": 111 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 110, "get_or_fail_fail": 0, "get_or_fail_success": 110, "get_started": 0, "get_sum": 140660, "max": 104857600, "put": 110, "put_sum": 140660, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 56, "put_sum": 6831, "take": 56, "take_sum": 6831, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 56, "put_sum": 56, "take": 56, "take_sum": 56, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55ca9a045ba0": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4338, "max": 3758096384, "put": 4, "put_sum": 4338, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-20T22:11:01.779 INFO:tasks.cephfs.test_openfiletable:omap_total_removes:1 2024-05-20T22:11:01.779 INFO:tasks.cephfs.test_openfiletable:omap_total_kv_pairs:1 2024-05-20T22:11:01.779 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T22:11:01.780 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T22:11:01.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T22:11:01.870 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T22:11:01.870 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T22:11:01.870 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:11:01.870 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:11:01.887 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:11:02.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.040+0000 7fa6e84f7640 1 Processor -- start 2024-05-20T22:11:02.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.040+0000 7fa6e84f7640 1 -- start start 2024-05-20T22:11:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.041+0000 7fa6e84f7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6e00d3400 0x7fa6e01600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.041+0000 7fa6e84f7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e015cd50 0x7fa6e00cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.041+0000 7fa6e84f7640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6e00d44d0 con 0x7fa6e00cfce0 2024-05-20T22:11:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.041+0000 7fa6e84f7640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa6e01605f0 con 0x7fa6e00d3400 2024-05-20T22:11:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.041+0000 7fa6e84f7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e00d00f0 con 0x7fa6e015cd50 2024-05-20T22:11:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.042+0000 7fa6e6293640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6e00d3400 0x7fa6e01600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.042+0000 7fa6e6a94640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa6e00cfce0 0x7fa6e00b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35920/0 (socket says 172.21.15.55:35920) 2024-05-20T22:11:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.042+0000 7fa6e6293640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6e00d3400 0x7fa6e01600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48652/0 (socket says 172.21.15.55:48652) 2024-05-20T22:11:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.042+0000 7fa6e6a94640 1 -- 172.21.15.55:0/4263747268 learned_addr learned my addr 172.21.15.55:0/4263747268 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.042+0000 7fa6e5a92640 1 --2- 172.21.15.55:0/4263747268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e015cd50 0x7fa6e00cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.042+0000 7fa6e6293640 1 -- 172.21.15.55:0/4263747268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e015cd50 msgr2=0x7fa6e00cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.042+0000 7fa6e6293640 1 --2- 172.21.15.55:0/4263747268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e015cd50 0x7fa6e00cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.042+0000 7fa6e6293640 1 -- 172.21.15.55:0/4263747268 >> v1:172.21.15.55:6789/0 conn(0x7fa6e00cfce0 legacy=0x7fa6e00b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.042+0000 7fa6e5291640 1 -- 172.21.15.55:0/4263747268 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379294223 0 0) 0x7fa6e00d44d0 con 0x7fa6e00cfce0 2024-05-20T22:11:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.042+0000 7fa6e6293640 1 -- 172.21.15.55:0/4263747268 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e00d2fe0 con 0x7fa6e00d3400 2024-05-20T22:11:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.043+0000 7fa6e6293640 1 --2- 172.21.15.55:0/4263747268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00d3400 0x7fa6e01600b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0097a0 tx=0x7fa6dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d0ca8733fef984 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.043+0000 7fa6e5291640 1 -- 172.21.15.55:0/4263747268 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc037070 con 0x7fa6e00d3400 2024-05-20T22:11:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.043+0000 7fa6e5291640 1 -- 172.21.15.55:0/4263747268 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6dc032440 con 0x7fa6e00d3400 2024-05-20T22:11:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.043+0000 7fa6e5291640 1 -- 172.21.15.55:0/4263747268 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc032810 con 0x7fa6e00d3400 2024-05-20T22:11:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.044+0000 7fa6e84f7640 1 -- 172.21.15.55:0/4263747268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00d3400 msgr2=0x7fa6e01600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.044+0000 7fa6e84f7640 1 --2- 172.21.15.55:0/4263747268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00d3400 0x7fa6e01600b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0097a0 tx=0x7fa6dc02a180 comp rx=0 tx=0).stop 2024-05-20T22:11:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.044+0000 7fa6e84f7640 1 -- 172.21.15.55:0/4263747268 shutdown_connections 2024-05-20T22:11:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.044+0000 7fa6e84f7640 1 --2- 172.21.15.55:0/4263747268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e015cd50 0x7fa6e00cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.044+0000 7fa6e84f7640 1 --2- 172.21.15.55:0/4263747268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e00d3400 0x7fa6e01600b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.044+0000 7fa6e84f7640 1 -- 172.21.15.55:0/4263747268 >> 172.21.15.55:0/4263747268 conn(0x7fa6e00d1c80 msgr2=0x7fa6e00d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:02.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.044+0000 7fa6e84f7640 1 -- 172.21.15.55:0/4263747268 shutdown_connections 2024-05-20T22:11:02.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.044+0000 7fa6e84f7640 1 -- 172.21.15.55:0/4263747268 wait complete. 2024-05-20T22:11:02.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.045+0000 7fa6e84f7640 1 Processor -- start 2024-05-20T22:11:02.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.046+0000 7fa6e84f7640 1 -- start start 2024-05-20T22:11:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e84f7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e00cfce0 0x7fa6e014f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e84f7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d3400 0x7fa6e00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e84f7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e015cd50 0x7fa6e0162940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e84f7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e00d3f10 con 0x7fa6e015cd50 2024-05-20T22:11:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e84f7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e00d3c30 con 0x7fa6e00d3400 2024-05-20T22:11:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e84f7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e00d41f0 con 0x7fa6e00cfce0 2024-05-20T22:11:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e6a94640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e015cd50 0x7fa6e0162940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e6a94640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e015cd50 0x7fa6e0162940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48660/0 (socket says 172.21.15.55:48660) 2024-05-20T22:11:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e6a94640 1 -- 172.21.15.55:0/2470658695 learned_addr learned my addr 172.21.15.55:0/2470658695 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e6293640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e00cfce0 0x7fa6e014f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e5a92640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d3400 0x7fa6e00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e6a94640 1 -- 172.21.15.55:0/2470658695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e00cfce0 msgr2=0x7fa6e014f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e6a94640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e00cfce0 0x7fa6e014f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e6a94640 1 -- 172.21.15.55:0/2470658695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d3400 msgr2=0x7fa6e00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e6a94640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d3400 0x7fa6e00ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e6a94640 1 -- 172.21.15.55:0/2470658695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e01f95f0 con 0x7fa6e015cd50 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.047+0000 7fa6e6a94640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e015cd50 0x7fa6e0162940 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c730 tx=0x7fa6d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.048+0000 7fa6d6ffd640 1 -- 172.21.15.55:0/2470658695 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8018070 con 0x7fa6e015cd50 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.048+0000 7fa6d6ffd640 1 -- 172.21.15.55:0/2470658695 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d800ce40 con 0x7fa6e015cd50 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.048+0000 7fa6d6ffd640 1 -- 172.21.15.55:0/2470658695 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800a860 con 0x7fa6e015cd50 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.048+0000 7fa6e84f7640 1 -- 172.21.15.55:0/2470658695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e01f98d0 con 0x7fa6e015cd50 2024-05-20T22:11:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.048+0000 7fa6e84f7640 1 -- 172.21.15.55:0/2470658695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e01f9e90 con 0x7fa6e015cd50 2024-05-20T22:11:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.049+0000 7fa6e84f7640 1 -- 172.21.15.55:0/2470658695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e01f9360 con 0x7fa6e015cd50 2024-05-20T22:11:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.049+0000 7fa6d6ffd640 1 -- 172.21.15.55:0/2470658695 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6d801b070 con 0x7fa6e015cd50 2024-05-20T22:11:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.050+0000 7fa6d6ffd640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6c406be60 0x7fa6c406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.050+0000 7fa6d6ffd640 1 -- 172.21.15.55:0/2470658695 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7fa6d808a5a0 con 0x7fa6e015cd50 2024-05-20T22:11:02.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.050+0000 7fa6e6293640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6c406be60 0x7fa6c406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.050+0000 7fa6e6293640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6c406be60 0x7fa6c406e370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc034000 tx=0x7fa6dc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:02.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.055+0000 7fa6d6ffd640 1 -- 172.21.15.55:0/2470658695 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa6d8058dc0 con 0x7fa6e015cd50 2024-05-20T22:11:02.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.241+0000 7fa6e84f7640 1 -- 172.21.15.55:0/2470658695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6e004f720 con 0x7fa6e015cd50 2024-05-20T22:11:02.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.243+0000 7fa6d6ffd640 1 -- 172.21.15.55:0/2470658695 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+17167 (secure 0 0 0) 0x7fa6d80586c0 con 0x7fa6e015cd50 2024-05-20T22:11:02.244 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:02.244 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":61,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:10:38.297580+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:10:22.369451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T22:10:22.970959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.25","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:6837/2445634740":"2024-05-21T22:10:38.297533+0000","172.21.15.55:6835/2445634740":"2024-05-21T22:10:38.297533+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:11:02.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 -- 172.21.15.55:0/2470658695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6c406be60 msgr2=0x7fa6c406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6c406be60 0x7fa6c406e370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc034000 tx=0x7fa6dc02e000 comp rx=0 tx=0).stop 2024-05-20T22:11:02.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 -- 172.21.15.55:0/2470658695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e015cd50 msgr2=0x7fa6e0162940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e015cd50 0x7fa6e0162940 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c730 tx=0x7fa6d800cbf0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 -- 172.21.15.55:0/2470658695 shutdown_connections 2024-05-20T22:11:02.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa6c406be60 0x7fa6c406e370 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e015cd50 0x7fa6e0162940 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6e00d3400 0x7fa6e00ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 --2- 172.21.15.55:0/2470658695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6e00cfce0 0x7fa6e014f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 -- 172.21.15.55:0/2470658695 >> 172.21.15.55:0/2470658695 conn(0x7fa6e00d1c80 msgr2=0x7fa6e006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:02.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 -- 172.21.15.55:0/2470658695 shutdown_connections 2024-05-20T22:11:02.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.245+0000 7fa6e84f7640 1 -- 172.21.15.55:0/2470658695 wait complete. 2024-05-20T22:11:02.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:10:22.369451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:10:22.970959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T22:11:02.258 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:11:02.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.375+0000 7fd940a91640 1 Processor -- start 2024-05-20T22:11:02.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.375+0000 7fd940a91640 1 -- start start 2024-05-20T22:11:02.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.376+0000 7fd940a91640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd93c0d3460 0x7fd93c15f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.376+0000 7fd940a91640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd93c15cd00 0x7fd93c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.376+0000 7fd940a91640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd93c0d0110 con 0x7fd93c0cfd00 2024-05-20T22:11:02.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.376+0000 7fd940a91640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fd93c15d110 con 0x7fd93c0d3460 2024-05-20T22:11:02.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.376+0000 7fd940a91640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd93c0d3a20 con 0x7fd93c15cd00 2024-05-20T22:11:02.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.376+0000 7fd93a59c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd93c0d3460 0x7fd93c15f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.376+0000 7fd93a59c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd93c0d3460 0x7fd93c15f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48664/0 (socket says 172.21.15.55:48664) 2024-05-20T22:11:02.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.376+0000 7fd93a59c640 1 -- 172.21.15.55:0/2964899296 learned_addr learned my addr 172.21.15.55:0/2964899296 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:02.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.377+0000 7fd933fff640 1 --2- 172.21.15.55:0/2964899296 >> v2:172.21.15.138:3300/0 conn(0x7fd93c15cd00 0x7fd93c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.377+0000 7fd93a59c640 1 -- 172.21.15.55:0/2964899296 >> v2:172.21.15.138:3300/0 conn(0x7fd93c15cd00 msgr2=0x7fd93c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:02.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.377+0000 7fd93a59c640 1 --2- 172.21.15.55:0/2964899296 >> v2:172.21.15.138:3300/0 conn(0x7fd93c15cd00 0x7fd93c0cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.377+0000 7fd93a59c640 1 -- 172.21.15.55:0/2964899296 >> v1:172.21.15.138:6789/0 conn(0x7fd93c0cfd00 legacy=0x7fd93c0b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:02.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.377+0000 7fd93a59c640 1 -- 172.21.15.55:0/2964899296 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c0d3d00 con 0x7fd93c0d3460 2024-05-20T22:11:02.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.377+0000 7fd93a59c640 1 --2- 172.21.15.55:0/2964899296 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c0d3460 0x7fd93c15f6e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0cbb70 tx=0x7fd92c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f44c16138d8404 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:02.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd939d9b640 1 -- 172.21.15.55:0/2964899296 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c037070 con 0x7fd93c0d3460 2024-05-20T22:11:02.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd939d9b640 1 -- 172.21.15.55:0/2964899296 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92c032440 con 0x7fd93c0d3460 2024-05-20T22:11:02.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd939d9b640 1 -- 172.21.15.55:0/2964899296 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c032810 con 0x7fd93c0d3460 2024-05-20T22:11:02.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd940a91640 1 -- 172.21.15.55:0/2964899296 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c0d3460 msgr2=0x7fd93c15f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd940a91640 1 --2- 172.21.15.55:0/2964899296 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c0d3460 0x7fd93c15f6e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0cbb70 tx=0x7fd92c02a230 comp rx=0 tx=0).stop 2024-05-20T22:11:02.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd940a91640 1 -- 172.21.15.55:0/2964899296 shutdown_connections 2024-05-20T22:11:02.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd940a91640 1 --2- 172.21.15.55:0/2964899296 >> v2:172.21.15.138:3300/0 conn(0x7fd93c15cd00 0x7fd93c0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd940a91640 1 --2- 172.21.15.55:0/2964899296 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c0d3460 0x7fd93c15f6e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd940a91640 1 -- 172.21.15.55:0/2964899296 >> 172.21.15.55:0/2964899296 conn(0x7fd93c0d1c70 msgr2=0x7fd93c0d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:02.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd940a91640 1 -- 172.21.15.55:0/2964899296 shutdown_connections 2024-05-20T22:11:02.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.378+0000 7fd940a91640 1 -- 172.21.15.55:0/2964899296 wait complete. 2024-05-20T22:11:02.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.380+0000 7fd940a91640 1 Processor -- start 2024-05-20T22:11:02.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.380+0000 7fd940a91640 1 -- start start 2024-05-20T22:11:02.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd940a91640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd93c0cfd00 0x7fd93c153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd940a91640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd93c0d3460 0x7fd93c156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd940a91640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c15cd00 0x7fd93c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd940a91640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c0d04a0 con 0x7fd93c15cd00 2024-05-20T22:11:02.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd940a91640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c0d42c0 con 0x7fd93c0d3460 2024-05-20T22:11:02.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd940a91640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd93c0d0620 con 0x7fd93c0cfd00 2024-05-20T22:11:02.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd93ad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c15cd00 0x7fd93c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd93ad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c15cd00 0x7fd93c157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48678/0 (socket says 172.21.15.55:48678) 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd93ad9d640 1 -- 172.21.15.55:0/2831879274 learned_addr learned my addr 172.21.15.55:0/2831879274 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd93a59c640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd93c0cfd00 0x7fd93c153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd933fff640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd93c0d3460 0x7fd93c156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd93ad9d640 1 -- 172.21.15.55:0/2831879274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd93c0cfd00 msgr2=0x7fd93c153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd93ad9d640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd93c0cfd00 0x7fd93c153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd93ad9d640 1 -- 172.21.15.55:0/2831879274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd93c0d3460 msgr2=0x7fd93c156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd93ad9d640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd93c0d3460 0x7fd93c156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd93ad9d640 1 -- 172.21.15.55:0/2831879274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c154b20 con 0x7fd93c15cd00 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd93ad9d640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c15cd00 0x7fd93c157520 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd93400b9e0 tx=0x7fd93400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.382+0000 7fd932ffd640 1 -- 172.21.15.55:0/2831879274 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934003b70 con 0x7fd93c15cd00 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.383+0000 7fd932ffd640 1 -- 172.21.15.55:0/2831879274 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd934003d10 con 0x7fd93c15cd00 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.383+0000 7fd940a91640 1 -- 172.21.15.55:0/2831879274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93c154d90 con 0x7fd93c15cd00 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.383+0000 7fd932ffd640 1 -- 172.21.15.55:0/2831879274 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93400cb60 con 0x7fd93c15cd00 2024-05-20T22:11:02.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.383+0000 7fd940a91640 1 -- 172.21.15.55:0/2831879274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd93c155350 con 0x7fd93c15cd00 2024-05-20T22:11:02.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.384+0000 7fd940a91640 1 -- 172.21.15.55:0/2831879274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93c04f720 con 0x7fd93c15cd00 2024-05-20T22:11:02.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.384+0000 7fd932ffd640 1 -- 172.21.15.55:0/2831879274 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd93400cd00 con 0x7fd93c15cd00 2024-05-20T22:11:02.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.385+0000 7fd932ffd640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd91c06be30 0x7fd91c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.385+0000 7fd93a59c640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd91c06be30 0x7fd91c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.385+0000 7fd932ffd640 1 -- 172.21.15.55:0/2831879274 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7fd93408af20 con 0x7fd93c15cd00 2024-05-20T22:11:02.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.386+0000 7fd93a59c640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd91c06be30 0x7fd91c06e340 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd92c034000 tx=0x7fd92c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:02.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.390+0000 7fd932ffd640 1 -- 172.21.15.55:0/2831879274 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd934059620 con 0x7fd93c15cd00 2024-05-20T22:11:02.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.575+0000 7fd940a91640 1 -- 172.21.15.55:0/2831879274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd93c157ac0 con 0x7fd93c15cd00 2024-05-20T22:11:02.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.575+0000 7fd932ffd640 1 -- 172.21.15.55:0/2831879274 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7fd934058f20 con 0x7fd93c15cd00 2024-05-20T22:11:02.576 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 -- 172.21.15.55:0/2831879274 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd91c06be30 msgr2=0x7fd91c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd91c06be30 0x7fd91c06e340 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd92c034000 tx=0x7fd92c02e000 comp rx=0 tx=0).stop 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 -- 172.21.15.55:0/2831879274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c15cd00 msgr2=0x7fd93c157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c15cd00 0x7fd93c157520 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd93400b9e0 tx=0x7fd93400bea0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 -- 172.21.15.55:0/2831879274 shutdown_connections 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd91c06be30 0x7fd91c06e340 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd93c15cd00 0x7fd93c157520 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd93c0d3460 0x7fd93c156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 --2- 172.21.15.55:0/2831879274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd93c0cfd00 0x7fd93c153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 -- 172.21.15.55:0/2831879274 >> 172.21.15.55:0/2831879274 conn(0x7fd93c0d1c70 msgr2=0x7fd93c162180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 -- 172.21.15.55:0/2831879274 shutdown_connections 2024-05-20T22:11:02.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.577+0000 7fd940a91640 1 -- 172.21.15.55:0/2831879274 wait complete. 2024-05-20T22:11:02.590 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:11:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.706+0000 7f29fddb7640 1 Processor -- start 2024-05-20T22:11:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.706+0000 7f29fddb7640 1 -- start start 2024-05-20T22:11:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.707+0000 7f29fddb7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29f80cc930 0x7f29f80ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.707+0000 7f29fddb7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29f80d0ab0 0x7f29f80d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.707+0000 7f29fddb7640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f815f7b0 con 0x7f29f809ad90 2024-05-20T22:11:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.707+0000 7f29fddb7640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f29f80ce0e0 con 0x7f29f80cc930 2024-05-20T22:11:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.707+0000 7f29fddb7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f29f80d1cf0 con 0x7f29f80d0ab0 2024-05-20T22:11:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.707+0000 7f29f6ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29f80cc930 0x7f29f80ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.707+0000 7f29f6ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29f80cc930 0x7f29f80ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48682/0 (socket says 172.21.15.55:48682) 2024-05-20T22:11:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.707+0000 7f29f6ffd640 1 -- 172.21.15.55:0/413017442 learned_addr learned my addr 172.21.15.55:0/413017442 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.707+0000 7f29f7fff640 1 --2- 172.21.15.55:0/413017442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29f80d0ab0 0x7f29f80d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.708+0000 7f29f6ffd640 1 -- 172.21.15.55:0/413017442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29f80d0ab0 msgr2=0x7f29f80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.708+0000 7f29f6ffd640 1 --2- 172.21.15.55:0/413017442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29f80d0ab0 0x7f29f80d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.708+0000 7f29f6ffd640 1 -- 172.21.15.55:0/413017442 >> v1:172.21.15.138:6789/0 conn(0x7f29f809ad90 legacy=0x7f29f80b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.708+0000 7f29f6ffd640 1 -- 172.21.15.55:0/413017442 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f80cdb20 con 0x7f29f80cc930 2024-05-20T22:11:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.708+0000 7f29f6ffd640 1 --2- 172.21.15.55:0/413017442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80cc930 0x7f29f80ccd40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009870 tx=0x7f29ec02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd71af7b2fc1a8b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.709+0000 7f29f67fc640 1 -- 172.21.15.55:0/413017442 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec037070 con 0x7f29f80cc930 2024-05-20T22:11:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.709+0000 7f29f67fc640 1 -- 172.21.15.55:0/413017442 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29ec032440 con 0x7f29f80cc930 2024-05-20T22:11:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.709+0000 7f29f67fc640 1 -- 172.21.15.55:0/413017442 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec032810 con 0x7f29f80cc930 2024-05-20T22:11:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.709+0000 7f29fddb7640 1 -- 172.21.15.55:0/413017442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80cc930 msgr2=0x7f29f80ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.709+0000 7f29fddb7640 1 --2- 172.21.15.55:0/413017442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80cc930 0x7f29f80ccd40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009870 tx=0x7f29ec02a1c0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.709+0000 7f29fddb7640 1 -- 172.21.15.55:0/413017442 shutdown_connections 2024-05-20T22:11:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.709+0000 7f29fddb7640 1 --2- 172.21.15.55:0/413017442 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29f80d0ab0 0x7f29f80d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.709+0000 7f29fddb7640 1 --2- 172.21.15.55:0/413017442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80cc930 0x7f29f80ccd40 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.710+0000 7f29fddb7640 1 -- 172.21.15.55:0/413017442 >> 172.21.15.55:0/413017442 conn(0x7f29f80b65f0 msgr2=0x7f29f80b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.710+0000 7f29fddb7640 1 -- 172.21.15.55:0/413017442 shutdown_connections 2024-05-20T22:11:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.710+0000 7f29fddb7640 1 -- 172.21.15.55:0/413017442 wait complete. 2024-05-20T22:11:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.711+0000 7f29fddb7640 1 Processor -- start 2024-05-20T22:11:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.712+0000 7f29fddb7640 1 -- start start 2024-05-20T22:11:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29fddb7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f809ad90 0x7f29f8160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29fddb7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29f80cc930 0x7f29f8151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29fddb7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80d0ab0 0x7f29f80b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29fddb7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f29f809b1a0 con 0x7f29f80d0ab0 2024-05-20T22:11:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29fddb7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29f80cd280 con 0x7f29f809ad90 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29fddb7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f29f80ca5b0 con 0x7f29f80cc930 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80d0ab0 0x7f29f80b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80d0ab0 0x7f29f80b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48692/0 (socket says 172.21.15.55:48692) 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29f7fff640 1 -- 172.21.15.55:0/4109305846 learned_addr learned my addr 172.21.15.55:0/4109305846 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29f6ffd640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29f80cc930 0x7f29f8151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29f77fe640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f809ad90 0x7f29f8160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29f7fff640 1 -- 172.21.15.55:0/4109305846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29f80cc930 msgr2=0x7f29f8151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29f7fff640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29f80cc930 0x7f29f8151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29f7fff640 1 -- 172.21.15.55:0/4109305846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f809ad90 msgr2=0x7f29f8160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29f7fff640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f809ad90 0x7f29f8160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.713+0000 7f29f7fff640 1 -- 172.21.15.55:0/4109305846 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f81522c0 con 0x7f29f80d0ab0 2024-05-20T22:11:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.714+0000 7f29f7fff640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80d0ab0 0x7f29f80b0ad0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f29f0007d80 tx=0x7f29f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.714+0000 7f29dffff640 1 -- 172.21.15.55:0/4109305846 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0014420 con 0x7f29f80d0ab0 2024-05-20T22:11:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.714+0000 7f29dffff640 1 -- 172.21.15.55:0/4109305846 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f0019070 con 0x7f29f80d0ab0 2024-05-20T22:11:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.714+0000 7f29dffff640 1 -- 172.21.15.55:0/4109305846 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0014800 con 0x7f29f80d0ab0 2024-05-20T22:11:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.714+0000 7f29fddb7640 1 -- 172.21.15.55:0/4109305846 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f80aee30 con 0x7f29f80d0ab0 2024-05-20T22:11:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.714+0000 7f29fddb7640 1 -- 172.21.15.55:0/4109305846 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f80af3f0 con 0x7f29f80d0ab0 2024-05-20T22:11:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.715+0000 7f29fddb7640 1 -- 172.21.15.55:0/4109305846 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f804f720 con 0x7f29f80d0ab0 2024-05-20T22:11:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.715+0000 7f29dffff640 1 -- 172.21.15.55:0/4109305846 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29f00149a0 con 0x7f29f80d0ab0 2024-05-20T22:11:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.716+0000 7f29dffff640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f29e406be60 0x7f29e406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.716+0000 7f29dffff640 1 -- 172.21.15.55:0/4109305846 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f29f008b8f0 con 0x7f29f80d0ab0 2024-05-20T22:11:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.716+0000 7f29f77fe640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f29e406be60 0x7f29e406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.716+0000 7f29f77fe640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f29e406be60 0x7f29e406e370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f29f80d2a00 tx=0x7f29e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:02.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.721+0000 7f29dffff640 1 -- 172.21.15.55:0/4109305846 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f29f0059ff0 con 0x7f29f80d0ab0 2024-05-20T22:11:02.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.908+0000 7f29fddb7640 1 -- 172.21.15.55:0/4109305846 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f29f809a5f0 con 0x7f29f80d0ab0 2024-05-20T22:11:02.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.908+0000 7f29dffff640 1 -- 172.21.15.55:0/4109305846 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) ==== 104+0+11 (secure 0 0 0) 0x7f29f00598f0 con 0x7f29f80d0ab0 2024-05-20T22:11:02.910 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:11:02.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 -- 172.21.15.55:0/4109305846 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f29e406be60 msgr2=0x7f29e406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f29e406be60 0x7f29e406e370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f29f80d2a00 tx=0x7f29e8009040 comp rx=0 tx=0).stop 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 -- 172.21.15.55:0/4109305846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80d0ab0 msgr2=0x7f29f80b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80d0ab0 0x7f29f80b0ad0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f29f0007d80 tx=0x7f29f000e4b0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 -- 172.21.15.55:0/4109305846 shutdown_connections 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f29e406be60 0x7f29e406e370 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29f80d0ab0 0x7f29f80b0ad0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f29f80cc930 0x7f29f8151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 --2- 172.21.15.55:0/4109305846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f809ad90 0x7f29f8160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 -- 172.21.15.55:0/4109305846 >> 172.21.15.55:0/4109305846 conn(0x7f29f80b65f0 msgr2=0x7f29f80704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.910+0000 7f29fddb7640 1 -- 172.21.15.55:0/4109305846 shutdown_connections 2024-05-20T22:11:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:02.911+0000 7f29fddb7640 1 -- 172.21.15.55:0/4109305846 wait complete. 2024-05-20T22:11:02.923 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:11:03.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.040+0000 7f075c5aa640 1 Processor -- start 2024-05-20T22:11:03.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.040+0000 7f075c5aa640 1 -- start start 2024-05-20T22:11:03.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f075c5aa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f07540b40a0 0x7f075415f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f075c5aa640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07540d3c10 con 0x7f0754163700 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f075c5aa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f075406b750 con 0x7f07540b84f0 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f075c5aa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f07540d39c0 con 0x7f07540b40a0 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f075ab47640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0754163700 0x7f075406bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35934/0 (socket says 172.21.15.55:35934) 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f075ab47640 1 -- 172.21.15.55:0/2593988764 learned_addr learned my addr 172.21.15.55:0/2593988764 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f075a346640 1 --2- 172.21.15.55:0/2593988764 >> v2:172.21.15.138:3300/0 conn(0x7f07540b40a0 0x7f075415f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f0759344640 1 -- 172.21.15.55:0/2593988764 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1578412628 0 0) 0x7f07540d3c10 con 0x7f0754163700 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f0759344640 1 -- 172.21.15.55:0/2593988764 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0740003520 con 0x7f0754163700 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f0759344640 1 -- 172.21.15.55:0/2593988764 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4075897355 0 0) 0x7f075406b750 con 0x7f07540b84f0 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f0759344640 1 -- 172.21.15.55:0/2593988764 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07540d3c10 con 0x7f07540b84f0 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f0759344640 1 -- 172.21.15.55:0/2593988764 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2557687431 0 0) 0x7f0740003520 con 0x7f0754163700 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f075a346640 1 -- 172.21.15.55:0/2593988764 >> v1:172.21.15.138:6789/0 conn(0x7f07540b84f0 legacy=0x7f0754162ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.042+0000 7f075a346640 1 -- 172.21.15.55:0/2593988764 >> v1:172.21.15.55:6789/0 conn(0x7f0754163700 legacy=0x7f075406bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.043+0000 7f075a346640 1 -- 172.21.15.55:0/2593988764 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075406ba30 con 0x7f07540b40a0 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.043+0000 7f075a346640 1 --2- 172.21.15.55:0/2593988764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07540b40a0 0x7f075415f490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f07500097a0 tx=0x7f075002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d865117f452ebbd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.043+0000 7f0759344640 1 -- 172.21.15.55:0/2593988764 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750037070 con 0x7f07540b40a0 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.043+0000 7f0759344640 1 -- 172.21.15.55:0/2593988764 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075002abc0 con 0x7f07540b40a0 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.043+0000 7f0759344640 1 -- 172.21.15.55:0/2593988764 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c001a10 con 0x7f07540b40a0 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.044+0000 7f075c5aa640 1 -- 172.21.15.55:0/2593988764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07540b40a0 msgr2=0x7f075415f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.044+0000 7f075c5aa640 1 --2- 172.21.15.55:0/2593988764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07540b40a0 0x7f075415f490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f07500097a0 tx=0x7f075002a130 comp rx=0 tx=0).stop 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.044+0000 7f075c5aa640 1 -- 172.21.15.55:0/2593988764 shutdown_connections 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.044+0000 7f075c5aa640 1 --2- 172.21.15.55:0/2593988764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07540b40a0 0x7f075415f490 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.044+0000 7f075c5aa640 1 -- 172.21.15.55:0/2593988764 >> 172.21.15.55:0/2593988764 conn(0x7f07540d15a0 msgr2=0x7f07540d19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.044+0000 7f075c5aa640 1 -- 172.21.15.55:0/2593988764 shutdown_connections 2024-05-20T22:11:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.044+0000 7f075c5aa640 1 -- 172.21.15.55:0/2593988764 wait complete. 2024-05-20T22:11:03.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.046+0000 7f075c5aa640 1 Processor -- start 2024-05-20T22:11:03.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.046+0000 7f075c5aa640 1 -- start start 2024-05-20T22:11:03.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.047+0000 7f075c5aa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07540b40a0 0x7f07540c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.047+0000 7f075c5aa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07540b84f0 0x7f0754160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.047+0000 7f075c5aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0754163700 0x7f07541458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.047+0000 7f075c5aa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0754163c90 con 0x7f07540b84f0 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.047+0000 7f075c5aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0754163b10 con 0x7f0754163700 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.047+0000 7f075c5aa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0754163e10 con 0x7f07540b40a0 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.047+0000 7f0759b45640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07540b84f0 0x7f0754160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.047+0000 7f0759b45640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07540b84f0 0x7f0754160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48704/0 (socket says 172.21.15.55:48704) 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.047+0000 7f0759b45640 1 -- 172.21.15.55:0/459650620 learned_addr learned my addr 172.21.15.55:0/459650620 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f075a346640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07540b40a0 0x7f07540c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f075ab47640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0754163700 0x7f07541458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f0759b45640 1 -- 172.21.15.55:0/459650620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07540b40a0 msgr2=0x7f07540c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f0759b45640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07540b40a0 0x7f07540c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f0759b45640 1 -- 172.21.15.55:0/459650620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0754163700 msgr2=0x7f07541458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f0759b45640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0754163700 0x7f07541458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f0759b45640 1 -- 172.21.15.55:0/459650620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0754146670 con 0x7f07540b84f0 2024-05-20T22:11:03.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f0759b45640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07540b84f0 0x7f0754160c90 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f074400c3c0 tx=0x7f074400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:03.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f074affd640 1 -- 172.21.15.55:0/459650620 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744017070 con 0x7f07540b84f0 2024-05-20T22:11:03.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f074affd640 1 -- 172.21.15.55:0/459650620 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07440052f0 con 0x7f07540b84f0 2024-05-20T22:11:03.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f075c5aa640 1 -- 172.21.15.55:0/459650620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0754146910 con 0x7f07540b84f0 2024-05-20T22:11:03.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f074affd640 1 -- 172.21.15.55:0/459650620 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744003830 con 0x7f07540b84f0 2024-05-20T22:11:03.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.048+0000 7f075c5aa640 1 -- 172.21.15.55:0/459650620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07540c9190 con 0x7f07540b84f0 2024-05-20T22:11:03.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.049+0000 7f075c5aa640 1 -- 172.21.15.55:0/459650620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07540c8f00 con 0x7f07540b84f0 2024-05-20T22:11:03.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.050+0000 7f074affd640 1 -- 172.21.15.55:0/459650620 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f074401a070 con 0x7f07540b84f0 2024-05-20T22:11:03.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.050+0000 7f074affd640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f073806be60 0x7f073806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.051+0000 7f075a346640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f073806be60 0x7f073806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.051+0000 7f074affd640 1 -- 172.21.15.55:0/459650620 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f074408a470 con 0x7f07540b84f0 2024-05-20T22:11:03.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.051+0000 7f075a346640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f073806be60 0x7f073806e370 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0750034000 tx=0x7f075002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:03.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.055+0000 7f074affd640 1 -- 172.21.15.55:0/459650620 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0744058c40 con 0x7f07540b84f0 2024-05-20T22:11:03.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.236+0000 7f075c5aa640 1 -- 172.21.15.55:0/459650620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f075404f720 con 0x7f07540b84f0 2024-05-20T22:11:03.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.237+0000 7f074affd640 1 -- 172.21.15.55:0/459650620 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) ==== 100+0+11 (secure 0 0 0) 0x7f0744058540 con 0x7f07540b84f0 2024-05-20T22:11:03.238 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 -- 172.21.15.55:0/459650620 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f073806be60 msgr2=0x7f073806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f073806be60 0x7f073806e370 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0750034000 tx=0x7f075002e000 comp rx=0 tx=0).stop 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 -- 172.21.15.55:0/459650620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07540b84f0 msgr2=0x7f0754160c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07540b84f0 0x7f0754160c90 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f074400c3c0 tx=0x7f074400c880 comp rx=0 tx=0).stop 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 -- 172.21.15.55:0/459650620 shutdown_connections 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f073806be60 0x7f073806e370 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0754163700 0x7f07541458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07540b84f0 0x7f0754160c90 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 --2- 172.21.15.55:0/459650620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07540b40a0 0x7f07540c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 -- 172.21.15.55:0/459650620 >> 172.21.15.55:0/459650620 conn(0x7f07540d15a0 msgr2=0x7f075406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 -- 172.21.15.55:0/459650620 shutdown_connections 2024-05-20T22:11:03.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.239+0000 7f075c5aa640 1 -- 172.21.15.55:0/459650620 wait complete. 2024-05-20T22:11:03.251 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T22:11:03.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.366+0000 7efddc273640 1 Processor -- start 2024-05-20T22:11:03.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.366+0000 7efddc273640 1 -- start start 2024-05-20T22:11:03.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efddc273640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efdd40d3460 0x7efdd415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efddc273640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efdd415cd00 0x7efdd40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efddc273640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd40d0110 con 0x7efdd40cfd00 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efddc273640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7efdd415d110 con 0x7efdd40d3460 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efddc273640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efdd40d3a20 con 0x7efdd415cd00 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdda00f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efdd40d3460 0x7efdd415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdda00f640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efdd40d3460 0x7efdd415f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48714/0 (socket says 172.21.15.55:48714) 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdda00f640 1 -- 172.21.15.55:0/2251852572 learned_addr learned my addr 172.21.15.55:0/2251852572 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdda810640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efdd40cfd00 0x7efdd40b3680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35938/0 (socket says 172.21.15.55:35938) 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdd980e640 1 --2- 172.21.15.55:0/2251852572 >> v2:172.21.15.138:3300/0 conn(0x7efdd415cd00 0x7efdd40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdda00f640 1 -- 172.21.15.55:0/2251852572 >> v2:172.21.15.138:3300/0 conn(0x7efdd415cd00 msgr2=0x7efdd40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdda00f640 1 --2- 172.21.15.55:0/2251852572 >> v2:172.21.15.138:3300/0 conn(0x7efdd415cd00 0x7efdd40cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdda00f640 1 -- 172.21.15.55:0/2251852572 >> v1:172.21.15.55:6789/0 conn(0x7efdd40cfd00 legacy=0x7efdd40b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdd900d640 1 -- 172.21.15.55:0/2251852572 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350158017 0 0) 0x7efdcc001110 con 0x7efdd40cfd00 2024-05-20T22:11:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdda00f640 1 -- 172.21.15.55:0/2251852572 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd40d3d00 con 0x7efdd40d3460 2024-05-20T22:11:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.368+0000 7efdd980e640 1 --2- 172.21.15.55:0/2251852572 >> v2:172.21.15.138:3300/0 conn(0x7efdd415cd00 0x7efdd40cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.369+0000 7efdda00f640 1 --2- 172.21.15.55:0/2251852572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd40d3460 0x7efdd415f6e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7efdd40cbb70 tx=0x7efdd002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa1d9be2537e723 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.369+0000 7efdd900d640 1 -- 172.21.15.55:0/2251852572 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0036070 con 0x7efdd40d3460 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.369+0000 7efdd900d640 1 -- 172.21.15.55:0/2251852572 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0031600 con 0x7efdd40d3460 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.369+0000 7efdd900d640 1 -- 172.21.15.55:0/2251852572 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00319d0 con 0x7efdd40d3460 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.370+0000 7efddc273640 1 -- 172.21.15.55:0/2251852572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd40d3460 msgr2=0x7efdd415f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.370+0000 7efddc273640 1 --2- 172.21.15.55:0/2251852572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd40d3460 0x7efdd415f6e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7efdd40cbb70 tx=0x7efdd002a730 comp rx=0 tx=0).stop 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.370+0000 7efddc273640 1 -- 172.21.15.55:0/2251852572 shutdown_connections 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.370+0000 7efddc273640 1 --2- 172.21.15.55:0/2251852572 >> v2:172.21.15.138:3300/0 conn(0x7efdd415cd00 0x7efdd40cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.370+0000 7efddc273640 1 --2- 172.21.15.55:0/2251852572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd40d3460 0x7efdd415f6e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.370+0000 7efddc273640 1 -- 172.21.15.55:0/2251852572 >> 172.21.15.55:0/2251852572 conn(0x7efdd40d1c70 msgr2=0x7efdd40d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.370+0000 7efddc273640 1 -- 172.21.15.55:0/2251852572 shutdown_connections 2024-05-20T22:11:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.370+0000 7efddc273640 1 -- 172.21.15.55:0/2251852572 wait complete. 2024-05-20T22:11:03.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.372+0000 7efddc273640 1 Processor -- start 2024-05-20T22:11:03.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.372+0000 7efddc273640 1 -- start start 2024-05-20T22:11:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efddc273640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efdd40cfd00 0x7efdd41e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efddc273640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd40d3460 0x7efdd41573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efddc273640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd415cd00 0x7efdd41578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efddc273640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40d04a0 con 0x7efdd415cd00 2024-05-20T22:11:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efddc273640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40d42c0 con 0x7efdd40d3460 2024-05-20T22:11:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efddc273640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efdd40d0620 con 0x7efdd40cfd00 2024-05-20T22:11:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdda810640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd415cd00 0x7efdd41578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdda810640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd415cd00 0x7efdd41578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48726/0 (socket says 172.21.15.55:48726) 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdda810640 1 -- 172.21.15.55:0/4107141945 learned_addr learned my addr 172.21.15.55:0/4107141945 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdda00f640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efdd40cfd00 0x7efdd41e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdd980e640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd40d3460 0x7efdd41573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdda810640 1 -- 172.21.15.55:0/4107141945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efdd40cfd00 msgr2=0x7efdd41e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdda810640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efdd40cfd00 0x7efdd41e8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdda810640 1 -- 172.21.15.55:0/4107141945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd40d3460 msgr2=0x7efdd41573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdda810640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd40d3460 0x7efdd41573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdda810640 1 -- 172.21.15.55:0/4107141945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd4154b90 con 0x7efdd415cd00 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdda00f640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efdd40cfd00 0x7efdd41e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.373+0000 7efdd980e640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd40d3460 0x7efdd41573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.374+0000 7efdda810640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd415cd00 0x7efdd41578e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c3e0 tx=0x7efdcc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.374+0000 7efdcaffd640 1 -- 172.21.15.55:0/4107141945 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0074e0 con 0x7efdd415cd00 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.374+0000 7efdcaffd640 1 -- 172.21.15.55:0/4107141945 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdcc0047e0 con 0x7efdd415cd00 2024-05-20T22:11:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.374+0000 7efddc273640 1 -- 172.21.15.55:0/4107141945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd4154e70 con 0x7efdd415cd00 2024-05-20T22:11:03.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.374+0000 7efdcaffd640 1 -- 172.21.15.55:0/4107141945 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc004bd0 con 0x7efdd415cd00 2024-05-20T22:11:03.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.374+0000 7efddc273640 1 -- 172.21.15.55:0/4107141945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd4155430 con 0x7efdd415cd00 2024-05-20T22:11:03.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.375+0000 7efddc273640 1 -- 172.21.15.55:0/4107141945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd40d11c0 con 0x7efdd415cd00 2024-05-20T22:11:03.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.375+0000 7efdcaffd640 1 -- 172.21.15.55:0/4107141945 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efdcc011070 con 0x7efdd415cd00 2024-05-20T22:11:03.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.376+0000 7efdcaffd640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efdb806be30 0x7efdb806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:03.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.376+0000 7efdcaffd640 1 -- 172.21.15.55:0/4107141945 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7efdcc08ad80 con 0x7efdd415cd00 2024-05-20T22:11:03.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.376+0000 7efdda00f640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efdb806be30 0x7efdb806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:03.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.377+0000 7efdda00f640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efdb806be30 0x7efdb806e340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efdd0033000 tx=0x7efdd0009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:03.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.381+0000 7efdcaffd640 1 -- 172.21.15.55:0/4107141945 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efdd40d11c0 con 0x7efdd415cd00 2024-05-20T22:11:03.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.578+0000 7efddc273640 1 -- 172.21.15.55:0/4107141945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efdd40d2f80 con 0x7efdd415cd00 2024-05-20T22:11:03.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.578+0000 7efdcaffd640 1 -- 172.21.15.55:0/4107141945 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v61) ==== 80+0+122 (secure 0 0 0) 0x7efdd40d0110 con 0x7efdd415cd00 2024-05-20T22:11:03.579 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6837/2445634740 2024-05-21T22:10:38.297533+0000 2024-05-20T22:11:03.580 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6835/2445634740 2024-05-21T22:10:38.297533+0000 2024-05-20T22:11:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:listed 2 entries 2024-05-20T22:11:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 -- 172.21.15.55:0/4107141945 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efdb806be30 msgr2=0x7efdb806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efdb806be30 0x7efdb806e340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efdd0033000 tx=0x7efdd0009000 comp rx=0 tx=0).stop 2024-05-20T22:11:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 -- 172.21.15.55:0/4107141945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd415cd00 msgr2=0x7efdd41578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd415cd00 0x7efdd41578e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c3e0 tx=0x7efdcc00c8a0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 -- 172.21.15.55:0/4107141945 shutdown_connections 2024-05-20T22:11:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7efdb806be30 0x7efdb806e340 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdd415cd00 0x7efdd41578e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd40d3460 0x7efdd41573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 --2- 172.21.15.55:0/4107141945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efdd40cfd00 0x7efdd41e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 -- 172.21.15.55:0/4107141945 >> 172.21.15.55:0/4107141945 conn(0x7efdd40d1c70 msgr2=0x7efdd415d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:03.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 -- 172.21.15.55:0/4107141945 shutdown_connections 2024-05-20T22:11:03.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:03.580+0000 7efddc273640 1 -- 172.21.15.55:0/4107141945 wait complete. 2024-05-20T22:11:03.592 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi055.front.sepia.ceph.com... 2024-05-20T22:11:03.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:03.593 DEBUG:teuthology.orchestra.run.smithi055:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-20T22:11:03.617 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi055.stderr:ceph-fuse[58032]: fuse finished with error 0 and tester_r 0 2024-05-20T22:11:03.619 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:11:03.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:03.619 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T22:11:03.711 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T22:11:03.711 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:11:03.712 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:11:03.731 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=450 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T22:11:03.731 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:11:03.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:03.731 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-20T22:11:03.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:11:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-20T22:11:03.789 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T22:11:03.789 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:11:03.789 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:11:03.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3618 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T22:11:03.806 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:11:03.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:03.807 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-20T22:11:03.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:11:03.863 INFO:teuthology.orchestra.run.smithi055.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-20T22:11:03.863 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T22:11:03.863 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:11:03.863 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:11:03.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=450 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T22:11:03.881 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:11:03.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:03.881 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-20T22:11:03.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:11:03.938 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-20T22:11:03.938 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:11:04.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.052+0000 7fd5deeb8640 1 Processor -- start 2024-05-20T22:11:04.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.052+0000 7fd5deeb8640 1 -- start start 2024-05-20T22:11:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5deeb8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd5d80d42a0 0x7fd5d80cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5deeb8640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d8160550 con 0x7fd5d815ccc0 2024-05-20T22:11:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5deeb8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d815f9d0 con 0x7fd5d80d3d80 2024-05-20T22:11:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5deeb8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d815fcb0 con 0x7fd5d80d42a0 2024-05-20T22:11:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d7fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd5d815ccc0 0x7fd5d80d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35954/0 (socket says 172.21.15.55:35954) 2024-05-20T22:11:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d7fff640 1 -- 172.21.15.55:0/354335419 learned_addr learned my addr 172.21.15.55:0/354335419 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5dcc54640 1 --2- 172.21.15.55:0/354335419 >> v2:172.21.15.138:3300/0 conn(0x7fd5d80d42a0 0x7fd5d80cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1761382602 0 0) 0x7fd5d8160550 con 0x7fd5d815ccc0 2024-05-20T22:11:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5c0003520 con 0x7fd5d815ccc0 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3873013174 0 0) 0x7fd5d815f9d0 con 0x7fd5d80d3d80 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5d8160550 con 0x7fd5d80d3d80 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2784467050 0 0) 0x7fd5c0003520 con 0x7fd5d815ccc0 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 >> v2:172.21.15.138:3300/0 conn(0x7fd5d80d42a0 msgr2=0x7fd5d80cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 --2- 172.21.15.55:0/354335419 >> v2:172.21.15.138:3300/0 conn(0x7fd5d80d42a0 0x7fd5d80cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 >> v1:172.21.15.138:6789/0 conn(0x7fd5d80d3d80 legacy=0x7fd5d80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d815ff90 con 0x7fd5d815ccc0 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fd5cc001670 con 0x7fd5d815ccc0 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.054+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 >> v1:172.21.15.55:6789/0 conn(0x7fd5d815ccc0 legacy=0x7fd5d80d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d77fe640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c0003b40 0x7fd5c0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d77fe640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5c0004800 0x7fd5c0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d77fe640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c0005260 0x7fd5c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d815fcb0 con 0x7fd5c0004800 2024-05-20T22:11:04.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c000c1c0 con 0x7fd5c0005260 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c000c3c0 con 0x7fd5c0003b40 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5dcc54640 1 --2- 172.21.15.55:0/354335419 >> v2:172.21.15.138:3300/0 conn(0x7fd5d80d42a0 0x7fd5d80cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d7fff640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5c0004800 0x7fd5c0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5dd455640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c0005260 0x7fd5c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5dcc54640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c0003b40 0x7fd5c0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d7fff640 1 -- 172.21.15.55:0/354335419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c0003b40 msgr2=0x7fd5c0003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d7fff640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5c0003b40 0x7fd5c0003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d7fff640 1 -- 172.21.15.55:0/354335419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c0005260 msgr2=0x7fd5c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d7fff640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c0005260 0x7fd5c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5dd455640 1 -- 172.21.15.55:0/354335419 reap_dead start 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.055+0000 7fd5d7fff640 1 -- 172.21.15.55:0/354335419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c0003520 con 0x7fd5c0004800 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.056+0000 7fd5d7fff640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5c0004800 0x7fd5c0004c10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00c190 tx=0x7fd5c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c49495c0df2433 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.056+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc007cc0 con 0x7fd5c0004800 2024-05-20T22:11:04.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.056+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5cc005af0 con 0x7fd5c0004800 2024-05-20T22:11:04.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.056+0000 7fd5d77fe640 1 -- 172.21.15.55:0/354335419 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc007070 con 0x7fd5c0004800 2024-05-20T22:11:04.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.057+0000 7fd5deeb8640 1 -- 172.21.15.55:0/354335419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5c0004800 msgr2=0x7fd5c0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.057+0000 7fd5deeb8640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5c0004800 0x7fd5c0004c10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00c190 tx=0x7fd5c0005830 comp rx=0 tx=0).stop 2024-05-20T22:11:04.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.057+0000 7fd5deeb8640 1 -- 172.21.15.55:0/354335419 shutdown_connections 2024-05-20T22:11:04.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.057+0000 7fd5deeb8640 1 --2- 172.21.15.55:0/354335419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5c0004800 0x7fd5c0004c10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.057+0000 7fd5deeb8640 1 -- 172.21.15.55:0/354335419 >> 172.21.15.55:0/354335419 conn(0x7fd5d80d1c50 msgr2=0x7fd5d80d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:04.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.057+0000 7fd5deeb8640 1 -- 172.21.15.55:0/354335419 shutdown_connections 2024-05-20T22:11:04.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.057+0000 7fd5deeb8640 1 -- 172.21.15.55:0/354335419 wait complete. 2024-05-20T22:11:04.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.059+0000 7fd5deeb8640 1 Processor -- start 2024-05-20T22:11:04.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.059+0000 7fd5deeb8640 1 -- start start 2024-05-20T22:11:04.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5deeb8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c0004800 0x7fd5d80cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5deeb8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5d8159040 0x7fd5d80cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5deeb8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5d80cdc00 0x7fd5d8162f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5deeb8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d815f7b0 con 0x7fd5d8159040 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5deeb8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d815f630 con 0x7fd5c0004800 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5deeb8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d80d42a0 con 0x7fd5d80cdc00 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5d7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5d8159040 0x7fd5d80cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5d7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5d8159040 0x7fd5d80cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48742/0 (socket says 172.21.15.55:48742) 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5d7fff640 1 -- 172.21.15.55:0/1991568725 learned_addr learned my addr 172.21.15.55:0/1991568725 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5d7fff640 1 -- 172.21.15.55:0/1991568725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5d80cdc00 msgr2=0x7fd5d8162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5dd455640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5d80cdc00 0x7fd5d8162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5dcc54640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c0004800 0x7fd5d80cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5d7fff640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5d80cdc00 0x7fd5d8162f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5d7fff640 1 -- 172.21.15.55:0/1991568725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c0004800 msgr2=0x7fd5d80cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5d7fff640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c0004800 0x7fd5d80cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.060+0000 7fd5d7fff640 1 -- 172.21.15.55:0/1991568725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d81f9010 con 0x7fd5d8159040 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.061+0000 7fd5d7fff640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5d8159040 0x7fd5d80cd6c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00c190 tx=0x7fd5d80d10a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:04.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.061+0000 7fd5d57fa640 1 -- 172.21.15.55:0/1991568725 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc040030 con 0x7fd5d8159040 2024-05-20T22:11:04.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.061+0000 7fd5d57fa640 1 -- 172.21.15.55:0/1991568725 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5cc001470 con 0x7fd5d8159040 2024-05-20T22:11:04.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.061+0000 7fd5deeb8640 1 -- 172.21.15.55:0/1991568725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d81f8490 con 0x7fd5d8159040 2024-05-20T22:11:04.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.061+0000 7fd5d57fa640 1 -- 172.21.15.55:0/1991568725 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc037040 con 0x7fd5d8159040 2024-05-20T22:11:04.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.061+0000 7fd5deeb8640 1 -- 172.21.15.55:0/1991568725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d81f8a50 con 0x7fd5d8159040 2024-05-20T22:11:04.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.062+0000 7fd5deeb8640 1 -- 172.21.15.55:0/1991568725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d81f8200 con 0x7fd5d8159040 2024-05-20T22:11:04.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.062+0000 7fd5d57fa640 1 -- 172.21.15.55:0/1991568725 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd5cc043070 con 0x7fd5d8159040 2024-05-20T22:11:04.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.063+0000 7fd5d57fa640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd5c406be60 0x7fd5c406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.063+0000 7fd5d57fa640 1 -- 172.21.15.55:0/1991568725 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7fd5cc0aad40 con 0x7fd5d8159040 2024-05-20T22:11:04.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.063+0000 7fd5dcc54640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd5c406be60 0x7fd5c406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.063+0000 7fd5dcc54640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd5c406be60 0x7fd5c406e370 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80090e0 tx=0x7fd5c8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:04.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.068+0000 7fd5d57fa640 1 -- 172.21.15.55:0/1991568725 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd5cc079560 con 0x7fd5d8159040 2024-05-20T22:11:04.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.288+0000 7fd5deeb8640 1 -- 172.21.15.55:0/1991568725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5d81f8200 con 0x7fd5d8159040 2024-05-20T22:11:04.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.289+0000 7fd5d57fa640 1 -- 172.21.15.55:0/1991568725 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7fd5cc078e60 con 0x7fd5d8159040 2024-05-20T22:11:04.290 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:04.290 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:11:04.290 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:11:04.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 -- 172.21.15.55:0/1991568725 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd5c406be60 msgr2=0x7fd5c406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd5c406be60 0x7fd5c406e370 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80090e0 tx=0x7fd5c8008040 comp rx=0 tx=0).stop 2024-05-20T22:11:04.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 -- 172.21.15.55:0/1991568725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5d8159040 msgr2=0x7fd5d80cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5d8159040 0x7fd5d80cd6c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00c190 tx=0x7fd5d80d10a0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 -- 172.21.15.55:0/1991568725 shutdown_connections 2024-05-20T22:11:04.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fd5c406be60 0x7fd5c406e370 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5d80cdc00 0x7fd5d8162f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5d8159040 0x7fd5d80cd6c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 --2- 172.21.15.55:0/1991568725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5c0004800 0x7fd5d80cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 -- 172.21.15.55:0/1991568725 >> 172.21.15.55:0/1991568725 conn(0x7fd5d80d1c50 msgr2=0x7fd5d815d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:04.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 -- 172.21.15.55:0/1991568725 shutdown_connections 2024-05-20T22:11:04.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.291+0000 7fd5deeb8640 1 -- 172.21.15.55:0/1991568725 wait complete. 2024-05-20T22:11:04.303 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:11:04.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.418+0000 7f6d5c60d640 1 Processor -- start 2024-05-20T22:11:04.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.418+0000 7f6d5c60d640 1 -- start start 2024-05-20T22:11:04.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.419+0000 7f6d5c60d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6d5406f0f0 0x7f6d5406f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.419+0000 7f6d5c60d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5406fa40 con 0x7f6d540b5b80 2024-05-20T22:11:04.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.419+0000 7f6d5c60d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d54161380 con 0x7f6d540cd290 2024-05-20T22:11:04.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.419+0000 7f6d5c60d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6d540d1090 con 0x7f6d5406f0f0 2024-05-20T22:11:04.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.419+0000 7f6d5a3a9640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6d540b5b80 0x7f6d54160bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35964/0 (socket says 172.21.15.55:35964) 2024-05-20T22:11:04.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.419+0000 7f6d5a3a9640 1 -- 172.21.15.55:0/1505438288 learned_addr learned my addr 172.21.15.55:0/1505438288 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:04.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.420+0000 7f6d5abaa640 1 --2- 172.21.15.55:0/1505438288 >> v2:172.21.15.138:3300/0 conn(0x7f6d5406f0f0 0x7f6d5406f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.420+0000 7f6d593a7640 1 -- 172.21.15.55:0/1505438288 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 312472587 0 0) 0x7f6d50001110 con 0x7f6d540b5b80 2024-05-20T22:11:04.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.420+0000 7f6d593a7640 1 -- 172.21.15.55:0/1505438288 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d40003520 con 0x7f6d540b5b80 2024-05-20T22:11:04.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.420+0000 7f6d593a7640 1 -- 172.21.15.55:0/1505438288 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2208760315 0 0) 0x7f6d54161380 con 0x7f6d540cd290 2024-05-20T22:11:04.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.420+0000 7f6d593a7640 1 -- 172.21.15.55:0/1505438288 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d50001110 con 0x7f6d540cd290 2024-05-20T22:11:04.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.420+0000 7f6d5abaa640 1 -- 172.21.15.55:0/1505438288 >> v1:172.21.15.138:6789/0 conn(0x7f6d540cd290 legacy=0x7f6d5415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.420+0000 7f6d5abaa640 1 -- 172.21.15.55:0/1505438288 >> v1:172.21.15.55:6789/0 conn(0x7f6d540b5b80 legacy=0x7f6d54160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.420+0000 7f6d593a7640 1 -- 172.21.15.55:0/1505438288 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1139607843 0 0) 0x7f6d40003520 con 0x7f6d540b5b80 2024-05-20T22:11:04.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.420+0000 7f6d5abaa640 1 -- 172.21.15.55:0/1505438288 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d54161660 con 0x7f6d5406f0f0 2024-05-20T22:11:04.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.421+0000 7f6d5abaa640 1 --2- 172.21.15.55:0/1505438288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5406f0f0 0x7f6d5406f500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009d90 tx=0x7f6d4c02a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe6cc8b5362abc14 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:04.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.421+0000 7f6d593a7640 1 -- 172.21.15.55:0/1505438288 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50001840 con 0x7f6d5406f0f0 2024-05-20T22:11:04.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.421+0000 7f6d593a7640 1 -- 172.21.15.55:0/1505438288 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4c007540 con 0x7f6d5406f0f0 2024-05-20T22:11:04.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.421+0000 7f6d593a7640 1 -- 172.21.15.55:0/1505438288 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0078f0 con 0x7f6d5406f0f0 2024-05-20T22:11:04.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.422+0000 7f6d5c60d640 1 -- 172.21.15.55:0/1505438288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5406f0f0 msgr2=0x7f6d5406f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.422+0000 7f6d5c60d640 1 --2- 172.21.15.55:0/1505438288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5406f0f0 0x7f6d5406f500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009d90 tx=0x7f6d4c02a560 comp rx=0 tx=0).stop 2024-05-20T22:11:04.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.422+0000 7f6d5c60d640 1 -- 172.21.15.55:0/1505438288 shutdown_connections 2024-05-20T22:11:04.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.422+0000 7f6d5c60d640 1 --2- 172.21.15.55:0/1505438288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5406f0f0 0x7f6d5406f500 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.422+0000 7f6d5c60d640 1 -- 172.21.15.55:0/1505438288 >> 172.21.15.55:0/1505438288 conn(0x7f6d540d2910 msgr2=0x7f6d540d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:04.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.422+0000 7f6d5c60d640 1 -- 172.21.15.55:0/1505438288 shutdown_connections 2024-05-20T22:11:04.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.422+0000 7f6d5c60d640 1 -- 172.21.15.55:0/1505438288 wait complete. 2024-05-20T22:11:04.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.424+0000 7f6d5c60d640 1 Processor -- start 2024-05-20T22:11:04.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.424+0000 7f6d5c60d640 1 -- start start 2024-05-20T22:11:04.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d5c60d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5406f0f0 0x7f6d540720d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d5c60d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540b5b80 0x7f6d540c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d5c60d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d540cd290 0x7f6d540c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d5c60d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54167ef0 con 0x7f6d540b5b80 2024-05-20T22:11:04.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d5c60d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d540d12e0 con 0x7f6d5406f0f0 2024-05-20T22:11:04.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d5c60d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6d54168070 con 0x7f6d540cd290 2024-05-20T22:11:04.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d59ba8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540b5b80 0x7f6d540c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d59ba8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540b5b80 0x7f6d540c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48752/0 (socket says 172.21.15.55:48752) 2024-05-20T22:11:04.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d59ba8640 1 -- 172.21.15.55:0/2395190070 learned_addr learned my addr 172.21.15.55:0/2395190070 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d5a3a9640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5406f0f0 0x7f6d540720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d5abaa640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d540cd290 0x7f6d540c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d59ba8640 1 -- 172.21.15.55:0/2395190070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d540cd290 msgr2=0x7f6d540c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d59ba8640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d540cd290 0x7f6d540c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d59ba8640 1 -- 172.21.15.55:0/2395190070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5406f0f0 msgr2=0x7f6d540720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d59ba8640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5406f0f0 0x7f6d540720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d59ba8640 1 -- 172.21.15.55:0/2395190070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d540c91c0 con 0x7f6d540b5b80 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.425+0000 7f6d5abaa640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d540cd290 0x7f6d540c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.426+0000 7f6d59ba8640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540b5b80 0x7f6d540c8470 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c3f0 tx=0x7f6d4400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.426+0000 7f6d4affd640 1 -- 172.21.15.55:0/2395190070 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44017070 con 0x7f6d540b5b80 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.426+0000 7f6d4affd640 1 -- 172.21.15.55:0/2395190070 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d440052f0 con 0x7f6d540b5b80 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.426+0000 7f6d5c60d640 1 -- 172.21.15.55:0/2395190070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5415d6a0 con 0x7f6d540b5b80 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.426+0000 7f6d4affd640 1 -- 172.21.15.55:0/2395190070 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44003830 con 0x7f6d540b5b80 2024-05-20T22:11:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.426+0000 7f6d5c60d640 1 -- 172.21.15.55:0/2395190070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5415dc60 con 0x7f6d540b5b80 2024-05-20T22:11:04.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.427+0000 7f6d5c60d640 1 -- 172.21.15.55:0/2395190070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5404f720 con 0x7f6d540b5b80 2024-05-20T22:11:04.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.427+0000 7f6d4affd640 1 -- 172.21.15.55:0/2395190070 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d4401a070 con 0x7f6d540b5b80 2024-05-20T22:11:04.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.428+0000 7f6d4affd640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d3806be30 0x7f6d3806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.428+0000 7f6d4affd640 1 -- 172.21.15.55:0/2395190070 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f6d4408a550 con 0x7f6d540b5b80 2024-05-20T22:11:04.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.428+0000 7f6d5a3a9640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d3806be30 0x7f6d3806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.428+0000 7f6d5a3a9640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d3806be30 0x7f6d3806e340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6d500013b0 tx=0x7f6d50001d50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:04.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.432+0000 7f6d4affd640 1 -- 172.21.15.55:0/2395190070 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6d440585a0 con 0x7f6d540b5b80 2024-05-20T22:11:04.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.617+0000 7f6d5c60d640 1 -- 172.21.15.55:0/2395190070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d540c8f50 con 0x7f6d540b5b80 2024-05-20T22:11:04.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.618+0000 7f6d4affd640 1 -- 172.21.15.55:0/2395190070 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+17167 (secure 0 0 0) 0x7f6d4405c8a0 con 0x7f6d540b5b80 2024-05-20T22:11:04.626 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:04.626 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":61,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:10:38.297580+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:10:22.369451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T22:10:22.970959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.25","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:6837/2445634740":"2024-05-21T22:10:38.297533+0000","172.21.15.55:6835/2445634740":"2024-05-21T22:10:38.297533+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:11:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.620+0000 7f6d5c60d640 1 -- 172.21.15.55:0/2395190070 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d3806be30 msgr2=0x7f6d3806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.620+0000 7f6d5c60d640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d3806be30 0x7f6d3806e340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6d500013b0 tx=0x7f6d50001d50 comp rx=0 tx=0).stop 2024-05-20T22:11:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.620+0000 7f6d5c60d640 1 -- 172.21.15.55:0/2395190070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540b5b80 msgr2=0x7f6d540c8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.620+0000 7f6d5c60d640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540b5b80 0x7f6d540c8470 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c3f0 tx=0x7f6d4400c8b0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.620+0000 7f6d5c60d640 1 -- 172.21.15.55:0/2395190070 shutdown_connections 2024-05-20T22:11:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.620+0000 7f6d5c60d640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6d3806be30 0x7f6d3806e340 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.621+0000 7f6d5c60d640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d540cd290 0x7f6d540c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.621+0000 7f6d5c60d640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d540b5b80 0x7f6d540c8470 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.621+0000 7f6d5c60d640 1 --2- 172.21.15.55:0/2395190070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5406f0f0 0x7f6d540720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.621+0000 7f6d5c60d640 1 -- 172.21.15.55:0/2395190070 >> 172.21.15.55:0/2395190070 conn(0x7f6d540d2910 msgr2=0x7f6d5415cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.621+0000 7f6d5c60d640 1 -- 172.21.15.55:0/2395190070 shutdown_connections 2024-05-20T22:11:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.621+0000 7f6d5c60d640 1 -- 172.21.15.55:0/2395190070 wait complete. 2024-05-20T22:11:04.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T22:07:40.373261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T22:10:22.369451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T22:10:22.970959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T22:11:04.634 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.752+0000 7f5ac4d45640 1 Processor -- start 2024-05-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.752+0000 7f5ac4d45640 1 -- start start 2024-05-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.753+0000 7f5ac4d45640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ac00d3460 0x7f5ac015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.753+0000 7f5ac4d45640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5ac015cd00 0x7f5ac00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.753+0000 7f5ac4d45640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac00d0110 con 0x7f5ac00cfd00 2024-05-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.753+0000 7f5ac4d45640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac015d110 con 0x7f5ac00d3460 2024-05-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.753+0000 7f5ac4d45640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac00d3a20 con 0x7f5ac015cd00 2024-05-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.753+0000 7f5abe59c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ac00d3460 0x7f5ac015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.753+0000 7f5abe59c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ac00d3460 0x7f5ac015f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48762/0 (socket says 172.21.15.55:48762) 2024-05-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.753+0000 7f5abe59c640 1 -- 172.21.15.55:0/2342867925 learned_addr learned my addr 172.21.15.55:0/2342867925 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.754+0000 7f5ab5d9b640 1 --2- 172.21.15.55:0/2342867925 >> v2:172.21.15.138:3300/0 conn(0x7f5ac015cd00 0x7f5ac00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.754+0000 7f5abe59c640 1 -- 172.21.15.55:0/2342867925 >> v2:172.21.15.138:3300/0 conn(0x7f5ac015cd00 msgr2=0x7f5ac00cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.754+0000 7f5abe59c640 1 --2- 172.21.15.55:0/2342867925 >> v2:172.21.15.138:3300/0 conn(0x7f5ac015cd00 0x7f5ac00cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.754+0000 7f5abe59c640 1 -- 172.21.15.55:0/2342867925 >> v1:172.21.15.55:6789/0 conn(0x7f5ac00cfd00 legacy=0x7f5ac00b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.754+0000 7f5abe59c640 1 -- 172.21.15.55:0/2342867925 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac00d3d00 con 0x7f5ac00d3460 2024-05-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.754+0000 7f5abe59c640 1 --2- 172.21.15.55:0/2342867925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 0x7f5ac015f6e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00cbb70 tx=0x7f5ab002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e69a8f6bc22e33 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5abdd9b640 1 -- 172.21.15.55:0/2342867925 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0037070 con 0x7f5ac00d3460 2024-05-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5abdd9b640 1 -- 172.21.15.55:0/2342867925 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0032440 con 0x7f5ac00d3460 2024-05-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5abdd9b640 1 -- 172.21.15.55:0/2342867925 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0032810 con 0x7f5ac00d3460 2024-05-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5ac4d45640 1 -- 172.21.15.55:0/2342867925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 msgr2=0x7f5ac015f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5ac4d45640 1 --2- 172.21.15.55:0/2342867925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 0x7f5ac015f6e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00cbb70 tx=0x7f5ab002a270 comp rx=0 tx=0).stop 2024-05-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5ac4d45640 1 -- 172.21.15.55:0/2342867925 shutdown_connections 2024-05-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5ac4d45640 1 --2- 172.21.15.55:0/2342867925 >> v2:172.21.15.138:3300/0 conn(0x7f5ac015cd00 0x7f5ac00cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5ac4d45640 1 --2- 172.21.15.55:0/2342867925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 0x7f5ac015f6e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5ac4d45640 1 -- 172.21.15.55:0/2342867925 >> 172.21.15.55:0/2342867925 conn(0x7f5ac00d1c70 msgr2=0x7f5ac00d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5ac4d45640 1 -- 172.21.15.55:0/2342867925 shutdown_connections 2024-05-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.755+0000 7f5ac4d45640 1 -- 172.21.15.55:0/2342867925 wait complete. 2024-05-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.757+0000 7f5ac4d45640 1 Processor -- start 2024-05-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.758+0000 7f5ac4d45640 1 -- start start 2024-05-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ac4d45640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac00cfd00 0x7f5ac0157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ac4d45640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 0x7f5ac0154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ac4d45640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac015cd00 0x7f5ac01f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ac4d45640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac00d04a0 con 0x7f5ac00d3460 2024-05-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ac4d45640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac00d42c0 con 0x7f5ac00cfd00 2024-05-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ac4d45640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac00d0620 con 0x7f5ac015cd00 2024-05-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ab5d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 0x7f5ac0154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ab5d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 0x7f5ac0154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48766/0 (socket says 172.21.15.55:48766) 2024-05-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ab5d9b640 1 -- 172.21.15.55:0/3761445248 learned_addr learned my addr 172.21.15.55:0/3761445248 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5abe59c640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac00cfd00 0x7f5ac0157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5abed9d640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac015cd00 0x7f5ac01f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ab5d9b640 1 -- 172.21.15.55:0/3761445248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac015cd00 msgr2=0x7f5ac01f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ab5d9b640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac015cd00 0x7f5ac01f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ab5d9b640 1 -- 172.21.15.55:0/3761445248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac00cfd00 msgr2=0x7f5ac0157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ab5d9b640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac00cfd00 0x7f5ac0157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.759+0000 7f5ab5d9b640 1 -- 172.21.15.55:0/3761445248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac0155e00 con 0x7f5ac00d3460 2024-05-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.760+0000 7f5ab5d9b640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 0x7f5ac0154130 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800c7f0 tx=0x7f5aa800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.760+0000 7f5ab77fe640 1 -- 172.21.15.55:0/3761445248 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8004230 con 0x7f5ac00d3460 2024-05-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.760+0000 7f5ab77fe640 1 -- 172.21.15.55:0/3761445248 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa80043d0 con 0x7f5ac00d3460 2024-05-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.760+0000 7f5ac4d45640 1 -- 172.21.15.55:0/3761445248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac01560e0 con 0x7f5ac00d3460 2024-05-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.760+0000 7f5ab77fe640 1 -- 172.21.15.55:0/3761445248 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8007860 con 0x7f5ac00d3460 2024-05-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.760+0000 7f5ac4d45640 1 -- 172.21.15.55:0/3761445248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac01e7970 con 0x7f5ac00d3460 2024-05-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.761+0000 7f5ac4d45640 1 -- 172.21.15.55:0/3761445248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac004f720 con 0x7f5ac00d3460 2024-05-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.761+0000 7f5ab77fe640 1 -- 172.21.15.55:0/3761445248 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5aa8007a00 con 0x7f5ac00d3460 2024-05-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.762+0000 7f5ab77fe640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5aa006be60 0x7f5aa006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.762+0000 7f5ab77fe640 1 -- 172.21.15.55:0/3761445248 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f5aa808af90 con 0x7f5ac00d3460 2024-05-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.762+0000 7f5abe59c640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5aa006be60 0x7f5aa006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.763+0000 7f5abe59c640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5aa006be60 0x7f5aa006e370 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0034000 tx=0x7f5ab002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.767+0000 7f5ab77fe640 1 -- 172.21.15.55:0/3761445248 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5aa8015020 con 0x7f5ac00d3460 2024-05-20T22:11:04.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.948+0000 7f5ac4d45640 1 -- 172.21.15.55:0/3761445248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ac00d2f80 con 0x7f5ac00d3460 2024-05-20T22:11:04.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.948+0000 7f5ab77fe640 1 -- 172.21.15.55:0/3761445248 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7f5aa8059050 con 0x7f5ac00d3460 2024-05-20T22:11:04.949 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-20T22:11:04.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 -- 172.21.15.55:0/3761445248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5aa006be60 msgr2=0x7f5aa006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5aa006be60 0x7f5aa006e370 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0034000 tx=0x7f5ab002e000 comp rx=0 tx=0).stop 2024-05-20T22:11:04.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 -- 172.21.15.55:0/3761445248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 msgr2=0x7f5ac0154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:04.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 0x7f5ac0154130 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800c7f0 tx=0x7f5aa800ccb0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 -- 172.21.15.55:0/3761445248 shutdown_connections 2024-05-20T22:11:04.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5aa006be60 0x7f5aa006e370 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac015cd00 0x7f5ac01f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac00d3460 0x7f5ac0154130 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 --2- 172.21.15.55:0/3761445248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac00cfd00 0x7f5ac0157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:04.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 -- 172.21.15.55:0/3761445248 >> 172.21.15.55:0/3761445248 conn(0x7f5ac00d1c70 msgr2=0x7f5ac015dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:04.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 -- 172.21.15.55:0/3761445248 shutdown_connections 2024-05-20T22:11:04.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:04.950+0000 7f5ac4d45640 1 -- 172.21.15.55:0/3761445248 wait complete. 2024-05-20T22:11:04.963 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T22:11:05.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.080+0000 7f82b4f10640 1 Processor -- start 2024-05-20T22:11:05.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.080+0000 7f82b4f10640 1 -- start start 2024-05-20T22:11:05.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.081+0000 7f82b4f10640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f82b00cf990 0x7f82b00cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.081+0000 7f82b4f10640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82b00b38b0 con 0x7f82b00d02e0 2024-05-20T22:11:05.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.081+0000 7f82b4f10640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82b00b3b90 con 0x7f82b00d40b0 2024-05-20T22:11:05.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.081+0000 7f82b4f10640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f82b00b3e70 con 0x7f82b00cf990 2024-05-20T22:11:05.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.081+0000 7f82aed9d640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f82b00d02e0 0x7f82b006bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35980/0 (socket says 172.21.15.55:35980) 2024-05-20T22:11:05.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.081+0000 7f82aed9d640 1 -- 172.21.15.55:0/4060454547 learned_addr learned my addr 172.21.15.55:0/4060454547 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:05.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.081+0000 7f82a5d9b640 1 --2- 172.21.15.55:0/4060454547 >> v2:172.21.15.138:3300/0 conn(0x7f82b00cf990 0x7f82b00cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.082+0000 7f82add9b640 1 -- 172.21.15.55:0/4060454547 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4101961382 0 0) 0x7f82b00b38b0 con 0x7f82b00d02e0 2024-05-20T22:11:05.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.082+0000 7f82add9b640 1 -- 172.21.15.55:0/4060454547 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f829c003520 con 0x7f82b00d02e0 2024-05-20T22:11:05.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.082+0000 7f82a5d9b640 1 -- 172.21.15.55:0/4060454547 >> v1:172.21.15.138:6789/0 conn(0x7f82b00d40b0 legacy=0x7f82b015f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.082+0000 7f82add9b640 1 -- 172.21.15.55:0/4060454547 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657271030 0 0) 0x7f82b00b3b90 con 0x7f82b00d40b0 2024-05-20T22:11:05.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.082+0000 7f82a5d9b640 1 -- 172.21.15.55:0/4060454547 >> v1:172.21.15.55:6789/0 conn(0x7f82b00d02e0 legacy=0x7f82b006bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.082+0000 7f82a5d9b640 1 -- 172.21.15.55:0/4060454547 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b006b750 con 0x7f82b00cf990 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.083+0000 7f82a5d9b640 1 --2- 172.21.15.55:0/4060454547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b00cf990 0x7f82b00cfda0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f82980097a0 tx=0x7f829802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d63464cebc923c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.083+0000 7f82add9b640 1 -- 172.21.15.55:0/4060454547 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298037070 con 0x7f82b00cf990 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.083+0000 7f82add9b640 1 -- 172.21.15.55:0/4060454547 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829802ab10 con 0x7f82b00cf990 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.083+0000 7f82add9b640 1 -- 172.21.15.55:0/4060454547 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a80019f0 con 0x7f82b00cf990 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.083+0000 7f82b4f10640 1 -- 172.21.15.55:0/4060454547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b00cf990 msgr2=0x7f82b00cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.083+0000 7f82b4f10640 1 --2- 172.21.15.55:0/4060454547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b00cf990 0x7f82b00cfda0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f82980097a0 tx=0x7f829802a080 comp rx=0 tx=0).stop 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.084+0000 7f82b4f10640 1 -- 172.21.15.55:0/4060454547 shutdown_connections 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.084+0000 7f82b4f10640 1 --2- 172.21.15.55:0/4060454547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b00cf990 0x7f82b00cfda0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.084+0000 7f82b4f10640 1 -- 172.21.15.55:0/4060454547 >> 172.21.15.55:0/4060454547 conn(0x7f82b006da20 msgr2=0x7f82b006de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.084+0000 7f82b4f10640 1 -- 172.21.15.55:0/4060454547 shutdown_connections 2024-05-20T22:11:05.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.084+0000 7f82b4f10640 1 -- 172.21.15.55:0/4060454547 wait complete. 2024-05-20T22:11:05.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.086+0000 7f82b4f10640 1 Processor -- start 2024-05-20T22:11:05.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.086+0000 7f82b4f10640 1 -- start start 2024-05-20T22:11:05.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82b4f10640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b00cf990 0x7f82b0152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82b4f10640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82b00d02e0 0x7f82b0152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82b4f10640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b00d40b0 0x7f82b01f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82b4f10640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f82b00d4640 con 0x7f82b00cf990 2024-05-20T22:11:05.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82b4f10640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82b00d44c0 con 0x7f82b00d40b0 2024-05-20T22:11:05.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82b4f10640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f82b006c380 con 0x7f82b00d02e0 2024-05-20T22:11:05.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82ae59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b00cf990 0x7f82b0152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82ae59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b00cf990 0x7f82b0152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48782/0 (socket says 172.21.15.55:48782) 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82ae59c640 1 -- 172.21.15.55:0/2527499826 learned_addr learned my addr 172.21.15.55:0/2527499826 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82a5d9b640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82b00d02e0 0x7f82b0152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82aed9d640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b00d40b0 0x7f82b01f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82ae59c640 1 -- 172.21.15.55:0/2527499826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82b00d02e0 msgr2=0x7f82b0152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82ae59c640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82b00d02e0 0x7f82b0152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.087+0000 7f82ae59c640 1 -- 172.21.15.55:0/2527499826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b00d40b0 msgr2=0x7f82b01f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.088+0000 7f82ae59c640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b00d40b0 0x7f82b01f5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.088+0000 7f82ae59c640 1 -- 172.21.15.55:0/2527499826 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b00cd890 con 0x7f82b00cf990 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.088+0000 7f82ae59c640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b00cf990 0x7f82b0152100 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f82a000c410 tx=0x7f82a000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.088+0000 7f82a77fe640 1 -- 172.21.15.55:0/2527499826 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0017070 con 0x7f82b00cf990 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.088+0000 7f82a77fe640 1 -- 172.21.15.55:0/2527499826 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a00035f0 con 0x7f82b00cf990 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.088+0000 7f82b4f10640 1 -- 172.21.15.55:0/2527499826 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b00cdb70 con 0x7f82b00cf990 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.088+0000 7f82a77fe640 1 -- 172.21.15.55:0/2527499826 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a00039e0 con 0x7f82b00cf990 2024-05-20T22:11:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.088+0000 7f82b4f10640 1 -- 172.21.15.55:0/2527499826 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b00ce130 con 0x7f82b00cf990 2024-05-20T22:11:05.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.089+0000 7f82b4f10640 1 -- 172.21.15.55:0/2527499826 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b004f720 con 0x7f82b00cf990 2024-05-20T22:11:05.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.089+0000 7f82a77fe640 1 -- 172.21.15.55:0/2527499826 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82a001a070 con 0x7f82b00cf990 2024-05-20T22:11:05.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.090+0000 7f82a77fe640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f829006be30 0x7f829006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.090+0000 7f82a77fe640 1 -- 172.21.15.55:0/2527499826 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f82a008a410 con 0x7f82b00cf990 2024-05-20T22:11:05.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.090+0000 7f82a5d9b640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f829006be30 0x7f829006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.090+0000 7f82a5d9b640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f829006be30 0x7f829006e340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8298034000 tx=0x7f829802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:05.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.095+0000 7f82a77fe640 1 -- 172.21.15.55:0/2527499826 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f82a0058c10 con 0x7f82b00cf990 2024-05-20T22:11:05.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.279+0000 7f82b4f10640 1 -- 172.21.15.55:0/2527499826 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82b00b5460 con 0x7f82b00cf990 2024-05-20T22:11:05.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.279+0000 7f82a77fe640 1 -- 172.21.15.55:0/2527499826 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) ==== 104+0+11 (secure 0 0 0) 0x7f82a0058510 con 0x7f82b00cf990 2024-05-20T22:11:05.280 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-20T22:11:05.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 -- 172.21.15.55:0/2527499826 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f829006be30 msgr2=0x7f829006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f829006be30 0x7f829006e340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8298034000 tx=0x7f829802e000 comp rx=0 tx=0).stop 2024-05-20T22:11:05.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 -- 172.21.15.55:0/2527499826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b00cf990 msgr2=0x7f82b0152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b00cf990 0x7f82b0152100 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f82a000c410 tx=0x7f82a000c8d0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 -- 172.21.15.55:0/2527499826 shutdown_connections 2024-05-20T22:11:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f829006be30 0x7f829006e340 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b00d40b0 0x7f82b01f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82b00d02e0 0x7f82b0152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 --2- 172.21.15.55:0/2527499826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b00cf990 0x7f82b0152100 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 -- 172.21.15.55:0/2527499826 >> 172.21.15.55:0/2527499826 conn(0x7f82b006da20 msgr2=0x7f82b015df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 -- 172.21.15.55:0/2527499826 shutdown_connections 2024-05-20T22:11:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.281+0000 7f82b4f10640 1 -- 172.21.15.55:0/2527499826 wait complete. 2024-05-20T22:11:05.293 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T22:11:05.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.410+0000 7f27dbade640 1 Processor -- start 2024-05-20T22:11:05.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.410+0000 7f27dbade640 1 -- start start 2024-05-20T22:11:05.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.411+0000 7f27dbade640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d40d40b0 0x7f27d415f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.411+0000 7f27dbade640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f27d41638d0 0x7f27d406bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.411+0000 7f27dbade640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27d40b38b0 con 0x7f27d4162db0 2024-05-20T22:11:05.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.411+0000 7f27dbade640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f27d40b3b90 con 0x7f27d41638d0 2024-05-20T22:11:05.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.411+0000 7f27dbade640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f27d40b3e70 con 0x7f27d40d40b0 2024-05-20T22:11:05.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.412+0000 7f27da07b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f27d41638d0 0x7f27d406bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.412+0000 7f27da07b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f27d41638d0 0x7f27d406bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48786/0 (socket says 172.21.15.55:48786) 2024-05-20T22:11:05.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.412+0000 7f27da07b640 1 -- 172.21.15.55:0/485004329 learned_addr learned my addr 172.21.15.55:0/485004329 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:05.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.412+0000 7f27d987a640 1 --2- 172.21.15.55:0/485004329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d40d40b0 0x7f27d415f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.412+0000 7f27da07b640 1 -- 172.21.15.55:0/485004329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d40d40b0 msgr2=0x7f27d415f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:05.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.412+0000 7f27da07b640 1 --2- 172.21.15.55:0/485004329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d40d40b0 0x7f27d415f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.412+0000 7f27da07b640 1 -- 172.21.15.55:0/485004329 >> v1:172.21.15.138:6789/0 conn(0x7f27d4162db0 legacy=0x7f27d41631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.412+0000 7f27da07b640 1 -- 172.21.15.55:0/485004329 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d406b750 con 0x7f27d41638d0 2024-05-20T22:11:05.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.412+0000 7f27d987a640 1 --2- 172.21.15.55:0/485004329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d40d40b0 0x7f27d415f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:05.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.413+0000 7f27da07b640 1 --2- 172.21.15.55:0/485004329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d41638d0 0x7f27d406bc70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009e50 tx=0x7f27cc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a89a784dc843aca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:05.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.413+0000 7f27d8878640 1 -- 172.21.15.55:0/485004329 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc036070 con 0x7f27d41638d0 2024-05-20T22:11:05.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.413+0000 7f27d8878640 1 -- 172.21.15.55:0/485004329 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27cc031600 con 0x7f27d41638d0 2024-05-20T22:11:05.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.413+0000 7f27d8878640 1 -- 172.21.15.55:0/485004329 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0319d0 con 0x7f27d41638d0 2024-05-20T22:11:05.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.413+0000 7f27dbade640 1 -- 172.21.15.55:0/485004329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d41638d0 msgr2=0x7f27d406bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.413+0000 7f27dbade640 1 --2- 172.21.15.55:0/485004329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d41638d0 0x7f27d406bc70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009e50 tx=0x7f27cc02a5f0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.413+0000 7f27dbade640 1 -- 172.21.15.55:0/485004329 shutdown_connections 2024-05-20T22:11:05.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.414+0000 7f27dbade640 1 --2- 172.21.15.55:0/485004329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d41638d0 0x7f27d406bc70 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.414+0000 7f27dbade640 1 --2- 172.21.15.55:0/485004329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d40d40b0 0x7f27d415f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.414+0000 7f27dbade640 1 -- 172.21.15.55:0/485004329 >> 172.21.15.55:0/485004329 conn(0x7f27d406da20 msgr2=0x7f27d406de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:05.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.414+0000 7f27dbade640 1 -- 172.21.15.55:0/485004329 shutdown_connections 2024-05-20T22:11:05.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.414+0000 7f27dbade640 1 -- 172.21.15.55:0/485004329 wait complete. 2024-05-20T22:11:05.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.415+0000 7f27dbade640 1 Processor -- start 2024-05-20T22:11:05.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.415+0000 7f27dbade640 1 -- start start 2024-05-20T22:11:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27dbade640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27d40d40b0 0x7f27d414d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27dbade640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d4162db0 0x7f27d414d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27dbade640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d41638d0 0x7f27d41607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27dbade640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f27d40d4640 con 0x7f27d4162db0 2024-05-20T22:11:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27dbade640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27d40d44c0 con 0x7f27d40d40b0 2024-05-20T22:11:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27dbade640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f27d406c1b0 con 0x7f27d41638d0 2024-05-20T22:11:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d9079640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d4162db0 0x7f27d414d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d9079640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d4162db0 0x7f27d414d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48794/0 (socket says 172.21.15.55:48794) 2024-05-20T22:11:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d9079640 1 -- 172.21.15.55:0/3225529651 learned_addr learned my addr 172.21.15.55:0/3225529651 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d987a640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27d40d40b0 0x7f27d414d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27da07b640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d41638d0 0x7f27d41607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d9079640 1 -- 172.21.15.55:0/3225529651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d41638d0 msgr2=0x7f27d41607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d9079640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d41638d0 0x7f27d41607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d9079640 1 -- 172.21.15.55:0/3225529651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27d40d40b0 msgr2=0x7f27d414d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d9079640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27d40d40b0 0x7f27d414d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d9079640 1 -- 172.21.15.55:0/3225529651 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d414b2c0 con 0x7f27d4162db0 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27da07b640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d41638d0 0x7f27d41607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d987a640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27d40d40b0 0x7f27d414d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.417+0000 7f27d9079640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d4162db0 0x7f27d414d6c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f27c400c3f0 tx=0x7f27c400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.418+0000 7f27ca7fc640 1 -- 172.21.15.55:0/3225529651 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c40052d0 con 0x7f27d4162db0 2024-05-20T22:11:05.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.418+0000 7f27ca7fc640 1 -- 172.21.15.55:0/3225529651 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c4005490 con 0x7f27d4162db0 2024-05-20T22:11:05.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.418+0000 7f27dbade640 1 -- 172.21.15.55:0/3225529651 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d414de80 con 0x7f27d4162db0 2024-05-20T22:11:05.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.418+0000 7f27ca7fc640 1 -- 172.21.15.55:0/3225529651 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4007260 con 0x7f27d4162db0 2024-05-20T22:11:05.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.418+0000 7f27dbade640 1 -- 172.21.15.55:0/3225529651 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d414cde0 con 0x7f27d4162db0 2024-05-20T22:11:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.419+0000 7f27dbade640 1 -- 172.21.15.55:0/3225529651 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d404f720 con 0x7f27d4162db0 2024-05-20T22:11:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.419+0000 7f27ca7fc640 1 -- 172.21.15.55:0/3225529651 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f27c4007400 con 0x7f27d4162db0 2024-05-20T22:11:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.420+0000 7f27ca7fc640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f27b806be30 0x7f27b806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.420+0000 7f27ca7fc640 1 -- 172.21.15.55:0/3225529651 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f27c408a480 con 0x7f27d4162db0 2024-05-20T22:11:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.420+0000 7f27d987a640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f27b806be30 0x7f27b806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.420+0000 7f27d987a640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f27b806be30 0x7f27b806e340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006fd0 tx=0x7f27d0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:05.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.424+0000 7f27ca7fc640 1 -- 172.21.15.55:0/3225529651 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f27c4058ca0 con 0x7f27d4162db0 2024-05-20T22:11:05.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.608+0000 7f27dbade640 1 -- 172.21.15.55:0/3225529651 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f27d414dc00 con 0x7f27d4162db0 2024-05-20T22:11:05.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.608+0000 7f27ca7fc640 1 -- 172.21.15.55:0/3225529651 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) ==== 100+0+11 (secure 0 0 0) 0x7f27d414dc00 con 0x7f27d4162db0 2024-05-20T22:11:05.609 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-20T22:11:05.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 -- 172.21.15.55:0/3225529651 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f27b806be30 msgr2=0x7f27b806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f27b806be30 0x7f27b806e340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006fd0 tx=0x7f27d0008040 comp rx=0 tx=0).stop 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 -- 172.21.15.55:0/3225529651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d4162db0 msgr2=0x7f27d414d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d4162db0 0x7f27d414d6c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f27c400c3f0 tx=0x7f27c400c7f0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 -- 172.21.15.55:0/3225529651 shutdown_connections 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f27b806be30 0x7f27b806e340 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27d41638d0 0x7f27d41607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27d4162db0 0x7f27d414d6c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 --2- 172.21.15.55:0/3225529651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27d40d40b0 0x7f27d414d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 -- 172.21.15.55:0/3225529651 >> 172.21.15.55:0/3225529651 conn(0x7f27d406da20 msgr2=0x7f27d40b4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 -- 172.21.15.55:0/3225529651 shutdown_connections 2024-05-20T22:11:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.610+0000 7f27dbade640 1 -- 172.21.15.55:0/3225529651 wait complete. 2024-05-20T22:11:05.623 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:11:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.736+0000 7f6f23a84640 1 Processor -- start 2024-05-20T22:11:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.737+0000 7f6f23a84640 1 -- start start 2024-05-20T22:11:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.738+0000 7f6f23a84640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f1c072580 0x7f6f1c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.738+0000 7f6f23a84640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f1c163d00 con 0x7f6f1c0b7490 2024-05-20T22:11:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.738+0000 7f6f23a84640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f1c163180 con 0x7f6f1c0b57b0 2024-05-20T22:11:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.738+0000 7f6f23a84640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1c0b7060 con 0x7f6f1c072580 2024-05-20T22:11:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.738+0000 7f6f22021640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6f1c0b7490 0x7f6f1c0b78a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35992/0 (socket says 172.21.15.55:35992) 2024-05-20T22:11:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.738+0000 7f6f22021640 1 -- 172.21.15.55:0/1030895040 learned_addr learned my addr 172.21.15.55:0/1030895040 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.738+0000 7f6f2101f640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f1c072580 0x7f6f1c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.738+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1789876724 0 0) 0x7f6f1c163d00 con 0x7f6f1c0b7490 2024-05-20T22:11:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.738+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f08003520 con 0x7f6f1c0b7490 2024-05-20T22:11:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785208516 0 0) 0x7f6f1c163180 con 0x7f6f1c0b57b0 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f1c163d00 con 0x7f6f1c0b57b0 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2491052695 0 0) 0x7f6f08003520 con 0x7f6f1c0b7490 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f1c072580 msgr2=0x7f6f1c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f1c072580 0x7f6f1c15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 >> v1:172.21.15.138:6789/0 conn(0x7f6f1c0b57b0 legacy=0x7f6f1c071e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c163460 con 0x7f6f1c0b7490 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f6f14001670 con 0x7f6f1c0b7490 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 >> v1:172.21.15.55:6789/0 conn(0x7f6f1c0b7490 legacy=0x7f6f1c0b78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f08003b70 0x7f6f08003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f08004800 0x7f6f08004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f08005260 0x7f6f0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0800bfb0 con 0x7f6f08003b70 2024-05-20T22:11:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0800c160 con 0x7f6f08004800 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0800c360 con 0x7f6f08005260 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f21820640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f08003b70 0x7f6f08003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f2101f640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f08004800 0x7f6f08004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f22021640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f08005260 0x7f6f0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f21820640 1 -- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f08005260 msgr2=0x7f6f0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f21820640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f08005260 0x7f6f0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f21820640 1 -- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f08004800 msgr2=0x7f6f08004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f21820640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f08004800 0x7f6f08004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.739+0000 7f6f21820640 1 -- 172.21.15.55:0/1030895040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f08003520 con 0x7f6f08003b70 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.740+0000 7f6f21820640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f08003b70 0x7f6f08003fc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6f18001fd0 tx=0x7f6f08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9afc8c740c487a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.740+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18001bf0 con 0x7f6f08003b70 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.740+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f18003b70 con 0x7f6f08003b70 2024-05-20T22:11:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.740+0000 7f6f2081e640 1 -- 172.21.15.55:0/1030895040 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f180050d0 con 0x7f6f08003b70 2024-05-20T22:11:05.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.741+0000 7f6f23a84640 1 -- 172.21.15.55:0/1030895040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f08003b70 msgr2=0x7f6f08003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.741+0000 7f6f23a84640 1 --2- 172.21.15.55:0/1030895040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f08003b70 0x7f6f08003fc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6f18001fd0 tx=0x7f6f08005830 comp rx=0 tx=0).stop 2024-05-20T22:11:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.741+0000 7f6f22021640 1 -- 172.21.15.55:0/1030895040 reap_dead start 2024-05-20T22:11:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.741+0000 7f6f23a84640 1 -- 172.21.15.55:0/1030895040 shutdown_connections 2024-05-20T22:11:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.741+0000 7f6f23a84640 1 -- 172.21.15.55:0/1030895040 >> 172.21.15.55:0/1030895040 conn(0x7f6f1c0d2960 msgr2=0x7f6f1c0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.741+0000 7f6f23a84640 1 -- 172.21.15.55:0/1030895040 shutdown_connections 2024-05-20T22:11:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.741+0000 7f6f23a84640 1 -- 172.21.15.55:0/1030895040 wait complete. 2024-05-20T22:11:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.743+0000 7f6f23a84640 1 Processor -- start 2024-05-20T22:11:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.743+0000 7f6f23a84640 1 -- start start 2024-05-20T22:11:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.744+0000 7f6f23a84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f08003b70 0x7f6f1c160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.744+0000 7f6f23a84640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f1c157510 0x7f6f1c14dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.744+0000 7f6f23a84640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c0b0d40 0x7f6f1c0b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.744+0000 7f6f23a84640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c162dc0 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.744+0000 7f6f23a84640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c163a20 con 0x7f6f08003b70 2024-05-20T22:11:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.744+0000 7f6f23a84640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1c162f40 con 0x7f6f1c157510 2024-05-20T22:11:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.744+0000 7f6f22021640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c0b0d40 0x7f6f1c0b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.744+0000 7f6f22021640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c0b0d40 0x7f6f1c0b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48810/0 (socket says 172.21.15.55:48810) 2024-05-20T22:11:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.744+0000 7f6f22021640 1 -- 172.21.15.55:0/2775426015 learned_addr learned my addr 172.21.15.55:0/2775426015 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f2101f640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f1c157510 0x7f6f1c14dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f21820640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f08003b70 0x7f6f1c160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f22021640 1 -- 172.21.15.55:0/2775426015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f1c157510 msgr2=0x7f6f1c14dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f22021640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f1c157510 0x7f6f1c14dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f22021640 1 -- 172.21.15.55:0/2775426015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f08003b70 msgr2=0x7f6f1c160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f22021640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f08003b70 0x7f6f1c160e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f22021640 1 -- 172.21.15.55:0/2775426015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c0b1690 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f22021640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c0b0d40 0x7f6f1c0b1150 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0b6af0 tx=0x7f6f14007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f127fc640 1 -- 172.21.15.55:0/2775426015 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1401a070 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f127fc640 1 -- 172.21.15.55:0/2775426015 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f1400f700 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f23a84640 1 -- 172.21.15.55:0/2775426015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c1f5ff0 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f127fc640 1 -- 172.21.15.55:0/2775426015 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400faf0 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.745+0000 7f6f23a84640 1 -- 172.21.15.55:0/2775426015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c0adae0 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.746+0000 7f6f23a84640 1 -- 172.21.15.55:0/2775426015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1c04f720 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.747+0000 7f6f127fc640 1 -- 172.21.15.55:0/2775426015 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f1401d070 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.747+0000 7f6f127fc640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6f0006be60 0x7f6f0006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:05.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.747+0000 7f6f127fc640 1 -- 172.21.15.55:0/2775426015 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f6f1408a430 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.747+0000 7f6f21820640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6f0006be60 0x7f6f0006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:05.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.748+0000 7f6f21820640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6f0006be60 0x7f6f0006e370 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6f18032000 tx=0x7f6f1802f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:05.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.752+0000 7f6f127fc640 1 -- 172.21.15.55:0/2775426015 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6f1c0b1690 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.979+0000 7f6f23a84640 1 -- 172.21.15.55:0/2775426015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f1c0d1f70 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.979+0000 7f6f127fc640 1 -- 172.21.15.55:0/2775426015 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7f6f14058bc0 con 0x7f6f1c0b0d40 2024-05-20T22:11:05.981 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:05.981 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:11:05.981 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:11:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 -- 172.21.15.55:0/2775426015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6f0006be60 msgr2=0x7f6f0006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6f0006be60 0x7f6f0006e370 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6f18032000 tx=0x7f6f1802f000 comp rx=0 tx=0).stop 2024-05-20T22:11:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 -- 172.21.15.55:0/2775426015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c0b0d40 msgr2=0x7f6f1c0b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c0b0d40 0x7f6f1c0b1150 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0b6af0 tx=0x7f6f14007430 comp rx=0 tx=0).stop 2024-05-20T22:11:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 -- 172.21.15.55:0/2775426015 shutdown_connections 2024-05-20T22:11:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6f0006be60 0x7f6f0006e370 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c0b0d40 0x7f6f1c0b1150 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f1c157510 0x7f6f1c14dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 --2- 172.21.15.55:0/2775426015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f08003b70 0x7f6f1c160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 -- 172.21.15.55:0/2775426015 >> 172.21.15.55:0/2775426015 conn(0x7f6f1c0d2960 msgr2=0x7f6f1c162500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 -- 172.21.15.55:0/2775426015 shutdown_connections 2024-05-20T22:11:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:05.982+0000 7f6f23a84640 1 -- 172.21.15.55:0/2775426015 wait complete. 2024-05-20T22:11:05.995 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:11:06.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.108+0000 7f9c810b8640 1 Processor -- start 2024-05-20T22:11:06.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.108+0000 7f9c810b8640 1 -- start start 2024-05-20T22:11:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.109+0000 7f9c810b8640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9c7c0d0210 0x7f9c7c06fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.109+0000 7f9c810b8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c06c550 0x7f9c7c06c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.109+0000 7f9c810b8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c7c06cea0 con 0x7f9c7c15d550 2024-05-20T22:11:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.109+0000 7f9c810b8640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f9c7c161a10 con 0x7f9c7c0d0210 2024-05-20T22:11:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.109+0000 7f9c810b8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c160b40 con 0x7f9c7c06c550 2024-05-20T22:11:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.110+0000 7f9c7ad9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9c7c0d0210 0x7f9c7c06fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.110+0000 7f9c7ad9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9c7c0d0210 0x7f9c7c06fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48820/0 (socket says 172.21.15.55:48820) 2024-05-20T22:11:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.110+0000 7f9c7ad9d640 1 -- 172.21.15.55:0/2884301696 learned_addr learned my addr 172.21.15.55:0/2884301696 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.110+0000 7f9c7b59e640 1 --2- 172.21.15.55:0/2884301696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c06c550 0x7f9c7c06c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.110+0000 7f9c7ad9d640 1 -- 172.21.15.55:0/2884301696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c06c550 msgr2=0x7f9c7c06c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.110+0000 7f9c7ad9d640 1 --2- 172.21.15.55:0/2884301696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c06c550 0x7f9c7c06c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.110+0000 7f9c7ad9d640 1 -- 172.21.15.55:0/2884301696 >> v1:172.21.15.138:6789/0 conn(0x7f9c7c15d550 legacy=0x7f9c7c161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.110+0000 7f9c7ad9d640 1 -- 172.21.15.55:0/2884301696 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c15dd80 con 0x7f9c7c0d0210 2024-05-20T22:11:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.110+0000 7f9c7b59e640 1 --2- 172.21.15.55:0/2884301696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c06c550 0x7f9c7c06c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c7ad9d640 1 --2- 172.21.15.55:0/2884301696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 0x7f9c7c06fdf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009870 tx=0x7f9c6c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf84d51275717433 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c7a59c640 1 -- 172.21.15.55:0/2884301696 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c037070 con 0x7f9c7c0d0210 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c7a59c640 1 -- 172.21.15.55:0/2884301696 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6c032440 con 0x7f9c7c0d0210 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c7a59c640 1 -- 172.21.15.55:0/2884301696 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c032810 con 0x7f9c7c0d0210 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c810b8640 1 -- 172.21.15.55:0/2884301696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 msgr2=0x7f9c7c06fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c810b8640 1 --2- 172.21.15.55:0/2884301696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 0x7f9c7c06fdf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009870 tx=0x7f9c6c02a250 comp rx=0 tx=0).stop 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c810b8640 1 -- 172.21.15.55:0/2884301696 shutdown_connections 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c810b8640 1 --2- 172.21.15.55:0/2884301696 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c06c550 0x7f9c7c06c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c810b8640 1 --2- 172.21.15.55:0/2884301696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 0x7f9c7c06fdf0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c810b8640 1 -- 172.21.15.55:0/2884301696 >> 172.21.15.55:0/2884301696 conn(0x7f9c7c072310 msgr2=0x7f9c7c072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.111+0000 7f9c810b8640 1 -- 172.21.15.55:0/2884301696 shutdown_connections 2024-05-20T22:11:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.112+0000 7f9c810b8640 1 -- 172.21.15.55:0/2884301696 wait complete. 2024-05-20T22:11:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.113+0000 7f9c810b8640 1 Processor -- start 2024-05-20T22:11:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.114+0000 7f9c810b8640 1 -- start start 2024-05-20T22:11:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c810b8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c7c06c550 0x7f9c7c157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c810b8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 0x7f9c7c143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c810b8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15d550 0x7f9c7c1466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c810b8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c160de0 con 0x7f9c7c0d0210 2024-05-20T22:11:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c810b8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c15e340 con 0x7f9c7c06c550 2024-05-20T22:11:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c810b8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c15d960 con 0x7f9c7c15d550 2024-05-20T22:11:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 0x7f9c7c143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 0x7f9c7c143c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48836/0 (socket says 172.21.15.55:48836) 2024-05-20T22:11:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7259c640 1 -- 172.21.15.55:0/2248296872 learned_addr learned my addr 172.21.15.55:0/2248296872 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:06.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7ad9d640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c7c06c550 0x7f9c7c157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7b59e640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15d550 0x7f9c7c1466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7259c640 1 -- 172.21.15.55:0/2248296872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15d550 msgr2=0x7f9c7c1466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7259c640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15d550 0x7f9c7c1466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7259c640 1 -- 172.21.15.55:0/2248296872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c7c06c550 msgr2=0x7f9c7c157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7259c640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c7c06c550 0x7f9c7c157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7259c640 1 -- 172.21.15.55:0/2248296872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c15a230 con 0x7f9c7c0d0210 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.115+0000 7f9c7259c640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 0x7f9c7c143c50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c340 tx=0x7f9c6400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.116+0000 7f9c73fff640 1 -- 172.21.15.55:0/2248296872 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64017070 con 0x7f9c7c0d0210 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.116+0000 7f9c73fff640 1 -- 172.21.15.55:0/2248296872 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c640052d0 con 0x7f9c7c0d0210 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.116+0000 7f9c73fff640 1 -- 172.21.15.55:0/2248296872 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64003850 con 0x7f9c7c0d0210 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.116+0000 7f9c810b8640 1 -- 172.21.15.55:0/2248296872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7c15a510 con 0x7f9c7c0d0210 2024-05-20T22:11:06.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.116+0000 7f9c810b8640 1 -- 172.21.15.55:0/2248296872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7c15b150 con 0x7f9c7c0d0210 2024-05-20T22:11:06.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.117+0000 7f9c810b8640 1 -- 172.21.15.55:0/2248296872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7c04f720 con 0x7f9c7c0d0210 2024-05-20T22:11:06.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.117+0000 7f9c73fff640 1 -- 172.21.15.55:0/2248296872 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c6401a070 con 0x7f9c7c0d0210 2024-05-20T22:11:06.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.118+0000 7f9c73fff640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c5c06be30 0x7f9c5c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.118+0000 7f9c73fff640 1 -- 172.21.15.55:0/2248296872 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f9c6408a4a0 con 0x7f9c7c0d0210 2024-05-20T22:11:06.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.118+0000 7f9c7ad9d640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c5c06be30 0x7f9c5c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.118+0000 7f9c7ad9d640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c5c06be30 0x7f9c5c06e340 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c034000 tx=0x7f9c6c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:06.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.123+0000 7f9c73fff640 1 -- 172.21.15.55:0/2248296872 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9c7c06cea0 con 0x7f9c7c0d0210 2024-05-20T22:11:06.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.305+0000 7f9c810b8640 1 -- 172.21.15.55:0/2248296872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c7c073360 con 0x7f9c7c0d0210 2024-05-20T22:11:06.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.306+0000 7f9c73fff640 1 -- 172.21.15.55:0/2248296872 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+17167 (secure 0 0 0) 0x7f9c64058b20 con 0x7f9c7c0d0210 2024-05-20T22:11:06.307 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:06.307 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":61,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:10:38.297580+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:10:22.369451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T22:10:22.970959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.25","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:6837/2445634740":"2024-05-21T22:10:38.297533+0000","172.21.15.55:6835/2445634740":"2024-05-21T22:10:38.297533+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:11:06.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.308+0000 7f9c810b8640 1 -- 172.21.15.55:0/2248296872 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c5c06be30 msgr2=0x7f9c5c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.308+0000 7f9c810b8640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c5c06be30 0x7f9c5c06e340 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c034000 tx=0x7f9c6c02e000 comp rx=0 tx=0).stop 2024-05-20T22:11:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.308+0000 7f9c810b8640 1 -- 172.21.15.55:0/2248296872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 msgr2=0x7f9c7c143c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.308+0000 7f9c810b8640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 0x7f9c7c143c50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c340 tx=0x7f9c6400c800 comp rx=0 tx=0).stop 2024-05-20T22:11:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.308+0000 7f9c810b8640 1 -- 172.21.15.55:0/2248296872 shutdown_connections 2024-05-20T22:11:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.308+0000 7f9c810b8640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c5c06be30 0x7f9c5c06e340 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.308+0000 7f9c810b8640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15d550 0x7f9c7c1466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.308+0000 7f9c810b8640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c7c0d0210 0x7f9c7c143c50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.308+0000 7f9c810b8640 1 --2- 172.21.15.55:0/2248296872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c7c06c550 0x7f9c7c157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.308+0000 7f9c810b8640 1 -- 172.21.15.55:0/2248296872 >> 172.21.15.55:0/2248296872 conn(0x7f9c7c072310 msgr2=0x7f9c7c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.309+0000 7f9c810b8640 1 -- 172.21.15.55:0/2248296872 shutdown_connections 2024-05-20T22:11:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.309+0000 7f9c810b8640 1 -- 172.21.15.55:0/2248296872 wait complete. 2024-05-20T22:11:06.322 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T22:11:06.322 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:11:06.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.436+0000 7f4c5921b640 1 Processor -- start 2024-05-20T22:11:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.437+0000 7f4c5921b640 1 -- start start 2024-05-20T22:11:06.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.438+0000 7f4c5921b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4c540b7ed0 0x7f4c540d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.438+0000 7f4c5921b640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c540cd890 con 0x7f4c540cfa00 2024-05-20T22:11:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.438+0000 7f4c5921b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c5408be00 con 0x7f4c540b1df0 2024-05-20T22:11:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.438+0000 7f4c5921b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f4c540ce150 con 0x7f4c540b7ed0 2024-05-20T22:11:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.438+0000 7f4c5359e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4c540b7ed0 0x7f4c540d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.438+0000 7f4c4bfff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4c540cfa00 0x7f4c540b2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36004/0 (socket says 172.21.15.55:36004) 2024-05-20T22:11:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.438+0000 7f4c5359e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4c540b7ed0 0x7f4c540d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48840/0 (socket says 172.21.15.55:48840) 2024-05-20T22:11:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.438+0000 7f4c4bfff640 1 -- 172.21.15.55:0/3964358625 learned_addr learned my addr 172.21.15.55:0/3964358625 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.438+0000 7f4c5359e640 1 -- 172.21.15.55:0/3964358625 >> v1:172.21.15.138:6789/0 conn(0x7f4c540b1df0 legacy=0x7f4c540cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.438+0000 7f4c5359e640 1 -- 172.21.15.55:0/3964358625 >> v1:172.21.15.55:6789/0 conn(0x7f4c540cfa00 legacy=0x7f4c540b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.439+0000 7f4c5359e640 1 -- 172.21.15.55:0/3964358625 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c540cdb00 con 0x7f4c540b7ed0 2024-05-20T22:11:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.439+0000 7f4c5359e640 1 --2- 172.21.15.55:0/3964358625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 0x7f4c540d1320 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d90 tx=0x7f4c4c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd1daa9e19196c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.439+0000 7f4c5259c640 1 -- 172.21.15.55:0/3964358625 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c007740 con 0x7f4c540b7ed0 2024-05-20T22:11:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.439+0000 7f4c5259c640 1 -- 172.21.15.55:0/3964358625 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4c0078e0 con 0x7f4c540b7ed0 2024-05-20T22:11:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.439+0000 7f4c5259c640 1 -- 172.21.15.55:0/3964358625 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c007cb0 con 0x7f4c540b7ed0 2024-05-20T22:11:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.440+0000 7f4c5921b640 1 -- 172.21.15.55:0/3964358625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 msgr2=0x7f4c540d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.440+0000 7f4c5921b640 1 --2- 172.21.15.55:0/3964358625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 0x7f4c540d1320 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d90 tx=0x7f4c4c02a6f0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.440+0000 7f4c5921b640 1 -- 172.21.15.55:0/3964358625 shutdown_connections 2024-05-20T22:11:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.440+0000 7f4c5921b640 1 --2- 172.21.15.55:0/3964358625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 0x7f4c540d1320 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.440+0000 7f4c5921b640 1 -- 172.21.15.55:0/3964358625 >> 172.21.15.55:0/3964358625 conn(0x7f4c540d29a0 msgr2=0x7f4c540d2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.440+0000 7f4c5921b640 1 -- 172.21.15.55:0/3964358625 shutdown_connections 2024-05-20T22:11:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.440+0000 7f4c5921b640 1 -- 172.21.15.55:0/3964358625 wait complete. 2024-05-20T22:11:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.442+0000 7f4c5921b640 1 Processor -- start 2024-05-20T22:11:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.442+0000 7f4c5921b640 1 -- start start 2024-05-20T22:11:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.443+0000 7f4c5921b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c540b1df0 0x7f4c541f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.443+0000 7f4c5921b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 0x7f4c5415f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.443+0000 7f4c5921b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c54159870 0x7f4c541e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.443+0000 7f4c5921b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4c540d1b80 con 0x7f4c540b7ed0 2024-05-20T22:11:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.443+0000 7f4c5921b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c540d1a00 con 0x7f4c540b1df0 2024-05-20T22:11:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.443+0000 7f4c5921b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c540b2a10 con 0x7f4c54159870 2024-05-20T22:11:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 0x7f4c5415f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 0x7f4c5415f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48846/0 (socket says 172.21.15.55:48846) 2024-05-20T22:11:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4bfff640 1 -- 172.21.15.55:0/2399517956 learned_addr learned my addr 172.21.15.55:0/2399517956 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c52d9d640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c540b1df0 0x7f4c541f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c5359e640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c54159870 0x7f4c541e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4bfff640 1 -- 172.21.15.55:0/2399517956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c54159870 msgr2=0x7f4c541e5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4bfff640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c54159870 0x7f4c541e5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4bfff640 1 -- 172.21.15.55:0/2399517956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c540b1df0 msgr2=0x7f4c541f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4bfff640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c540b1df0 0x7f4c541f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4bfff640 1 -- 172.21.15.55:0/2399517956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c541f7230 con 0x7f4c540b7ed0 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4bfff640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 0x7f4c5415f2b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00c480 tx=0x7f4c3c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4b7fe640 1 -- 172.21.15.55:0/2399517956 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c017070 con 0x7f4c540b7ed0 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4b7fe640 1 -- 172.21.15.55:0/2399517956 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c3c005ce0 con 0x7f4c540b7ed0 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c5921b640 1 -- 172.21.15.55:0/2399517956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c541f4190 con 0x7f4c540b7ed0 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c4b7fe640 1 -- 172.21.15.55:0/2399517956 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c0072d0 con 0x7f4c540b7ed0 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.444+0000 7f4c5921b640 1 -- 172.21.15.55:0/2399517956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c541f4750 con 0x7f4c540b7ed0 2024-05-20T22:11:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.445+0000 7f4c5921b640 1 -- 172.21.15.55:0/2399517956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5404f720 con 0x7f4c540b7ed0 2024-05-20T22:11:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.446+0000 7f4c4b7fe640 1 -- 172.21.15.55:0/2399517956 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c3c01a070 con 0x7f4c540b7ed0 2024-05-20T22:11:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.446+0000 7f4c4b7fe640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4c3406be60 0x7f4c3406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.447+0000 7f4c4b7fe640 1 -- 172.21.15.55:0/2399517956 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f4c3c08a300 con 0x7f4c540b7ed0 2024-05-20T22:11:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.447+0000 7f4c52d9d640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4c3406be60 0x7f4c3406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.447+0000 7f4c52d9d640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4c3406be60 0x7f4c3406e370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4c540b88d0 tx=0x7f4c44009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:06.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.451+0000 7f4c4b7fe640 1 -- 172.21.15.55:0/2399517956 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4c3c058450 con 0x7f4c540b7ed0 2024-05-20T22:11:06.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.676+0000 7f4c5921b640 1 -- 172.21.15.55:0/2399517956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c5415eee0 con 0x7f4c540b7ed0 2024-05-20T22:11:06.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.677+0000 7f4c4b7fe640 1 -- 172.21.15.55:0/2399517956 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7f4c3c05c750 con 0x7f4c540b7ed0 2024-05-20T22:11:06.678 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:06.678 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:11:06.679 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:11:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 -- 172.21.15.55:0/2399517956 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4c3406be60 msgr2=0x7f4c3406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4c3406be60 0x7f4c3406e370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4c540b88d0 tx=0x7f4c44009040 comp rx=0 tx=0).stop 2024-05-20T22:11:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 -- 172.21.15.55:0/2399517956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 msgr2=0x7f4c5415f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 0x7f4c5415f2b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00c480 tx=0x7f4c3c00c940 comp rx=0 tx=0).stop 2024-05-20T22:11:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 -- 172.21.15.55:0/2399517956 shutdown_connections 2024-05-20T22:11:06.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4c3406be60 0x7f4c3406e370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c54159870 0x7f4c541e5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c540b7ed0 0x7f4c5415f2b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 --2- 172.21.15.55:0/2399517956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c540b1df0 0x7f4c541f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 -- 172.21.15.55:0/2399517956 >> 172.21.15.55:0/2399517956 conn(0x7f4c540d29a0 msgr2=0x7f4c540b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:06.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 -- 172.21.15.55:0/2399517956 shutdown_connections 2024-05-20T22:11:06.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.679+0000 7f4c5921b640 1 -- 172.21.15.55:0/2399517956 wait complete. 2024-05-20T22:11:06.692 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:11:06.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.805+0000 7f03d5faa640 1 Processor -- start 2024-05-20T22:11:06.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.805+0000 7f03d5faa640 1 -- start start 2024-05-20T22:11:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03d5faa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f03d00d42a0 0x7f03d00cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03d5faa640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d0160550 con 0x7f03d00d3d80 2024-05-20T22:11:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03d5faa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d015f9d0 con 0x7f03d015ccc0 2024-05-20T22:11:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03d5faa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f03d015fcb0 con 0x7f03d00d42a0 2024-05-20T22:11:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03cffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f03d00d3d80 0x7f03d00b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36006/0 (socket says 172.21.15.55:36006) 2024-05-20T22:11:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03cffff640 1 -- 172.21.15.55:0/963310882 learned_addr learned my addr 172.21.15.55:0/963310882 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:06.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2143051182 0 0) 0x7f03d0160550 con 0x7f03d00d3d80 2024-05-20T22:11:06.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03cf7fe640 1 --2- 172.21.15.55:0/963310882 >> v2:172.21.15.138:3300/0 conn(0x7f03d00d42a0 0x7f03d00cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03b8003520 con 0x7f03d00d3d80 2024-05-20T22:11:06.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1574462660 0 0) 0x7f03b8003520 con 0x7f03d00d3d80 2024-05-20T22:11:06.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 >> v2:172.21.15.138:3300/0 conn(0x7f03d00d42a0 msgr2=0x7f03d00cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:06.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03ce7fc640 1 --2- 172.21.15.55:0/963310882 >> v2:172.21.15.138:3300/0 conn(0x7f03d00d42a0 0x7f03d00cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 >> v1:172.21.15.138:6789/0 conn(0x7f03d015ccc0 legacy=0x7f03d00d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d015ff90 con 0x7f03d00d3d80 2024-05-20T22:11:06.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f03c80019f0 con 0x7f03d00d3d80 2024-05-20T22:11:06.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.807+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 >> v1:172.21.15.55:6789/0 conn(0x7f03d00d3d80 legacy=0x7f03d00b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03ce7fc640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03ce7fc640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03b8004800 0x7f03b8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03ce7fc640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b8005250 0x7f03b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f03d015fcb0 con 0x7f03b80039b0 2024-05-20T22:11:06.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03b800c1e0 con 0x7f03b8004800 2024-05-20T22:11:06.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03b800c3e0 con 0x7f03b8005250 2024-05-20T22:11:06.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03cf7fe640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03ceffd640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03b8004800 0x7f03b8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03cffff640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b8005250 0x7f03b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03cf7fe640 1 -- 172.21.15.55:0/963310882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b8005250 msgr2=0x7f03b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:06.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03cf7fe640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03b8005250 0x7f03b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03cf7fe640 1 -- 172.21.15.55:0/963310882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03b8004800 msgr2=0x7f03b8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:06.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03cf7fe640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03b8004800 0x7f03b8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03cf7fe640 1 -- 172.21.15.55:0/963310882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b8003520 con 0x7f03b80039b0 2024-05-20T22:11:06.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03cffff640 1 -- 172.21.15.55:0/963310882 reap_dead start 2024-05-20T22:11:06.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.808+0000 7f03ceffd640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03b8004800 0x7f03b8004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:06.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.809+0000 7f03cf7fe640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03b8003e20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009ac0 tx=0x7f03b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b2b143d66085c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:06.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.809+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0036070 con 0x7f03b80039b0 2024-05-20T22:11:06.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.809+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c0031400 con 0x7f03b80039b0 2024-05-20T22:11:06.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.809+0000 7f03ce7fc640 1 -- 172.21.15.55:0/963310882 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0009690 con 0x7f03b80039b0 2024-05-20T22:11:06.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.810+0000 7f03d5faa640 1 -- 172.21.15.55:0/963310882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 msgr2=0x7f03b8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.810+0000 7f03d5faa640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03b8003e20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009ac0 tx=0x7f03b8005830 comp rx=0 tx=0).stop 2024-05-20T22:11:06.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.810+0000 7f03d5faa640 1 -- 172.21.15.55:0/963310882 shutdown_connections 2024-05-20T22:11:06.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.810+0000 7f03d5faa640 1 --2- 172.21.15.55:0/963310882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03b8003e20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.810+0000 7f03d5faa640 1 -- 172.21.15.55:0/963310882 >> 172.21.15.55:0/963310882 conn(0x7f03d00d1c50 msgr2=0x7f03d00d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:06.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.810+0000 7f03d5faa640 1 -- 172.21.15.55:0/963310882 shutdown_connections 2024-05-20T22:11:06.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.810+0000 7f03d5faa640 1 -- 172.21.15.55:0/963310882 wait complete. 2024-05-20T22:11:06.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.811+0000 7f03d5faa640 1 Processor -- start 2024-05-20T22:11:06.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.812+0000 7f03d5faa640 1 -- start start 2024-05-20T22:11:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03d5faa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03d01f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03d5faa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03d01f4a50 0x7f03d015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03d5faa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03d0160760 0x7f03d006bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03d5faa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f03d015f7b0 con 0x7f03b80039b0 2024-05-20T22:11:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03d5faa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03d015f630 con 0x7f03d0160760 2024-05-20T22:11:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03d5faa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03d00d42a0 con 0x7f03d01f4a50 2024-05-20T22:11:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03cf7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03d01f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03cf7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03d01f89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48870/0 (socket says 172.21.15.55:48870) 2024-05-20T22:11:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03cf7fe640 1 -- 172.21.15.55:0/3842099775 learned_addr learned my addr 172.21.15.55:0/3842099775 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03cffff640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03d0160760 0x7f03d006bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03ceffd640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03d01f4a50 0x7f03d015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03cf7fe640 1 -- 172.21.15.55:0/3842099775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03d01f4a50 msgr2=0x7f03d015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03cf7fe640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03d01f4a50 0x7f03d015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03cf7fe640 1 -- 172.21.15.55:0/3842099775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03d0160760 msgr2=0x7f03d006bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03cf7fe640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03d0160760 0x7f03d006bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03cf7fe640 1 -- 172.21.15.55:0/3842099775 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d01f8f00 con 0x7f03b80039b0 2024-05-20T22:11:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03cffff640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03d0160760 0x7f03d006bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.813+0000 7f03ceffd640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03d01f4a50 0x7f03d015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.814+0000 7f03cf7fe640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03d01f89c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f03c0034040 tx=0x7f03c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.814+0000 7f03b7fff640 1 -- 172.21.15.55:0/3842099775 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c003f020 con 0x7f03b80039b0 2024-05-20T22:11:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.814+0000 7f03b7fff640 1 -- 172.21.15.55:0/3842099775 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c0037040 con 0x7f03b80039b0 2024-05-20T22:11:06.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.814+0000 7f03b7fff640 1 -- 172.21.15.55:0/3842099775 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0036070 con 0x7f03b80039b0 2024-05-20T22:11:06.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.814+0000 7f03d5faa640 1 -- 172.21.15.55:0/3842099775 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d01f54a0 con 0x7f03b80039b0 2024-05-20T22:11:06.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.814+0000 7f03d5faa640 1 -- 172.21.15.55:0/3842099775 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d01f81c0 con 0x7f03b80039b0 2024-05-20T22:11:06.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.815+0000 7f03d5faa640 1 -- 172.21.15.55:0/3842099775 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d004f720 con 0x7f03b80039b0 2024-05-20T22:11:06.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.815+0000 7f03b7fff640 1 -- 172.21.15.55:0/3842099775 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03c0042020 con 0x7f03b80039b0 2024-05-20T22:11:06.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.816+0000 7f03b7fff640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03bc06be30 0x7f03bc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:06.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.816+0000 7f03b7fff640 1 -- 172.21.15.55:0/3842099775 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f03c00aafc0 con 0x7f03b80039b0 2024-05-20T22:11:06.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.816+0000 7f03ceffd640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03bc06be30 0x7f03bc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:06.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.816+0000 7f03ceffd640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03bc06be30 0x7f03bc06e340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000c00 tx=0x7f03c4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:06.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:06.820+0000 7f03b7fff640 1 -- 172.21.15.55:0/3842099775 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f03c0049070 con 0x7f03b80039b0 2024-05-20T22:11:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.042+0000 7f03d5faa640 1 -- 172.21.15.55:0/3842099775 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03d00d2f40 con 0x7f03b80039b0 2024-05-20T22:11:07.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.043+0000 7f03b7fff640 1 -- 172.21.15.55:0/3842099775 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5188 (secure 0 0 0) 0x7f03c00796b0 con 0x7f03b80039b0 2024-05-20T22:11:07.044 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:07.044 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T22:10:23.995145+0000","modified":"2024-05-20T22:10:42.319833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6189},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6189":{"gid":6189,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.55:6837/3463078364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3463078364},{"type":"v1","addr":"172.21.15.55:6837","nonce":3463078364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6189,"qdb_cluster":[6189]},"id":3}]} 2024-05-20T22:11:07.044 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 43 2024-05-20T22:11:07.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 -- 172.21.15.55:0/3842099775 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03bc06be30 msgr2=0x7f03bc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03bc06be30 0x7f03bc06e340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000c00 tx=0x7f03c4009040 comp rx=0 tx=0).stop 2024-05-20T22:11:07.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 -- 172.21.15.55:0/3842099775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 msgr2=0x7f03d01f89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03d01f89c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f03c0034040 tx=0x7f03c0004730 comp rx=0 tx=0).stop 2024-05-20T22:11:07.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 -- 172.21.15.55:0/3842099775 shutdown_connections 2024-05-20T22:11:07.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f03bc06be30 0x7f03bc06e340 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03d0160760 0x7f03d006bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03d01f4a50 0x7f03d015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 --2- 172.21.15.55:0/3842099775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b80039b0 0x7f03d01f89c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 -- 172.21.15.55:0/3842099775 >> 172.21.15.55:0/3842099775 conn(0x7f03d00d1c50 msgr2=0x7f03d00c74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:07.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 -- 172.21.15.55:0/3842099775 shutdown_connections 2024-05-20T22:11:07.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.045+0000 7f03d5faa640 1 -- 172.21.15.55:0/3842099775 wait complete. 2024-05-20T22:11:07.058 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:11:07.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.171+0000 7f93b1028640 1 Processor -- start 2024-05-20T22:11:07.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.171+0000 7f93b1028640 1 -- start start 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93b1028640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f93ac06c060 0x7f93ac15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93b1028640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ac0d3290 con 0x7f93ac0b3fd0 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93b1028640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ac0d3570 con 0x7f93ac06b540 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93b1028640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f93ac0d3ea0 con 0x7f93ac06c060 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93aad9d640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f93ac0b3fd0 0x7f93ac1635e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36014/0 (socket says 172.21.15.55:36014) 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93aad9d640 1 -- 172.21.15.55:0/388463268 learned_addr learned my addr 172.21.15.55:0/388463268 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93ab59e640 1 --2- 172.21.15.55:0/388463268 >> v2:172.21.15.138:3300/0 conn(0x7f93ac06c060 0x7f93ac15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1768998058 0 0) 0x7f93ac0d3290 con 0x7f93ac0b3fd0 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9394003520 con 0x7f93ac0b3fd0 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3953320198 0 0) 0x7f93ac0d3570 con 0x7f93ac06b540 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93ac0d3290 con 0x7f93ac06b540 2024-05-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2713424362 0 0) 0x7f9394003520 con 0x7f93ac0b3fd0 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 >> v2:172.21.15.138:3300/0 conn(0x7f93ac06c060 msgr2=0x7f93ac15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 --2- 172.21.15.55:0/388463268 >> v2:172.21.15.138:3300/0 conn(0x7f93ac06c060 0x7f93ac15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 >> v1:172.21.15.138:6789/0 conn(0x7f93ac06b540 legacy=0x7f93ac06b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac0d3850 con 0x7f93ac0b3fd0 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f939c001670 con 0x7f93ac0b3fd0 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 >> v1:172.21.15.55:6789/0 conn(0x7f93ac0b3fd0 legacy=0x7f93ac1635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394003b40 0x7f9394003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9394004800 0x7f9394004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93940051a0 0x7f939400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac0d3ea0 con 0x7f9394003b40 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f939400c140 con 0x7f93940051a0 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f939400c340 con 0x7f9394004800 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.173+0000 7f93aad9d640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394003b40 0x7f9394003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.174+0000 7f93ab59e640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93940051a0 0x7f939400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.174+0000 7f93aa59c640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9394004800 0x7f9394004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.174+0000 7f93aad9d640 1 -- 172.21.15.55:0/388463268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9394004800 msgr2=0x7f9394004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.174+0000 7f93aad9d640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9394004800 0x7f9394004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.174+0000 7f93aad9d640 1 -- 172.21.15.55:0/388463268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93940051a0 msgr2=0x7f939400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.174+0000 7f93aad9d640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93940051a0 0x7f939400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.174+0000 7f93aad9d640 1 -- 172.21.15.55:0/388463268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9394003520 con 0x7f9394003b40 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.174+0000 7f93ab59e640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93940051a0 0x7f939400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.174+0000 7f93ab59e640 1 -- 172.21.15.55:0/388463268 reap_dead start 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.174+0000 7f93aad9d640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394003b40 0x7f9394003f90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9394004780 tx=0x7f9394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d78a46de12c01a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.175+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c007ce0 con 0x7f9394003b40 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.175+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939c005ad0 con 0x7f9394003b40 2024-05-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.175+0000 7f93a9d9b640 1 -- 172.21.15.55:0/388463268 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c007070 con 0x7f9394003b40 2024-05-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.175+0000 7f93b1028640 1 -- 172.21.15.55:0/388463268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394003b40 msgr2=0x7f9394003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.175+0000 7f93b1028640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394003b40 0x7f9394003f90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9394004780 tx=0x7f9394005830 comp rx=0 tx=0).stop 2024-05-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.175+0000 7f93b1028640 1 -- 172.21.15.55:0/388463268 shutdown_connections 2024-05-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.175+0000 7f93b1028640 1 --2- 172.21.15.55:0/388463268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9394003b40 0x7f9394003f90 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.175+0000 7f93b1028640 1 -- 172.21.15.55:0/388463268 >> 172.21.15.55:0/388463268 conn(0x7f93ac0d1c50 msgr2=0x7f93ac0d4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.175+0000 7f93b1028640 1 -- 172.21.15.55:0/388463268 shutdown_connections 2024-05-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.175+0000 7f93b1028640 1 -- 172.21.15.55:0/388463268 wait complete. 2024-05-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.177+0000 7f93b1028640 1 Processor -- start 2024-05-20T22:11:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.178+0000 7f93b1028640 1 -- start start 2024-05-20T22:11:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93b1028640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9394003b40 0x7f93ac157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93b1028640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f93ac0b8a30 0x7f93ac0c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93b1028640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93ac15c7e0 0x7f93ac160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93b1028640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac06c470 con 0x7f93ac15c7e0 2024-05-20T22:11:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93b1028640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac0d4330 con 0x7f9394003b40 2024-05-20T22:11:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93b1028640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f93ac06c5f0 con 0x7f93ac0b8a30 2024-05-20T22:11:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93ab59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93ac15c7e0 0x7f93ac160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93ab59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93ac15c7e0 0x7f93ac160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48888/0 (socket says 172.21.15.55:48888) 2024-05-20T22:11:07.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93ab59e640 1 -- 172.21.15.55:0/2147588730 learned_addr learned my addr 172.21.15.55:0/2147588730 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93aa59c640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f93ac0b8a30 0x7f93ac0c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93aad9d640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9394003b40 0x7f93ac157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93ab59e640 1 -- 172.21.15.55:0/2147588730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f93ac0b8a30 msgr2=0x7f93ac0c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93ab59e640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f93ac0b8a30 0x7f93ac0c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93ab59e640 1 -- 172.21.15.55:0/2147588730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9394003b40 msgr2=0x7f93ac157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93ab59e640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9394003b40 0x7f93ac157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.179+0000 7f93ab59e640 1 -- 172.21.15.55:0/2147588730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac15cbf0 con 0x7f93ac15c7e0 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.180+0000 7f93ab59e640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93ac15c7e0 0x7f93ac160db0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0d3e10 tx=0x7f93a4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.180+0000 7f93937fe640 1 -- 172.21.15.55:0/2147588730 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4010e00 con 0x7f93ac15c7e0 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.180+0000 7f93937fe640 1 -- 172.21.15.55:0/2147588730 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a4003990 con 0x7f93ac15c7e0 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.180+0000 7f93b1028640 1 -- 172.21.15.55:0/2147588730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ac15a4c0 con 0x7f93ac15c7e0 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.180+0000 7f93937fe640 1 -- 172.21.15.55:0/2147588730 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4003d80 con 0x7f93ac15c7e0 2024-05-20T22:11:07.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.180+0000 7f93b1028640 1 -- 172.21.15.55:0/2147588730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ac15aa80 con 0x7f93ac15c7e0 2024-05-20T22:11:07.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.181+0000 7f93b1028640 1 -- 172.21.15.55:0/2147588730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ac04f720 con 0x7f93ac15c7e0 2024-05-20T22:11:07.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.181+0000 7f93937fe640 1 -- 172.21.15.55:0/2147588730 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93a401e020 con 0x7f93ac15c7e0 2024-05-20T22:11:07.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.182+0000 7f93937fe640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f939806be60 0x7f939806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.182+0000 7f93937fe640 1 -- 172.21.15.55:0/2147588730 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f93a408b740 con 0x7f93ac15c7e0 2024-05-20T22:11:07.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.182+0000 7f93aad9d640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f939806be60 0x7f939806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.182+0000 7f93aad9d640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f939806be60 0x7f939806e370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f939c032000 tx=0x7f939c009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.187+0000 7f93937fe640 1 -- 172.21.15.55:0/2147588730 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f93a4059f60 con 0x7f93ac15c7e0 2024-05-20T22:11:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.375+0000 7f93b1028640 1 -- 172.21.15.55:0/2147588730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93ac158610 con 0x7f93ac15c7e0 2024-05-20T22:11:07.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.376+0000 7f93937fe640 1 -- 172.21.15.55:0/2147588730 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+17167 (secure 0 0 0) 0x7f93a4059860 con 0x7f93ac15c7e0 2024-05-20T22:11:07.377 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:07.377 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":61,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:10:38.297580+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T22:10:22.369451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T22:10:22.970959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.25","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:6837/2445634740":"2024-05-21T22:10:38.297533+0000","172.21.15.55:6835/2445634740":"2024-05-21T22:10:38.297533+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:11:07.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 -- 172.21.15.55:0/2147588730 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f939806be60 msgr2=0x7f939806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f939806be60 0x7f939806e370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f939c032000 tx=0x7f939c009030 comp rx=0 tx=0).stop 2024-05-20T22:11:07.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 -- 172.21.15.55:0/2147588730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93ac15c7e0 msgr2=0x7f93ac160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93ac15c7e0 0x7f93ac160db0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0d3e10 tx=0x7f93a4010590 comp rx=0 tx=0).stop 2024-05-20T22:11:07.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 -- 172.21.15.55:0/2147588730 shutdown_connections 2024-05-20T22:11:07.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f939806be60 0x7f939806e370 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93ac15c7e0 0x7f93ac160db0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f93ac0b8a30 0x7f93ac0c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 --2- 172.21.15.55:0/2147588730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9394003b40 0x7f93ac157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 -- 172.21.15.55:0/2147588730 >> 172.21.15.55:0/2147588730 conn(0x7f93ac0d1c50 msgr2=0x7f93ac158950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:07.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 -- 172.21.15.55:0/2147588730 shutdown_connections 2024-05-20T22:11:07.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.378+0000 7f93b1028640 1 -- 172.21.15.55:0/2147588730 wait complete. 2024-05-20T22:11:07.391 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T22:11:07.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.506+0000 7f58bc27a640 1 Processor -- start 2024-05-20T22:11:07.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.506+0000 7f58bc27a640 1 -- start start 2024-05-20T22:11:07.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.507+0000 7f58bc27a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58b4162af0 0x7f58b40d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.507+0000 7f58bc27a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b40d2050 con 0x7f58b406dde0 2024-05-20T22:11:07.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.507+0000 7f58bc27a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b40d2330 con 0x7f58b40b7df0 2024-05-20T22:11:07.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.507+0000 7f58bc27a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f58b40cd170 con 0x7f58b4162af0 2024-05-20T22:11:07.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.507+0000 7f58ba817640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58b4162af0 0x7f58b40d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.507+0000 7f58ba817640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58b4162af0 0x7f58b40d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48902/0 (socket says 172.21.15.55:48902) 2024-05-20T22:11:07.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.508+0000 7f58ba817640 1 -- 172.21.15.55:0/2335205924 learned_addr learned my addr 172.21.15.55:0/2335205924 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:07.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.508+0000 7f58ba817640 1 -- 172.21.15.55:0/2335205924 >> v1:172.21.15.138:6789/0 conn(0x7f58b40b7df0 legacy=0x7f58b406d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.508+0000 7f58ba817640 1 -- 172.21.15.55:0/2335205924 >> v1:172.21.15.55:6789/0 conn(0x7f58b406dde0 legacy=0x7f58b40cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.508+0000 7f58ba817640 1 -- 172.21.15.55:0/2335205924 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b40cd450 con 0x7f58b4162af0 2024-05-20T22:11:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.509+0000 7f58ba817640 1 --2- 172.21.15.55:0/2335205924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b4162af0 0x7f58b40d1920 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009e50 tx=0x7f58ac02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ad3fdd81c5222a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.509+0000 7f58b9014640 1 -- 172.21.15.55:0/2335205924 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac036070 con 0x7f58b4162af0 2024-05-20T22:11:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.509+0000 7f58b9014640 1 -- 172.21.15.55:0/2335205924 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac031600 con 0x7f58b4162af0 2024-05-20T22:11:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.509+0000 7f58b9014640 1 -- 172.21.15.55:0/2335205924 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0319d0 con 0x7f58b4162af0 2024-05-20T22:11:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.509+0000 7f58bc27a640 1 -- 172.21.15.55:0/2335205924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b4162af0 msgr2=0x7f58b40d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.509+0000 7f58bc27a640 1 --2- 172.21.15.55:0/2335205924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b4162af0 0x7f58b40d1920 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009e50 tx=0x7f58ac02a730 comp rx=0 tx=0).stop 2024-05-20T22:11:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.510+0000 7f58bc27a640 1 -- 172.21.15.55:0/2335205924 shutdown_connections 2024-05-20T22:11:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.510+0000 7f58bc27a640 1 --2- 172.21.15.55:0/2335205924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b4162af0 0x7f58b40d1920 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.510+0000 7f58bc27a640 1 -- 172.21.15.55:0/2335205924 >> 172.21.15.55:0/2335205924 conn(0x7f58b40d29a0 msgr2=0x7f58b40d2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.510+0000 7f58bc27a640 1 -- 172.21.15.55:0/2335205924 shutdown_connections 2024-05-20T22:11:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.510+0000 7f58bc27a640 1 -- 172.21.15.55:0/2335205924 wait complete. 2024-05-20T22:11:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.511+0000 7f58bc27a640 1 Processor -- start 2024-05-20T22:11:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.512+0000 7f58bc27a640 1 -- start start 2024-05-20T22:11:07.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58bc27a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b40b7df0 0x7f58b4158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58bc27a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58b4162af0 0x7f58b414c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58bc27a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b41488c0 0x7f58b414a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58bc27a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40ce410 con 0x7f58b40b7df0 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58bc27a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40ce290 con 0x7f58b41488c0 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58bc27a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f58b40ce590 con 0x7f58b4162af0 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba016640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b40b7df0 0x7f58b4158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba016640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b40b7df0 0x7f58b4158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48910/0 (socket says 172.21.15.55:48910) 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba016640 1 -- 172.21.15.55:0/181590435 learned_addr learned my addr 172.21.15.55:0/181590435 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba817640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b41488c0 0x7f58b414a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58b9815640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58b4162af0 0x7f58b414c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba016640 1 -- 172.21.15.55:0/181590435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58b4162af0 msgr2=0x7f58b414c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba016640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58b4162af0 0x7f58b414c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba016640 1 -- 172.21.15.55:0/181590435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b41488c0 msgr2=0x7f58b414a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba016640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b41488c0 0x7f58b414a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba016640 1 -- 172.21.15.55:0/181590435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b41491a0 con 0x7f58b40b7df0 2024-05-20T22:11:07.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba817640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b41488c0 0x7f58b414a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:07.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.513+0000 7f58ba016640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b40b7df0 0x7f58b4158130 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c3c0 tx=0x7f58b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:07.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.514+0000 7f58aaffd640 1 -- 172.21.15.55:0/181590435 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b00052d0 con 0x7f58b40b7df0 2024-05-20T22:11:07.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.514+0000 7f58aaffd640 1 -- 172.21.15.55:0/181590435 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b0005470 con 0x7f58b40b7df0 2024-05-20T22:11:07.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.514+0000 7f58bc27a640 1 -- 172.21.15.55:0/181590435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b4149480 con 0x7f58b40b7df0 2024-05-20T22:11:07.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.514+0000 7f58aaffd640 1 -- 172.21.15.55:0/181590435 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0006a60 con 0x7f58b40b7df0 2024-05-20T22:11:07.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.514+0000 7f58bc27a640 1 -- 172.21.15.55:0/181590435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b4149a40 con 0x7f58b40b7df0 2024-05-20T22:11:07.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.515+0000 7f58bc27a640 1 -- 172.21.15.55:0/181590435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b40d1ae0 con 0x7f58b40b7df0 2024-05-20T22:11:07.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.515+0000 7f58aaffd640 1 -- 172.21.15.55:0/181590435 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58b0003640 con 0x7f58b40b7df0 2024-05-20T22:11:07.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.516+0000 7f58aaffd640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f589806be60 0x7f589806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.516+0000 7f58aaffd640 1 -- 172.21.15.55:0/181590435 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6245+0+0 (secure 0 0 0) 0x7f58b008a530 con 0x7f58b40b7df0 2024-05-20T22:11:07.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.516+0000 7f58b9815640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f589806be60 0x7f589806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.516+0000 7f58b9815640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f589806be60 0x7f589806e370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f58b40cde00 tx=0x7f58a4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:07.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.521+0000 7f58aaffd640 1 -- 172.21.15.55:0/181590435 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f58b0058c20 con 0x7f58b40b7df0 2024-05-20T22:11:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.741+0000 7f58bc27a640 1 -- 172.21.15.55:0/181590435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f58b40d1800 con 0x7f58b40b7df0 2024-05-20T22:11:07.742 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-20T22:11:07.741+0000 7f0b3ef60640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T22:11:07.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.743+0000 7f58aaffd640 1 -- 172.21.15.55:0/181590435 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 161+0+0 (secure 0 0 0) 0x7f58b0058520 con 0x7f58b40b7df0 2024-05-20T22:11:07.744 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T22:11:07.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 -- 172.21.15.55:0/181590435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f589806be60 msgr2=0x7f589806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f589806be60 0x7f589806e370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f58b40cde00 tx=0x7f58a4009040 comp rx=0 tx=0).stop 2024-05-20T22:11:07.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 -- 172.21.15.55:0/181590435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b40b7df0 msgr2=0x7f58b4158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b40b7df0 0x7f58b4158130 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c3c0 tx=0x7f58b000c880 comp rx=0 tx=0).stop 2024-05-20T22:11:07.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 -- 172.21.15.55:0/181590435 shutdown_connections 2024-05-20T22:11:07.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f589806be60 0x7f589806e370 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b41488c0 0x7f58b414a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58b4162af0 0x7f58b414c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 --2- 172.21.15.55:0/181590435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58b40b7df0 0x7f58b4158130 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 -- 172.21.15.55:0/181590435 >> 172.21.15.55:0/181590435 conn(0x7f58b40d29a0 msgr2=0x7f58b40b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:07.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 -- 172.21.15.55:0/181590435 shutdown_connections 2024-05-20T22:11:07.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.745+0000 7f58bc27a640 1 -- 172.21.15.55:0/181590435 wait complete. 2024-05-20T22:11:07.748 INFO:tasks.ceph.mds.a.smithi055.stderr: -1507> 2024-05-20T22:10:43.309+0000 7efd6d0e9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T22:11:07.759 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T22:11:07.803 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-20T22:11:07.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.877+0000 7faccc99d640 1 Processor -- start 2024-05-20T22:11:07.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.877+0000 7faccc99d640 1 -- start start 2024-05-20T22:11:07.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.878+0000 7faccc99d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facc815f170 0x7facc80d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.878+0000 7faccc99d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7facc806bde0 0x7facc806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.878+0000 7faccc99d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facc80b3730 con 0x7facc80cf870 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.878+0000 7faccc99d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7facc815f580 con 0x7facc806bde0 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.878+0000 7faccc99d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7facc80b3f90 con 0x7facc815f170 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.878+0000 7facc6d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7facc806bde0 0x7facc806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.878+0000 7facc5d9b640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7facc80cf870 0x7facc80cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36044/0 (socket says 172.21.15.55:36044) 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.878+0000 7facc5d9b640 1 -- 172.21.15.55:0/792744506 learned_addr learned my addr 172.21.15.55:0/792744506 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.879+0000 7facc659c640 1 --2- 172.21.15.55:0/792744506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facc815f170 0x7facc80d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.879+0000 7facc6d9d640 1 -- 172.21.15.55:0/792744506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facc815f170 msgr2=0x7facc80d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.879+0000 7facc6d9d640 1 --2- 172.21.15.55:0/792744506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facc815f170 0x7facc80d3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.879+0000 7facc6d9d640 1 -- 172.21.15.55:0/792744506 >> v1:172.21.15.55:6789/0 conn(0x7facc80cf870 legacy=0x7facc80cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.879+0000 7facc6d9d640 1 -- 172.21.15.55:0/792744506 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc80b39d0 con 0x7facc806bde0 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.879+0000 7facc6d9d640 1 --2- 172.21.15.55:0/792744506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 0x7facc806c1f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7facc0009d90 tx=0x7facc002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294cbf2e362e07 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7facc559a640 1 -- 172.21.15.55:0/792744506 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0007740 con 0x7facc806bde0 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7facc559a640 1 -- 172.21.15.55:0/792744506 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc00078e0 con 0x7facc806bde0 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7facc559a640 1 -- 172.21.15.55:0/792744506 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0007cb0 con 0x7facc806bde0 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7faccc99d640 1 -- 172.21.15.55:0/792744506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 msgr2=0x7facc806c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7faccc99d640 1 --2- 172.21.15.55:0/792744506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 0x7facc806c1f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7facc0009d90 tx=0x7facc002a680 comp rx=0 tx=0).stop 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7faccc99d640 1 -- 172.21.15.55:0/792744506 shutdown_connections 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7faccc99d640 1 --2- 172.21.15.55:0/792744506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 0x7facc806c1f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7faccc99d640 1 --2- 172.21.15.55:0/792744506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facc815f170 0x7facc80d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7faccc99d640 1 -- 172.21.15.55:0/792744506 >> 172.21.15.55:0/792744506 conn(0x7facc80d4b30 msgr2=0x7facc80d4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7faccc99d640 1 -- 172.21.15.55:0/792744506 shutdown_connections 2024-05-20T22:11:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.880+0000 7faccc99d640 1 -- 172.21.15.55:0/792744506 wait complete. 2024-05-20T22:11:07.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.882+0000 7faccc99d640 1 Processor -- start 2024-05-20T22:11:07.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.882+0000 7faccc99d640 1 -- start start 2024-05-20T22:11:07.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7faccc99d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 0x7facc8153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7faccc99d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facc80cf870 0x7facc8159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7faccc99d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc815f170 0x7facc81ea760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7faccc99d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7facc806b8c0 con 0x7facc806bde0 2024-05-20T22:11:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7faccc99d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facc806b5e0 con 0x7facc815f170 2024-05-20T22:11:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7faccc99d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7facc80d0530 con 0x7facc80cf870 2024-05-20T22:11:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 0x7facc8153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 0x7facc8153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48936/0 (socket says 172.21.15.55:48936) 2024-05-20T22:11:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc659c640 1 -- 172.21.15.55:0/2831283738 learned_addr learned my addr 172.21.15.55:0/2831283738 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc5d9b640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facc80cf870 0x7facc8159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc6d9d640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc815f170 0x7facc81ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc659c640 1 -- 172.21.15.55:0/2831283738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facc80cf870 msgr2=0x7facc8159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc659c640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facc80cf870 0x7facc8159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc659c640 1 -- 172.21.15.55:0/2831283738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc815f170 msgr2=0x7facc81ea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc659c640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc815f170 0x7facc81ea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc659c640 1 -- 172.21.15.55:0/2831283738 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc8154110 con 0x7facc806bde0 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc6d9d640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc815f170 0x7facc81ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facc659c640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 0x7facc8153450 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7facb800c7f0 tx=0x7facb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facb6ffd640 1 -- 172.21.15.55:0/2831283738 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8015070 con 0x7facc806bde0 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facb6ffd640 1 -- 172.21.15.55:0/2831283738 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb8004220 con 0x7facc806bde0 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7faccc99d640 1 -- 172.21.15.55:0/2831283738 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc81543f0 con 0x7facc806bde0 2024-05-20T22:11:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7facb6ffd640 1 -- 172.21.15.55:0/2831283738 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8007620 con 0x7facc806bde0 2024-05-20T22:11:07.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.884+0000 7faccc99d640 1 -- 172.21.15.55:0/2831283738 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc81549b0 con 0x7facc806bde0 2024-05-20T22:11:07.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.886+0000 7faccc99d640 1 -- 172.21.15.55:0/2831283738 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc815a6a0 con 0x7facc806bde0 2024-05-20T22:11:07.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.886+0000 7facb6ffd640 1 -- 172.21.15.55:0/2831283738 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7facb80079d0 con 0x7facc806bde0 2024-05-20T22:11:07.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.886+0000 7facb6ffd640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faca806be60 0x7faca806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:07.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.886+0000 7facb6ffd640 1 -- 172.21.15.55:0/2831283738 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6331+0+0 (secure 0 0 0) 0x7facb808b070 con 0x7facc806bde0 2024-05-20T22:11:07.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.887+0000 7facc5d9b640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faca806be60 0x7faca806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:07.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.887+0000 7facc5d9b640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faca806be60 0x7faca806e370 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7facbc007fd0 tx=0x7facbc007e80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:07.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:07.891+0000 7facb6ffd640 1 -- 172.21.15.55:0/2831283738 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7facb80590d0 con 0x7facc806bde0 2024-05-20T22:11:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.120+0000 7faccc99d640 1 -- 172.21.15.55:0/2831283738 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7facc804f720 con 0x7facc806bde0 2024-05-20T22:11:08.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.756+0000 7facb6ffd640 1 -- 172.21.15.55:0/2831283738 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7facb805d3d0 con 0x7facc806bde0 2024-05-20T22:11:08.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 -- 172.21.15.55:0/2831283738 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faca806be60 msgr2=0x7faca806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:08.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faca806be60 0x7faca806e370 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7facbc007fd0 tx=0x7facbc007e80 comp rx=0 tx=0).stop 2024-05-20T22:11:08.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 -- 172.21.15.55:0/2831283738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 msgr2=0x7facc8153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:08.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 0x7facc8153450 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7facb800c7f0 tx=0x7facb800ccb0 comp rx=0 tx=0).stop 2024-05-20T22:11:08.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 -- 172.21.15.55:0/2831283738 shutdown_connections 2024-05-20T22:11:08.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7faca806be60 0x7faca806e370 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:08.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc815f170 0x7facc81ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:08.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facc80cf870 0x7facc8159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:08.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 --2- 172.21.15.55:0/2831283738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc806bde0 0x7facc8153450 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:08.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 -- 172.21.15.55:0/2831283738 >> 172.21.15.55:0/2831283738 conn(0x7facc80d4b30 msgr2=0x7facc80b6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:08.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 -- 172.21.15.55:0/2831283738 shutdown_connections 2024-05-20T22:11:08.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.759+0000 7faccc99d640 1 -- 172.21.15.55:0/2831283738 wait complete. 2024-05-20T22:11:08.775 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T22:11:08.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.902+0000 7fdb7c65a640 1 Processor -- start 2024-05-20T22:11:08.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.902+0000 7fdb7c65a640 1 -- start start 2024-05-20T22:11:08.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.903+0000 7fdb7c65a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdb740d3560 0x7fdb7415cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:08.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.903+0000 7fdb7c65a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb741601d0 0x7fdb740cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.903+0000 7fdb7c65a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb740d3160 con 0x7fdb740cfe40 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.903+0000 7fdb7c65a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fdb740d0250 con 0x7fdb740d3560 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.903+0000 7fdb7c65a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdb740d3a20 con 0x7fdb741601d0 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.903+0000 7fdb7a3f6640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdb740d3560 0x7fdb7415cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.903+0000 7fdb7abf7640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdb740cfe40 0x7fdb74162d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44268/0 (socket says 172.21.15.55:44268) 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.903+0000 7fdb7abf7640 1 -- 172.21.15.55:0/1805053216 learned_addr learned my addr 172.21.15.55:0/1805053216 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.904+0000 7fdb79bf5640 1 --2- 172.21.15.55:0/1805053216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb741601d0 0x7fdb740cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.904+0000 7fdb7a3f6640 1 -- 172.21.15.55:0/1805053216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb741601d0 msgr2=0x7fdb740cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.904+0000 7fdb7a3f6640 1 --2- 172.21.15.55:0/1805053216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb741601d0 0x7fdb740cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.904+0000 7fdb7a3f6640 1 -- 172.21.15.55:0/1805053216 >> v1:172.21.15.55:6789/0 conn(0x7fdb740cfe40 legacy=0x7fdb74162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.904+0000 7fdb7a3f6640 1 -- 172.21.15.55:0/1805053216 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb740d3d00 con 0x7fdb740d3560 2024-05-20T22:11:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.904+0000 7fdb793f4640 1 -- 172.21.15.55:0/1805053216 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2886325196 0 0) 0x7fdb740d3160 con 0x7fdb740cfe40 2024-05-20T22:11:08.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.904+0000 7fdb7a3f6640 1 --2- 172.21.15.55:0/1805053216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb740d3560 0x7fdb7415cd00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fdb700097a0 tx=0x7fdb7002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38965664c9ab658a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:08.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.905+0000 7fdb793f4640 1 -- 172.21.15.55:0/1805053216 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70037070 con 0x7fdb740d3560 2024-05-20T22:11:08.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.905+0000 7fdb793f4640 1 -- 172.21.15.55:0/1805053216 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7002ae50 con 0x7fdb740d3560 2024-05-20T22:11:08.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.905+0000 7fdb793f4640 1 -- 172.21.15.55:0/1805053216 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70032660 con 0x7fdb740d3560 2024-05-20T22:11:08.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.905+0000 7fdb7c65a640 1 -- 172.21.15.55:0/1805053216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb740d3560 msgr2=0x7fdb7415cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:08.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.905+0000 7fdb7c65a640 1 --2- 172.21.15.55:0/1805053216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb740d3560 0x7fdb7415cd00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fdb700097a0 tx=0x7fdb7002a100 comp rx=0 tx=0).stop 2024-05-20T22:11:08.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.905+0000 7fdb7c65a640 1 -- 172.21.15.55:0/1805053216 shutdown_connections 2024-05-20T22:11:08.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.905+0000 7fdb7c65a640 1 --2- 172.21.15.55:0/1805053216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb741601d0 0x7fdb740cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:08.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.905+0000 7fdb7c65a640 1 --2- 172.21.15.55:0/1805053216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb740d3560 0x7fdb7415cd00 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.905+0000 7fdb7c65a640 1 -- 172.21.15.55:0/1805053216 >> 172.21.15.55:0/1805053216 conn(0x7fdb740d1f80 msgr2=0x7fdb740d4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.906+0000 7fdb7c65a640 1 -- 172.21.15.55:0/1805053216 shutdown_connections 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.906+0000 7fdb7c65a640 1 -- 172.21.15.55:0/1805053216 wait complete. 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.907+0000 7fdb7c65a640 1 Processor -- start 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.907+0000 7fdb7c65a640 1 -- start start 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7c65a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb740cfe40 0x7fdb740c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7c65a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb741601d0 0x7fdb7415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7c65a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb74160fb0 0x7fdb740ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7c65a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741605e0 con 0x7fdb74160fb0 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7c65a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb740d42c0 con 0x7fdb741601d0 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7c65a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdb740cad00 con 0x7fdb740cfe40 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7abf7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb74160fb0 0x7fdb740ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:08.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7abf7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb74160fb0 0x7fdb740ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35314/0 (socket says 172.21.15.55:35314) 2024-05-20T22:11:08.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7abf7640 1 -- 172.21.15.55:0/3265852496 learned_addr learned my addr 172.21.15.55:0/3265852496 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:08.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7a3f6640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb740cfe40 0x7fdb740c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:08.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb79bf5640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb741601d0 0x7fdb7415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:08.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7abf7640 1 -- 172.21.15.55:0/3265852496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb740cfe40 msgr2=0x7fdb740c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:08.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7abf7640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb740cfe40 0x7fdb740c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:08.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7abf7640 1 -- 172.21.15.55:0/3265852496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb741601d0 msgr2=0x7fdb7415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:08.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7abf7640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb741601d0 0x7fdb7415b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:08.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7abf7640 1 -- 172.21.15.55:0/3265852496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb740c8920 con 0x7fdb74160fb0 2024-05-20T22:11:08.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7abf7640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb74160fb0 0x7fdb740ce410 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00c7b0 tx=0x7fdb6c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:08.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb6affd640 1 -- 172.21.15.55:0/3265852496 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c018070 con 0x7fdb74160fb0 2024-05-20T22:11:08.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb6affd640 1 -- 172.21.15.55:0/3265852496 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6c00a640 con 0x7fdb74160fb0 2024-05-20T22:11:08.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb7c65a640 1 -- 172.21.15.55:0/3265852496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb740c8c00 con 0x7fdb74160fb0 2024-05-20T22:11:08.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.909+0000 7fdb6affd640 1 -- 172.21.15.55:0/3265852496 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c00ce60 con 0x7fdb74160fb0 2024-05-20T22:11:08.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.910+0000 7fdb7c65a640 1 -- 172.21.15.55:0/3265852496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb740c91c0 con 0x7fdb74160fb0 2024-05-20T22:11:08.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.910+0000 7fdb7c65a640 1 -- 172.21.15.55:0/3265852496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7408b990 con 0x7fdb74160fb0 2024-05-20T22:11:08.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.911+0000 7fdb6affd640 1 -- 172.21.15.55:0/3265852496 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb6c01b070 con 0x7fdb74160fb0 2024-05-20T22:11:08.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.912+0000 7fdb6affd640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdb5806be30 0x7fdb5806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:08.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.912+0000 7fdb7a3f6640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdb5806be30 0x7fdb5806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:08.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.912+0000 7fdb6affd640 1 -- 172.21.15.55:0/3265852496 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6331+0+0 (secure 0 0 0) 0x7fdb6c08a690 con 0x7fdb74160fb0 2024-05-20T22:11:08.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.912+0000 7fdb7a3f6640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdb5806be30 0x7fdb5806e340 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdb70034000 tx=0x7fdb7002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:08.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:08.917+0000 7fdb6affd640 1 -- 172.21.15.55:0/3265852496 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdb6c058dc0 con 0x7fdb74160fb0 2024-05-20T22:11:09.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.102+0000 7fdb7c65a640 1 -- 172.21.15.55:0/3265852496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdb740d2ee0 con 0x7fdb74160fb0 2024-05-20T22:11:09.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.758+0000 7fdb6affd640 1 -- 172.21.15.55:0/3265852496 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) ==== 181+0+0 (secure 0 0 0) 0x7fdb6c0586c0 con 0x7fdb74160fb0 2024-05-20T22:11:09.759 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-05-20T22:11:09.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.761+0000 7fdb7c65a640 1 -- 172.21.15.55:0/3265852496 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdb5806be30 msgr2=0x7fdb5806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.761+0000 7fdb7c65a640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdb5806be30 0x7fdb5806e340 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdb70034000 tx=0x7fdb7002e000 comp rx=0 tx=0).stop 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.761+0000 7fdb7c65a640 1 -- 172.21.15.55:0/3265852496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb74160fb0 msgr2=0x7fdb740ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.761+0000 7fdb7c65a640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb74160fb0 0x7fdb740ce410 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00c7b0 tx=0x7fdb6c00cc70 comp rx=0 tx=0).stop 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.762+0000 7fdb7c65a640 1 -- 172.21.15.55:0/3265852496 shutdown_connections 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.762+0000 7fdb7c65a640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fdb5806be30 0x7fdb5806e340 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.762+0000 7fdb7c65a640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb74160fb0 0x7fdb740ce410 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.762+0000 7fdb7c65a640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb741601d0 0x7fdb7415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.762+0000 7fdb7c65a640 1 --2- 172.21.15.55:0/3265852496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb740cfe40 0x7fdb740c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.762+0000 7fdb7c65a640 1 -- 172.21.15.55:0/3265852496 >> 172.21.15.55:0/3265852496 conn(0x7fdb740d1f80 msgr2=0x7fdb7415bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.762+0000 7fdb7c65a640 1 -- 172.21.15.55:0/3265852496 shutdown_connections 2024-05-20T22:11:09.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.762+0000 7fdb7c65a640 1 -- 172.21.15.55:0/3265852496 wait complete. 2024-05-20T22:11:09.777 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T22:11:09.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.900+0000 7facd92c3640 1 Processor -- start 2024-05-20T22:11:09.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.901+0000 7facd92c3640 1 -- start start 2024-05-20T22:11:09.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd92c3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7facd4162e80 0x7facd415f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:09.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd92c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd40cb8d0 0x7facd406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd92c3640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7facd406cb20 0x7facd40cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd92c3640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7facd406c150 con 0x7facd406cb20 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd92c3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7facd406b540 con 0x7facd4162e80 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd92c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7facd406b820 con 0x7facd40cb8d0 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd37fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7facd406cb20 0x7facd40cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd37fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7facd406cb20 0x7facd40cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35330/0 (socket says 172.21.15.55:35330) 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd37fe640 1 -- 172.21.15.55:0/1534780295 learned_addr learned my addr 172.21.15.55:0/1534780295 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd37fe640 1 -- 172.21.15.55:0/1534780295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd40cb8d0 msgr2=0x7facd406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd37fe640 1 --2- 172.21.15.55:0/1534780295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd40cb8d0 0x7facd406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd37fe640 1 -- 172.21.15.55:0/1534780295 >> v2:172.21.15.138:3300/0 conn(0x7facd4162e80 msgr2=0x7facd415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd2ffd640 1 --2- 172.21.15.55:0/1534780295 >> v2:172.21.15.138:3300/0 conn(0x7facd4162e80 0x7facd415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd37fe640 1 --2- 172.21.15.55:0/1534780295 >> v2:172.21.15.138:3300/0 conn(0x7facd4162e80 0x7facd415f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd37fe640 1 -- 172.21.15.55:0/1534780295 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd406bb00 con 0x7facd406cb20 2024-05-20T22:11:09.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.902+0000 7facd2ffd640 1 --2- 172.21.15.55:0/1534780295 >> v2:172.21.15.138:3300/0 conn(0x7facd4162e80 0x7facd415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.903+0000 7facd37fe640 1 --2- 172.21.15.55:0/1534780295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd406cb20 0x7facd40cfc90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7facd408d580 tx=0x7faccc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b7bad74dba58c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.903+0000 7facd1ffb640 1 -- 172.21.15.55:0/1534780295 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc037070 con 0x7facd406cb20 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.903+0000 7facd1ffb640 1 -- 172.21.15.55:0/1534780295 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faccc032440 con 0x7facd406cb20 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.903+0000 7facd1ffb640 1 -- 172.21.15.55:0/1534780295 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc032810 con 0x7facd406cb20 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.904+0000 7facd92c3640 1 -- 172.21.15.55:0/1534780295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd406cb20 msgr2=0x7facd40cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.904+0000 7facd92c3640 1 --2- 172.21.15.55:0/1534780295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd406cb20 0x7facd40cfc90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7facd408d580 tx=0x7faccc02a1e0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.904+0000 7facd92c3640 1 -- 172.21.15.55:0/1534780295 shutdown_connections 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.904+0000 7facd92c3640 1 --2- 172.21.15.55:0/1534780295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd406cb20 0x7facd40cfc90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.904+0000 7facd92c3640 1 --2- 172.21.15.55:0/1534780295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd40cb8d0 0x7facd406c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.904+0000 7facd92c3640 1 --2- 172.21.15.55:0/1534780295 >> v2:172.21.15.138:3300/0 conn(0x7facd4162e80 0x7facd415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.904+0000 7facd92c3640 1 -- 172.21.15.55:0/1534780295 >> 172.21.15.55:0/1534780295 conn(0x7facd40d1f40 msgr2=0x7facd40d4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.904+0000 7facd92c3640 1 -- 172.21.15.55:0/1534780295 shutdown_connections 2024-05-20T22:11:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.904+0000 7facd92c3640 1 -- 172.21.15.55:0/1534780295 wait complete. 2024-05-20T22:11:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.906+0000 7facd92c3640 1 Processor -- start 2024-05-20T22:11:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.906+0000 7facd92c3640 1 -- start start 2024-05-20T22:11:09.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.907+0000 7facd92c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd406cb20 0x7facd41622a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.907+0000 7facd92c3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd40cb8d0 0x7facd414f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.907+0000 7facd92c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd4162e80 0x7facd414fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.907+0000 7facd92c3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7facd415f700 con 0x7facd40cb8d0 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.907+0000 7facd92c3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facd415f580 con 0x7facd4162e80 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.907+0000 7facd92c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7facd40baa20 con 0x7facd406cb20 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd40cb8d0 0x7facd414f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd40cb8d0 0x7facd414f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35338/0 (socket says 172.21.15.55:35338) 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd27fc640 1 -- 172.21.15.55:0/3845826459 learned_addr learned my addr 172.21.15.55:0/3845826459 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd2ffd640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd406cb20 0x7facd41622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd37fe640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd4162e80 0x7facd414fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd27fc640 1 -- 172.21.15.55:0/3845826459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd406cb20 msgr2=0x7facd41622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd27fc640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd406cb20 0x7facd41622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd27fc640 1 -- 172.21.15.55:0/3845826459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd4162e80 msgr2=0x7facd414fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd27fc640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd4162e80 0x7facd414fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd27fc640 1 -- 172.21.15.55:0/3845826459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd4161310 con 0x7facd40cb8d0 2024-05-20T22:11:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd27fc640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd40cb8d0 0x7facd414f9a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7facc800b440 tx=0x7facc800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facbb7fe640 1 -- 172.21.15.55:0/3845826459 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8015020 con 0x7facd40cb8d0 2024-05-20T22:11:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facbb7fe640 1 -- 172.21.15.55:0/3845826459 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc8011890 con 0x7facd40cb8d0 2024-05-20T22:11:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd92c3640 1 -- 172.21.15.55:0/3845826459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd41615f0 con 0x7facd40cb8d0 2024-05-20T22:11:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facbb7fe640 1 -- 172.21.15.55:0/3845826459 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8011c80 con 0x7facd40cb8d0 2024-05-20T22:11:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.908+0000 7facd92c3640 1 -- 172.21.15.55:0/3845826459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd40cc210 con 0x7facd40cb8d0 2024-05-20T22:11:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.909+0000 7facd92c3640 1 -- 172.21.15.55:0/3845826459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd408b990 con 0x7facd40cb8d0 2024-05-20T22:11:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.910+0000 7facbb7fe640 1 -- 172.21.15.55:0/3845826459 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7facc8018020 con 0x7facd40cb8d0 2024-05-20T22:11:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.911+0000 7facbb7fe640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7facc006be60 0x7facc006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.911+0000 7facbb7fe640 1 -- 172.21.15.55:0/3845826459 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5893+0+0 (secure 0 0 0) 0x7facc808afe0 con 0x7facd40cb8d0 2024-05-20T22:11:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.911+0000 7facd2ffd640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7facc006be60 0x7facc006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.911+0000 7facd2ffd640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7facc006be60 0x7facc006e370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7facc400f500 tx=0x7facc4007770 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:09.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:09.915+0000 7facbb7fe640 1 -- 172.21.15.55:0/3845826459 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7facc801f020 con 0x7facd40cb8d0 2024-05-20T22:11:10.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.106+0000 7facd92c3640 1 -- 172.21.15.55:0/3845826459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7facd404f800 con 0x7facd40cb8d0 2024-05-20T22:11:10.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.764+0000 7facbb7fe640 1 -- 172.21.15.55:0/3845826459 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) ==== 169+0+0 (secure 0 0 0) 0x7facc800d040 con 0x7facd40cb8d0 2024-05-20T22:11:10.765 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-05-20T22:11:10.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.766+0000 7facd92c3640 1 -- 172.21.15.55:0/3845826459 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7facc006be60 msgr2=0x7facc006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:10.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.766+0000 7facd92c3640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7facc006be60 0x7facc006e370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7facc400f500 tx=0x7facc4007770 comp rx=0 tx=0).stop 2024-05-20T22:11:10.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.766+0000 7facd92c3640 1 -- 172.21.15.55:0/3845826459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd40cb8d0 msgr2=0x7facd414f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:10.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.766+0000 7facd92c3640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd40cb8d0 0x7facd414f9a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7facc800b440 tx=0x7facc800b900 comp rx=0 tx=0).stop 2024-05-20T22:11:10.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.766+0000 7facd92c3640 1 -- 172.21.15.55:0/3845826459 shutdown_connections 2024-05-20T22:11:10.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.766+0000 7facd92c3640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7facc006be60 0x7facc006e370 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:10.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.766+0000 7facd92c3640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facd4162e80 0x7facd414fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:10.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.766+0000 7facd92c3640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facd40cb8d0 0x7facd414f9a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:10.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.766+0000 7facd92c3640 1 --2- 172.21.15.55:0/3845826459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7facd406cb20 0x7facd41622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:10.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.766+0000 7facd92c3640 1 -- 172.21.15.55:0/3845826459 >> 172.21.15.55:0/3845826459 conn(0x7facd40d1f40 msgr2=0x7facd40b9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:10.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.767+0000 7facd92c3640 1 -- 172.21.15.55:0/3845826459 shutdown_connections 2024-05-20T22:11:10.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:10.767+0000 7facd92c3640 1 -- 172.21.15.55:0/3845826459 wait complete. 2024-05-20T22:11:10.785 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T22:11:10.788 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T22:11:10.834 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2024-05-20T22:11:10.834 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2024-05-20T22:11:10.834 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T22:11:10.834 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2024-05-20T22:11:10.834 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.834 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2024-05-20T22:11:10.834 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.834 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.835 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: mon_allow_pool_size_one = true 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.836 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2024-05-20T22:11:10.837 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: osd mclock profile = high_recovery_ops 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug trim objects = true 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2024-05-20T22:11:10.838 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2024-05-20T22:11:10.839 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2024-05-20T22:11:10.840 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-20T22:11:10.841 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.842 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: # 1m isn't quite enough 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: mon_down_mkfs_grace = 2m 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_filestore_osds = false 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: mon down mkfs grace = 300 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2024-05-20T22:11:10.843 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = "" 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2024-05-20T22:11:10.844 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds balancer = 20 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-20T22:11:10.845 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-05-20T22:11:10.846 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-05-20T22:11:10.847 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.848 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.55,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = de8ca569-3fd1-4cb2-8bdc-99f367745927 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-05-20T22:11:10.849 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-05-20T22:11:10.850 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-05-20T22:11:10.851 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.852 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-20T22:11:10.853 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T22:11:10.854 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: mon down mkfs grace = 300 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-05-20T22:11:10.855 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-05-20T22:11:10.856 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-20T22:11:10.857 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-20T22:11:10.859 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-20T22:11:11.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.016+0000 7f2ab3cd7640 1 Processor -- start 2024-05-20T22:11:11.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.016+0000 7f2ab3cd7640 1 -- start start 2024-05-20T22:11:11.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.018+0000 7f2ab3cd7640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2aac0d4210 0x7f2aac0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.018+0000 7f2ab3cd7640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aac163980 con 0x7f2aac0d0350 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.018+0000 7f2ab3cd7640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aac0d3450 con 0x7f2aac0cfec0 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.018+0000 7f2ab3cd7640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2aac0d3d80 con 0x7f2aac0d4210 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.018+0000 7f2ab1272640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2aac0d0350 0x7f2aac0cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44278/0 (socket says 172.21.15.55:44278) 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.018+0000 7f2ab1272640 1 -- 172.21.15.55:0/1959960423 learned_addr learned my addr 172.21.15.55:0/1959960423 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.018+0000 7f2ab1a73640 1 --2- 172.21.15.55:0/1959960423 >> v2:172.21.15.138:3300/0 conn(0x7f2aac0d4210 0x7f2aac0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.018+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 541178005 0 0) 0x7f2aac163980 con 0x7f2aac0d0350 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.018+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a94003520 con 0x7f2aac0d0350 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 806446881 0 0) 0x7f2a94003520 con 0x7f2aac0d0350 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 >> v2:172.21.15.138:3300/0 conn(0x7f2aac0d4210 msgr2=0x7f2aac0b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 --2- 172.21.15.55:0/1959960423 >> v2:172.21.15.138:3300/0 conn(0x7f2aac0d4210 0x7f2aac0b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 >> v1:172.21.15.138:6789/0 conn(0x7f2aac0cfec0 legacy=0x7f2aac15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac0d3730 con 0x7f2aac0d0350 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f2aa0001670 con 0x7f2aac0d0350 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 >> v1:172.21.15.55:6789/0 conn(0x7f2aac0d0350 legacy=0x7f2aac0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940039b0 0x7f2a94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a94004800 0x7f2a94004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a940051c0 0x7f2a9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac0d3d80 con 0x7f2a940039b0 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9400c160 con 0x7f2a94004800 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9400c360 con 0x7f2a940051c0 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab1a73640 1 --2- 172.21.15.55:0/1959960423 >> v2:172.21.15.138:3300/0 conn(0x7f2aac0d4210 0x7f2aac0b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab1a73640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940039b0 0x7f2a94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab1272640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a94004800 0x7f2a94004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab2274640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a940051c0 0x7f2a9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab1a73640 1 -- 172.21.15.55:0/1959960423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a940051c0 msgr2=0x7f2a9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:11.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab1a73640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a940051c0 0x7f2a9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab1a73640 1 -- 172.21.15.55:0/1959960423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a94004800 msgr2=0x7f2a94004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:11.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab1a73640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a94004800 0x7f2a94004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab1a73640 1 -- 172.21.15.55:0/1959960423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a94003520 con 0x7f2a940039b0 2024-05-20T22:11:11.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.019+0000 7f2ab2274640 1 -- 172.21.15.55:0/1959960423 reap_dead start 2024-05-20T22:11:11.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.020+0000 7f2ab1a73640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940039b0 0x7f2a94003e20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0c8210 tx=0x7f2a94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec079a5a1e6f36f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:11.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.020+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4036070 con 0x7f2a940039b0 2024-05-20T22:11:11.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.020+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa4031400 con 0x7f2a940039b0 2024-05-20T22:11:11.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.020+0000 7f2ab0a71640 1 -- 172.21.15.55:0/1959960423 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4009860 con 0x7f2a940039b0 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.021+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/1959960423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940039b0 msgr2=0x7f2a94003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.021+0000 7f2ab3cd7640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940039b0 0x7f2a94003e20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0c8210 tx=0x7f2a94005830 comp rx=0 tx=0).stop 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.021+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/1959960423 shutdown_connections 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.021+0000 7f2ab3cd7640 1 --2- 172.21.15.55:0/1959960423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940039b0 0x7f2a94003e20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.021+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/1959960423 >> 172.21.15.55:0/1959960423 conn(0x7f2aac0d1bb0 msgr2=0x7f2aac0d4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.021+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/1959960423 shutdown_connections 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.021+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/1959960423 wait complete. 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.023+0000 7f2ab3cd7640 1 Processor -- start 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.023+0000 7f2ab3cd7640 1 -- start start 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.024+0000 7f2ab3cd7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a940039b0 0x7f2aac0c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.024+0000 7f2ab3cd7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aac14c630 0x7f2aac14ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab3cd7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac1f86e0 0x7f2aac1f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab3cd7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac163720 con 0x7f2aac1f86e0 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab3cd7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac1635a0 con 0x7f2a940039b0 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab3cd7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2aac0b3b20 con 0x7f2aac14c630 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab2274640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac1f86e0 0x7f2aac1f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab2274640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac1f86e0 0x7f2aac1f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35358/0 (socket says 172.21.15.55:35358) 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab2274640 1 -- 172.21.15.55:0/712866011 learned_addr learned my addr 172.21.15.55:0/712866011 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab1a73640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a940039b0 0x7f2aac0c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab1272640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aac14c630 0x7f2aac14ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab2274640 1 -- 172.21.15.55:0/712866011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aac14c630 msgr2=0x7f2aac14ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab2274640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aac14c630 0x7f2aac14ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab2274640 1 -- 172.21.15.55:0/712866011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a940039b0 msgr2=0x7f2aac0c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab2274640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a940039b0 0x7f2aac0c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab2274640 1 -- 172.21.15.55:0/712866011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac1f9030 con 0x7f2aac1f86e0 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab1a73640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a940039b0 0x7f2aac0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab1272640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aac14c630 0x7f2aac14ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab2274640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac1f86e0 0x7f2aac1f8af0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0b44f0 tx=0x7f2a9c00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2aaa7fc640 1 -- 172.21.15.55:0/712866011 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c003b80 con 0x7f2aac1f86e0 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2aaa7fc640 1 -- 172.21.15.55:0/712866011 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a9c018070 con 0x7f2aac1f86e0 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/712866011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aac0ca120 con 0x7f2aac1f86e0 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2aaa7fc640 1 -- 172.21.15.55:0/712866011 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0050d0 con 0x7f2aac1f86e0 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.025+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/712866011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aac0ca6e0 con 0x7f2aac1f86e0 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.027+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/712866011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aac069090 con 0x7f2aac1f86e0 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.027+0000 7f2aaa7fc640 1 -- 172.21.15.55:0/712866011 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a9c005270 con 0x7f2aac1f86e0 2024-05-20T22:11:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.028+0000 7f2aaa7fc640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2a9806be30 0x7f2a9806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.028+0000 7f2aaa7fc640 1 -- 172.21.15.55:0/712866011 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f2a9c08ab90 con 0x7f2aac1f86e0 2024-05-20T22:11:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.028+0000 7f2ab1a73640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2a9806be30 0x7f2a9806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.028+0000 7f2ab1a73640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2a9806be30 0x7f2a9806e340 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4033000 tx=0x7f2aa4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:11.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.032+0000 7f2aaa7fc640 1 -- 172.21.15.55:0/712866011 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2a9c059660 con 0x7f2aac1f86e0 2024-05-20T22:11:11.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.216+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/712866011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f2aac0c9eb0 con 0x7f2aac1f86e0 2024-05-20T22:11:11.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.219+0000 7f2aaa7fc640 1 -- 172.21.15.55:0/712866011 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f2a9c058f60 con 0x7f2aac1f86e0 2024-05-20T22:11:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.221+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/712866011 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2a9806be30 msgr2=0x7f2a9806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.221+0000 7f2ab3cd7640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2a9806be30 0x7f2a9806e340 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4033000 tx=0x7f2aa4008030 comp rx=0 tx=0).stop 2024-05-20T22:11:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.221+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/712866011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac1f86e0 msgr2=0x7f2aac1f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.221+0000 7f2ab3cd7640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac1f86e0 0x7f2aac1f8af0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0b44f0 tx=0x7f2a9c00fd40 comp rx=0 tx=0).stop 2024-05-20T22:11:11.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.221+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/712866011 shutdown_connections 2024-05-20T22:11:11.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.221+0000 7f2ab3cd7640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2a9806be30 0x7f2a9806e340 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.221+0000 7f2ab3cd7640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac1f86e0 0x7f2aac1f8af0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.221+0000 7f2ab3cd7640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aac14c630 0x7f2aac14ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.221+0000 7f2ab3cd7640 1 --2- 172.21.15.55:0/712866011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a940039b0 0x7f2aac0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.221+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/712866011 >> 172.21.15.55:0/712866011 conn(0x7f2aac0d1bb0 msgr2=0x7f2aac0b8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:11.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.222+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/712866011 shutdown_connections 2024-05-20T22:11:11.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.222+0000 7f2ab3cd7640 1 -- 172.21.15.55:0/712866011 wait complete. 2024-05-20T22:11:11.234 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters' 2024-05-20T22:11:11.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.348+0000 7f79ef196640 1 Processor -- start 2024-05-20T22:11:11.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.348+0000 7f79ef196640 1 -- start start 2024-05-20T22:11:11.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.349+0000 7f79ef196640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e80d1f40 0x7f79e80ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.349+0000 7f79ef196640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79e806dfd0 0x7f79e80ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.349+0000 7f79ef196640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e80b2720 con 0x7f79e806e520 2024-05-20T22:11:11.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.349+0000 7f79ef196640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f79e806bd90 con 0x7f79e806dfd0 2024-05-20T22:11:11.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.349+0000 7f79ef196640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f79e80cd330 con 0x7f79e80d1f40 2024-05-20T22:11:11.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79e7fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e80d1f40 0x7f79e80ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79ed733640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79e806dfd0 0x7f79e80ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79ecf32640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f79e806e520 0x7f79e80d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.55:35082/0 (socket says 172.21.15.55:35082) 2024-05-20T22:11:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79ecf32640 1 -- 172.21.15.55:0/3472975095 learned_addr learned my addr 172.21.15.55:0/3472975095 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79e77fe640 1 -- 172.21.15.55:0/3472975095 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930372945 0 0) 0x7f79e80b2720 con 0x7f79e806e520 2024-05-20T22:11:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79e77fe640 1 -- 172.21.15.55:0/3472975095 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79d4003520 con 0x7f79e806e520 2024-05-20T22:11:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79e7fff640 1 -- 172.21.15.55:0/3472975095 >> v2:172.21.15.138:3300/0 conn(0x7f79e806dfd0 msgr2=0x7f79e80ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:11.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79e7fff640 1 --2- 172.21.15.55:0/3472975095 >> v2:172.21.15.138:3300/0 conn(0x7f79e806dfd0 0x7f79e80ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79e7fff640 1 -- 172.21.15.55:0/3472975095 >> v1:172.21.15.138:6789/0 conn(0x7f79e806e520 legacy=0x7f79e80d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79e7fff640 1 -- 172.21.15.55:0/3472975095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e80cd560 con 0x7f79e80d1f40 2024-05-20T22:11:11.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.350+0000 7f79ed733640 1 --2- 172.21.15.55:0/3472975095 >> v2:172.21.15.138:3300/0 conn(0x7f79e806dfd0 0x7f79e80ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:11:11.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.351+0000 7f79e7fff640 1 --2- 172.21.15.55:0/3472975095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e80d1f40 0x7f79e80ce000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79d80097a0 tx=0x7f79d802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a79be999bb9dcbfc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:11.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.351+0000 7f79e77fe640 1 -- 172.21.15.55:0/3472975095 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8037070 con 0x7f79e80d1f40 2024-05-20T22:11:11.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.351+0000 7f79e77fe640 1 -- 172.21.15.55:0/3472975095 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d8032440 con 0x7f79e80d1f40 2024-05-20T22:11:11.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.351+0000 7f79e77fe640 1 -- 172.21.15.55:0/3472975095 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8032810 con 0x7f79e80d1f40 2024-05-20T22:11:11.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.352+0000 7f79ef196640 1 -- 172.21.15.55:0/3472975095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e80d1f40 msgr2=0x7f79e80ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.352+0000 7f79ef196640 1 --2- 172.21.15.55:0/3472975095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e80d1f40 0x7f79e80ce000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79d80097a0 tx=0x7f79d802a180 comp rx=0 tx=0).stop 2024-05-20T22:11:11.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.352+0000 7f79ef196640 1 -- 172.21.15.55:0/3472975095 shutdown_connections 2024-05-20T22:11:11.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.352+0000 7f79ef196640 1 --2- 172.21.15.55:0/3472975095 >> v2:172.21.15.138:3300/0 conn(0x7f79e806dfd0 0x7f79e80ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.352+0000 7f79ef196640 1 --2- 172.21.15.55:0/3472975095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e80d1f40 0x7f79e80ce000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.352+0000 7f79ef196640 1 -- 172.21.15.55:0/3472975095 >> 172.21.15.55:0/3472975095 conn(0x7f79e80b6580 msgr2=0x7f79e80b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:11.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.352+0000 7f79ef196640 1 -- 172.21.15.55:0/3472975095 shutdown_connections 2024-05-20T22:11:11.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.352+0000 7f79ef196640 1 -- 172.21.15.55:0/3472975095 wait complete. 2024-05-20T22:11:11.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.354+0000 7f79ef196640 1 Processor -- start 2024-05-20T22:11:11.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.354+0000 7f79ef196640 1 -- start start 2024-05-20T22:11:11.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79ef196640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e806dfd0 0x7f79e81f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79ef196640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e806e520 0x7f79e8152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79ef196640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e80c9f40 0x7f79e80ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79ef196640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f79e80b2160 con 0x7f79e80c9f40 2024-05-20T22:11:11.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79ef196640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79e80cd800 con 0x7f79e806e520 2024-05-20T22:11:11.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79ef196640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f79e815d4f0 con 0x7f79e806dfd0 2024-05-20T22:11:11.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79ed733640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e80c9f40 0x7f79e80ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79ed733640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e80c9f40 0x7f79e80ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35372/0 (socket says 172.21.15.55:35372) 2024-05-20T22:11:11.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79ed733640 1 -- 172.21.15.55:0/2933430814 learned_addr learned my addr 172.21.15.55:0/2933430814 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:11.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79e7fff640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e806e520 0x7f79e8152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.355+0000 7f79ecf32640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e806dfd0 0x7f79e81f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79ed733640 1 -- 172.21.15.55:0/2933430814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e806dfd0 msgr2=0x7f79e81f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79ed733640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e806dfd0 0x7f79e81f7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79ed733640 1 -- 172.21.15.55:0/2933430814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e806e520 msgr2=0x7f79e8152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79ed733640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e806e520 0x7f79e8152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79ed733640 1 -- 172.21.15.55:0/2933430814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e815c260 con 0x7f79e80c9f40 2024-05-20T22:11:11.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79ed733640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e80c9f40 0x7f79e80ca350 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f79e000efb0 tx=0x7f79e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:11.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79e57fa640 1 -- 172.21.15.55:0/2933430814 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0018020 con 0x7f79e80c9f40 2024-05-20T22:11:11.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79e57fa640 1 -- 172.21.15.55:0/2933430814 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e0014480 con 0x7f79e80c9f40 2024-05-20T22:11:11.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79ef196640 1 -- 172.21.15.55:0/2933430814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e80ca890 con 0x7f79e80c9f40 2024-05-20T22:11:11.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79e57fa640 1 -- 172.21.15.55:0/2933430814 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0014850 con 0x7f79e80c9f40 2024-05-20T22:11:11.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.356+0000 7f79ef196640 1 -- 172.21.15.55:0/2933430814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e809c600 con 0x7f79e80c9f40 2024-05-20T22:11:11.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.357+0000 7f79ef196640 1 -- 172.21.15.55:0/2933430814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e8069090 con 0x7f79e80c9f40 2024-05-20T22:11:11.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.358+0000 7f79e57fa640 1 -- 172.21.15.55:0/2933430814 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79e0005d10 con 0x7f79e80c9f40 2024-05-20T22:11:11.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.358+0000 7f79e57fa640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79c806be60 0x7f79c806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:11.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.358+0000 7f79ecf32640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79c806be60 0x7f79c806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:11.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.358+0000 7f79e57fa640 1 -- 172.21.15.55:0/2933430814 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f79e008bb00 con 0x7f79e80c9f40 2024-05-20T22:11:11.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.359+0000 7f79ecf32640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79c806be60 0x7f79c806e370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f79d00011e0 tx=0x7f79d0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:11.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.363+0000 7f79e57fa640 1 -- 172.21.15.55:0/2933430814 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f79e0022030 con 0x7f79e80c9f40 2024-05-20T22:11:11.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.625+0000 7f79ef196640 1 -- 172.21.15.55:0/2933430814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]} v 0) -- 0x7f79e80cee40 con 0x7f79e80c9f40 2024-05-20T22:11:11.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.765+0000 7f79e57fa640 1 -- 172.21.15.55:0/2933430814 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]}]=0 v190) ==== 143+0+0 (secure 0 0 0) 0x7f79e80ca890 con 0x7f79e80c9f40 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.767+0000 7f79ef196640 1 -- 172.21.15.55:0/2933430814 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79c806be60 msgr2=0x7f79c806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.767+0000 7f79ef196640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79c806be60 0x7f79c806e370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f79d00011e0 tx=0x7f79d0009040 comp rx=0 tx=0).stop 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.767+0000 7f79ef196640 1 -- 172.21.15.55:0/2933430814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e80c9f40 msgr2=0x7f79e80ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.767+0000 7f79ef196640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e80c9f40 0x7f79e80ca350 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f79e000efb0 tx=0x7f79e000c490 comp rx=0 tx=0).stop 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.768+0000 7f79ef196640 1 -- 172.21.15.55:0/2933430814 shutdown_connections 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.768+0000 7f79ef196640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f79c806be60 0x7f79c806e370 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.768+0000 7f79ef196640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79e80c9f40 0x7f79e80ca350 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.768+0000 7f79ef196640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e806e520 0x7f79e8152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.768+0000 7f79ef196640 1 --2- 172.21.15.55:0/2933430814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79e806dfd0 0x7f79e81f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.768+0000 7f79ef196640 1 -- 172.21.15.55:0/2933430814 >> 172.21.15.55:0/2933430814 conn(0x7f79e80b6580 msgr2=0x7f79e8070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.768+0000 7f79ef196640 1 -- 172.21.15.55:0/2933430814 shutdown_connections 2024-05-20T22:11:11.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:11.768+0000 7f79ef196640 1 -- 172.21.15.55:0/2933430814 wait complete. 2024-05-20T22:11:11.781 INFO:tasks.cephfs_test_runner:Opening a file should increment omap_total_updates by 1. ... ok 2024-05-20T22:11:11.781 INFO:tasks.cephfs_test_runner: 2024-05-20T22:11:11.781 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-20T22:11:11.781 INFO:tasks.cephfs_test_runner:Ran 2 tests in 152.783s 2024-05-20T22:11:11.781 INFO:tasks.cephfs_test_runner: 2024-05-20T22:11:11.781 INFO:tasks.cephfs_test_runner:OK 2024-05-20T22:11:11.782 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-20T22:11:11.790 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-20T22:11:11.819 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-20T22:11:11.819 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:11:11.819 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:11:11.836 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3618 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T22:11:11.836 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-20T22:11:11.836 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:11:11.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3618 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T22:11:11.891 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:11:11.892 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:11:11.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=450 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T22:11:11.911 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-20T22:11:11.911 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T22:11:11.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=450 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T22:11:11.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:11.971 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-20T22:11:12.029 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-20T22:11:12.029 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-20T22:11:12.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:12.029 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-20T22:11:12.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:12.098 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T22:11:12.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:12.167 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-05-20T22:11:12.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:12.261 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-20T22:11:12.278 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T22:11:12.278 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T22:11:12.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:12.278 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T22:11:12.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:12.347 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T22:11:12.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T22:11:12.415 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2024-05-20T22:11:12.505 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-20T22:11:12.514 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-20T22:11:12.514 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:11:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.630+0000 7f0fe9a79640 1 Processor -- start 2024-05-20T22:11:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.630+0000 7f0fe9a79640 1 -- start start 2024-05-20T22:11:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.631+0000 7f0fe9a79640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe40af980 0x7f0fe40cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.631+0000 7f0fe9a79640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0fe41611b0 0x7f0fe406bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.631+0000 7f0fe9a79640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fe40cbc60 con 0x7f0fe40cfda0 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.631+0000 7f0fe9a79640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f0fe40cbf40 con 0x7f0fe41611b0 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.631+0000 7f0fe9a79640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe40cc220 con 0x7f0fe40af980 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.631+0000 7f0fe3fff640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0fe41611b0 0x7f0fe406bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.631+0000 7f0fe2ffd640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0fe40cfda0 0x7f0fe4160aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44280/0 (socket says 172.21.15.55:44280) 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.631+0000 7f0fe2ffd640 1 -- 172.21.15.55:0/1945337780 learned_addr learned my addr 172.21.15.55:0/1945337780 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.632+0000 7f0fe37fe640 1 --2- 172.21.15.55:0/1945337780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe40af980 0x7f0fe40cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.632+0000 7f0fe3fff640 1 -- 172.21.15.55:0/1945337780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe40af980 msgr2=0x7f0fe40cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.632+0000 7f0fe3fff640 1 --2- 172.21.15.55:0/1945337780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe40af980 0x7f0fe40cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.632+0000 7f0fe3fff640 1 -- 172.21.15.55:0/1945337780 >> v1:172.21.15.55:6789/0 conn(0x7f0fe40cfda0 legacy=0x7f0fe4160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.632+0000 7f0fe3fff640 1 -- 172.21.15.55:0/1945337780 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe40cc500 con 0x7f0fe41611b0 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.632+0000 7f0fe27fc640 1 -- 172.21.15.55:0/1945337780 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147180431 0 0) 0x7f0fe40cbc60 con 0x7f0fe40cfda0 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.632+0000 7f0fe37fe640 1 --2- 172.21.15.55:0/1945337780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe40af980 0x7f0fe40cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.632+0000 7f0fe3fff640 1 --2- 172.21.15.55:0/1945337780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe41611b0 0x7f0fe406bac0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009d90 tx=0x7f0fdc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c169ace8fc2c66e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe27fc640 1 -- 172.21.15.55:0/1945337780 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc031600 con 0x7f0fe41611b0 2024-05-20T22:11:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe27fc640 1 -- 172.21.15.55:0/1945337780 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fdc0317a0 con 0x7f0fe41611b0 2024-05-20T22:11:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe27fc640 1 -- 172.21.15.55:0/1945337780 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc031b70 con 0x7f0fe41611b0 2024-05-20T22:11:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe9a79640 1 -- 172.21.15.55:0/1945337780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe41611b0 msgr2=0x7f0fe406bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe9a79640 1 --2- 172.21.15.55:0/1945337780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe41611b0 0x7f0fe406bac0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009d90 tx=0x7f0fdc02a5d0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe9a79640 1 -- 172.21.15.55:0/1945337780 shutdown_connections 2024-05-20T22:11:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe9a79640 1 --2- 172.21.15.55:0/1945337780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe41611b0 0x7f0fe406bac0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe9a79640 1 --2- 172.21.15.55:0/1945337780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe40af980 0x7f0fe40cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe9a79640 1 -- 172.21.15.55:0/1945337780 >> 172.21.15.55:0/1945337780 conn(0x7f0fe40cd4d0 msgr2=0x7f0fe40cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe9a79640 1 -- 172.21.15.55:0/1945337780 shutdown_connections 2024-05-20T22:11:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.633+0000 7f0fe9a79640 1 -- 172.21.15.55:0/1945337780 wait complete. 2024-05-20T22:11:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.635+0000 7f0fe9a79640 1 Processor -- start 2024-05-20T22:11:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.635+0000 7f0fe9a79640 1 -- start start 2024-05-20T22:11:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe9a79640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe40af980 0x7f0fe409db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe9a79640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe40cfda0 0x7f0fe406fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe9a79640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe409b8e0 0x7f0fe40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe9a79640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe406c180 con 0x7f0fe40af980 2024-05-20T22:11:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe9a79640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe406c000 con 0x7f0fe40cfda0 2024-05-20T22:11:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe9a79640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe406c300 con 0x7f0fe409b8e0 2024-05-20T22:11:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe40af980 0x7f0fe409db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe40af980 0x7f0fe409db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35398/0 (socket says 172.21.15.55:35398) 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe37fe640 1 -- 172.21.15.55:0/3176165789 learned_addr learned my addr 172.21.15.55:0/3176165789 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe3fff640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe409b8e0 0x7f0fe40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe2ffd640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe40cfda0 0x7f0fe406fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe37fe640 1 -- 172.21.15.55:0/3176165789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe409b8e0 msgr2=0x7f0fe40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe37fe640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe409b8e0 0x7f0fe40c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe37fe640 1 -- 172.21.15.55:0/3176165789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe40cfda0 msgr2=0x7f0fe406fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe37fe640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe40cfda0 0x7f0fe406fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe37fe640 1 -- 172.21.15.55:0/3176165789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe409e060 con 0x7f0fe40af980 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe3fff640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe409b8e0 0x7f0fe40c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe2ffd640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe40cfda0 0x7f0fe406fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.637+0000 7f0fe37fe640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe40af980 0x7f0fe409db20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400d800 tx=0x7f0fd400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.638+0000 7f0fcbfff640 1 -- 172.21.15.55:0/3176165789 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4009640 con 0x7f0fe40af980 2024-05-20T22:11:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.638+0000 7f0fcbfff640 1 -- 172.21.15.55:0/3176165789 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd4019070 con 0x7f0fe40af980 2024-05-20T22:11:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.638+0000 7f0fcbfff640 1 -- 172.21.15.55:0/3176165789 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4009a20 con 0x7f0fe40af980 2024-05-20T22:11:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.638+0000 7f0fe9a79640 1 -- 172.21.15.55:0/3176165789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe40701f0 con 0x7f0fe40af980 2024-05-20T22:11:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.638+0000 7f0fe9a79640 1 -- 172.21.15.55:0/3176165789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe409ab20 con 0x7f0fe40af980 2024-05-20T22:11:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.639+0000 7f0fe9a79640 1 -- 172.21.15.55:0/3176165789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe4069090 con 0x7f0fe40af980 2024-05-20T22:11:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.639+0000 7f0fcbfff640 1 -- 172.21.15.55:0/3176165789 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fd4009bc0 con 0x7f0fe40af980 2024-05-20T22:11:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.640+0000 7f0fcbfff640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0fd006be60 0x7f0fd006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.640+0000 7f0fcbfff640 1 -- 172.21.15.55:0/3176165789 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f0fd408b0d0 con 0x7f0fe40af980 2024-05-20T22:11:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.640+0000 7f0fe2ffd640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0fd006be60 0x7f0fd006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.641+0000 7f0fe2ffd640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0fd006be60 0x7f0fd006e370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40b0ec0 tx=0x7f0fd8007cc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.645+0000 7f0fcbfff640 1 -- 172.21.15.55:0/3176165789 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0fd4059b20 con 0x7f0fe40af980 2024-05-20T22:11:12.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.826+0000 7f0fe9a79640 1 -- 172.21.15.55:0/3176165789 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0fe40ceed0 con 0x7f0fd006be60 2024-05-20T22:11:12.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.827+0000 7f0fcbfff640 1 -- 172.21.15.55:0/3176165789 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13482 (secure 0 0 0) 0x7f0fe40ceed0 con 0x7f0fd006be60 2024-05-20T22:11:12.829 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:12.829 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:11:12.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 -- 172.21.15.55:0/3176165789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0fd006be60 msgr2=0x7f0fd006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:12.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0fd006be60 0x7f0fd006e370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40b0ec0 tx=0x7f0fd8007cc0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 -- 172.21.15.55:0/3176165789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe40af980 msgr2=0x7f0fe409db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:12.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe40af980 0x7f0fe409db20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400d800 tx=0x7f0fd400dcc0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 -- 172.21.15.55:0/3176165789 shutdown_connections 2024-05-20T22:11:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f0fd006be60 0x7f0fd006e370 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fe409b8e0 0x7f0fe40c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe40cfda0 0x7f0fe406fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 --2- 172.21.15.55:0/3176165789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fe40af980 0x7f0fe409db20 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 -- 172.21.15.55:0/3176165789 >> 172.21.15.55:0/3176165789 conn(0x7f0fe40cd4d0 msgr2=0x7f0fe40cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 -- 172.21.15.55:0/3176165789 shutdown_connections 2024-05-20T22:11:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.829+0000 7f0fe9a79640 1 -- 172.21.15.55:0/3176165789 wait complete. 2024-05-20T22:11:12.841 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-05-20T22:11:12.374154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":244048,"kb_used_data":7088,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730672,"statfs":{"total":773094113280,"available":772844208128,"internally_reserved":0,"allocated":7258112,"data_stored":3961998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7353,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-72,"num_write_kb":-30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001236"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":259,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-20T22:11:10.767738+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:11:10.767738+0000","last_peered":"2024-05-20T22:11:10.767738+0000","last_clean":"2024-05-20T22:11:10.767738+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:11:10.767738+0000","last_undegraded":"2024-05-20T22:11:10.767738+0000","last_fullsized":"2024-05-20T22:11:10.767738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":938024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":347663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31920,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":347663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":938024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:11:12.841 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:11:12.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.956+0000 7f4e9b88d640 1 Processor -- start 2024-05-20T22:11:12.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.956+0000 7f4e9b88d640 1 -- start start 2024-05-20T22:11:12.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.957+0000 7f4e9b88d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4e94168750 0x7f4e9416ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.957+0000 7f4e9b88d640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e94164330 con 0x7f4e9406d8f0 2024-05-20T22:11:12.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.957+0000 7f4e9b88d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e9406e370 con 0x7f4e9406dd00 2024-05-20T22:11:12.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.957+0000 7f4e9b88d640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f4e9406e110 con 0x7f4e94168750 2024-05-20T22:11:12.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.958+0000 7f4e99629640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4e9406d8f0 0x7f4e94163510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44296/0 (socket says 172.21.15.55:44296) 2024-05-20T22:11:12.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.958+0000 7f4e99e2a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4e94168750 0x7f4e9416ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.958+0000 7f4e99629640 1 -- 172.21.15.55:0/1727396735 learned_addr learned my addr 172.21.15.55:0/1727396735 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:12.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.958+0000 7f4e99e2a640 1 -- 172.21.15.55:0/1727396735 >> v1:172.21.15.138:6789/0 conn(0x7f4e9406dd00 legacy=0x7f4e94163c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:12.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.958+0000 7f4e99e2a640 1 -- 172.21.15.55:0/1727396735 >> v1:172.21.15.55:6789/0 conn(0x7f4e9406d8f0 legacy=0x7f4e94163510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:12.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.958+0000 7f4e8bfff640 1 -- 172.21.15.55:0/1727396735 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1339046284 0 0) 0x7f4e94164330 con 0x7f4e9406d8f0 2024-05-20T22:11:12.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.958+0000 7f4e99e2a640 1 -- 172.21.15.55:0/1727396735 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e940d2000 con 0x7f4e94168750 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.959+0000 7f4e99e2a640 1 --2- 172.21.15.55:0/1727396735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 0x7f4e9416ab70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009d90 tx=0x7f4e8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df11bcf98ed80392 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.959+0000 7f4e8bfff640 1 -- 172.21.15.55:0/1727396735 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c007740 con 0x7f4e94168750 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.959+0000 7f4e8bfff640 1 -- 172.21.15.55:0/1727396735 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e8c0078e0 con 0x7f4e94168750 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.959+0000 7f4e8bfff640 1 -- 172.21.15.55:0/1727396735 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c007cb0 con 0x7f4e94168750 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.960+0000 7f4e9b88d640 1 -- 172.21.15.55:0/1727396735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 msgr2=0x7f4e9416ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.960+0000 7f4e9b88d640 1 --2- 172.21.15.55:0/1727396735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 0x7f4e9416ab70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009d90 tx=0x7f4e8c02a6a0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.960+0000 7f4e9b88d640 1 -- 172.21.15.55:0/1727396735 shutdown_connections 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.960+0000 7f4e9b88d640 1 --2- 172.21.15.55:0/1727396735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 0x7f4e9416ab70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.960+0000 7f4e9b88d640 1 -- 172.21.15.55:0/1727396735 >> 172.21.15.55:0/1727396735 conn(0x7f4e940b4340 msgr2=0x7f4e940b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.960+0000 7f4e9b88d640 1 -- 172.21.15.55:0/1727396735 shutdown_connections 2024-05-20T22:11:12.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.960+0000 7f4e9b88d640 1 -- 172.21.15.55:0/1727396735 wait complete. 2024-05-20T22:11:12.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.961+0000 7f4e9b88d640 1 Processor -- start 2024-05-20T22:11:12.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.962+0000 7f4e9b88d640 1 -- start start 2024-05-20T22:11:12.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e9b88d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e9406d8f0 0x7f4e94164520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e9b88d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e9406dd00 0x7f4e9414fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e9b88d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 0x7f4e940b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e9b88d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9409c060 con 0x7f4e94168750 2024-05-20T22:11:12.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e9b88d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e940d1cb0 con 0x7f4e9406dd00 2024-05-20T22:11:12.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e9b88d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9409c1e0 con 0x7f4e9406d8f0 2024-05-20T22:11:12.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e99e2a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 0x7f4e940b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e99e2a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 0x7f4e940b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35424/0 (socket says 172.21.15.55:35424) 2024-05-20T22:11:12.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e99e2a640 1 -- 172.21.15.55:0/2044984709 learned_addr learned my addr 172.21.15.55:0/2044984709 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e99629640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e9406d8f0 0x7f4e94164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e98e28640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e9406dd00 0x7f4e9414fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e99e2a640 1 -- 172.21.15.55:0/2044984709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e9406d8f0 msgr2=0x7f4e94164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e99e2a640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e9406d8f0 0x7f4e94164520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e99e2a640 1 -- 172.21.15.55:0/2044984709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e9406dd00 msgr2=0x7f4e9414fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e99e2a640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e9406dd00 0x7f4e9414fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.963+0000 7f4e99e2a640 1 -- 172.21.15.55:0/2044984709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e94164d90 con 0x7f4e94168750 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.964+0000 7f4e99e2a640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 0x7f4e940b0490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009eb0 tx=0x7f4e8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.964+0000 7f4e89ffb640 1 -- 172.21.15.55:0/2044984709 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c040070 con 0x7f4e94168750 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.964+0000 7f4e89ffb640 1 -- 172.21.15.55:0/2044984709 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e8c031c90 con 0x7f4e94168750 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.964+0000 7f4e89ffb640 1 -- 172.21.15.55:0/2044984709 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c045680 con 0x7f4e94168750 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.964+0000 7f4e9b88d640 1 -- 172.21.15.55:0/2044984709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e94167420 con 0x7f4e94168750 2024-05-20T22:11:12.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.964+0000 7f4e9b88d640 1 -- 172.21.15.55:0/2044984709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e941679e0 con 0x7f4e94168750 2024-05-20T22:11:12.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.965+0000 7f4e9b88d640 1 -- 172.21.15.55:0/2044984709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e94069090 con 0x7f4e94168750 2024-05-20T22:11:12.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.966+0000 7f4e89ffb640 1 -- 172.21.15.55:0/2044984709 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e8c039040 con 0x7f4e94168750 2024-05-20T22:11:12.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.966+0000 7f4e89ffb640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e7806be30 0x7f4e7806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:12.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.966+0000 7f4e89ffb640 1 -- 172.21.15.55:0/2044984709 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f4e8c0ac740 con 0x7f4e94168750 2024-05-20T22:11:12.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.966+0000 7f4e99629640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e7806be30 0x7f4e7806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:12.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.967+0000 7f4e99629640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e7806be30 0x7f4e7806e340 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4e900010e0 tx=0x7f4e90009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:12.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:12.971+0000 7f4e89ffb640 1 -- 172.21.15.55:0/2044984709 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4e8c07b2d0 con 0x7f4e94168750 2024-05-20T22:11:13.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.151+0000 7f4e9b88d640 1 -- 172.21.15.55:0/2044984709 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e940b4960 con 0x7f4e7806be30 2024-05-20T22:11:13.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.152+0000 7f4e89ffb640 1 -- 172.21.15.55:0/2044984709 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13482 (secure 0 0 0) 0x7f4e940b4960 con 0x7f4e7806be30 2024-05-20T22:11:13.153 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:13.153 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:11:13.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 -- 172.21.15.55:0/2044984709 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e7806be30 msgr2=0x7f4e7806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e7806be30 0x7f4e7806e340 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4e900010e0 tx=0x7f4e90009040 comp rx=0 tx=0).stop 2024-05-20T22:11:13.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 -- 172.21.15.55:0/2044984709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 msgr2=0x7f4e940b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 0x7f4e940b0490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009eb0 tx=0x7f4e8c004a40 comp rx=0 tx=0).stop 2024-05-20T22:11:13.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 -- 172.21.15.55:0/2044984709 shutdown_connections 2024-05-20T22:11:13.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f4e7806be30 0x7f4e7806e340 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e94168750 0x7f4e940b0490 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e9406dd00 0x7f4e9414fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 --2- 172.21.15.55:0/2044984709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e9406d8f0 0x7f4e94164520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 -- 172.21.15.55:0/2044984709 >> 172.21.15.55:0/2044984709 conn(0x7f4e940b4340 msgr2=0x7f4e940b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:13.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 -- 172.21.15.55:0/2044984709 shutdown_connections 2024-05-20T22:11:13.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.154+0000 7f4e9b88d640 1 -- 172.21.15.55:0/2044984709 wait complete. 2024-05-20T22:11:13.166 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-05-20T22:11:12.374154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":244048,"kb_used_data":7088,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730672,"statfs":{"total":773094113280,"available":772844208128,"internally_reserved":0,"allocated":7258112,"data_stored":3961998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7353,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-72,"num_write_kb":-30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001236"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":259,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-20T22:11:10.767738+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:11:10.767738+0000","last_peered":"2024-05-20T22:11:10.767738+0000","last_clean":"2024-05-20T22:11:10.767738+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:11:10.767738+0000","last_undegraded":"2024-05-20T22:11:10.767738+0000","last_fullsized":"2024-05-20T22:11:10.767738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":938024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":347663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31920,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":347663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":938024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:11:13.166 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-20T22:11:13.167 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:11:13.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.281+0000 7f889d813640 1 Processor -- start 2024-05-20T22:11:13.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.281+0000 7f889d813640 1 -- start start 2024-05-20T22:11:13.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.282+0000 7f889d813640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889815f100 0x7f88980b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.282+0000 7f889d813640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f889806c790 0x7f88980cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.282+0000 7f889d813640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88980d3dd0 con 0x7f88980b3b30 2024-05-20T22:11:13.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.282+0000 7f889d813640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f88980d40b0 con 0x7f889806c790 2024-05-20T22:11:13.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.282+0000 7f889d813640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f88980d4390 con 0x7f889815f100 2024-05-20T22:11:13.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.283+0000 7f88977fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f889806c790 0x7f88980cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.283+0000 7f88977fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f889806c790 0x7f88980cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35426/0 (socket says 172.21.15.55:35426) 2024-05-20T22:11:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.283+0000 7f88977fe640 1 -- 172.21.15.55:0/1741283772 learned_addr learned my addr 172.21.15.55:0/1741283772 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.283+0000 7f88967fc640 1 --2- 172.21.15.55:0/1741283772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889815f100 0x7f88980b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.283+0000 7f88977fe640 1 -- 172.21.15.55:0/1741283772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889815f100 msgr2=0x7f88980b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.283+0000 7f88977fe640 1 --2- 172.21.15.55:0/1741283772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889815f100 0x7f88980b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.283+0000 7f88977fe640 1 -- 172.21.15.55:0/1741283772 >> v1:172.21.15.138:6789/0 conn(0x7f88980b3b30 legacy=0x7f889806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.283+0000 7f88977fe640 1 -- 172.21.15.55:0/1741283772 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889806b5a0 con 0x7f889806c790 2024-05-20T22:11:13.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.284+0000 7f88977fe640 1 --2- 172.21.15.55:0/1741283772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f889806c790 0x7f88980cf740 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8890009e50 tx=0x7f889002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9efb424c8a7557 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:13.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.284+0000 7f8895ffb640 1 -- 172.21.15.55:0/1741283772 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890036070 con 0x7f889806c790 2024-05-20T22:11:13.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.284+0000 7f8895ffb640 1 -- 172.21.15.55:0/1741283772 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8890031600 con 0x7f889806c790 2024-05-20T22:11:13.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.284+0000 7f8895ffb640 1 -- 172.21.15.55:0/1741283772 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88900319d0 con 0x7f889806c790 2024-05-20T22:11:13.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.285+0000 7f889d813640 1 -- 172.21.15.55:0/1741283772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f889806c790 msgr2=0x7f88980cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.285+0000 7f889d813640 1 --2- 172.21.15.55:0/1741283772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f889806c790 0x7f88980cf740 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8890009e50 tx=0x7f889002a710 comp rx=0 tx=0).stop 2024-05-20T22:11:13.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.285+0000 7f889d813640 1 -- 172.21.15.55:0/1741283772 shutdown_connections 2024-05-20T22:11:13.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.285+0000 7f889d813640 1 --2- 172.21.15.55:0/1741283772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f889806c790 0x7f88980cf740 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.285+0000 7f889d813640 1 --2- 172.21.15.55:0/1741283772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889815f100 0x7f88980b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.285+0000 7f889d813640 1 -- 172.21.15.55:0/1741283772 >> 172.21.15.55:0/1741283772 conn(0x7f88980d51d0 msgr2=0x7f88980b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:13.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.285+0000 7f889d813640 1 -- 172.21.15.55:0/1741283772 shutdown_connections 2024-05-20T22:11:13.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.285+0000 7f889d813640 1 -- 172.21.15.55:0/1741283772 wait complete. 2024-05-20T22:11:13.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.286+0000 7f889d813640 1 Processor -- start 2024-05-20T22:11:13.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.287+0000 7f889d813640 1 -- start start 2024-05-20T22:11:13.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f889d813640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889806c790 0x7f889815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f889d813640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88980b3b30 0x7f8898157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f889d813640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f889815f100 0x7f889815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f889d813640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f889806cba0 con 0x7f88980b3b30 2024-05-20T22:11:13.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f889d813640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f889806bb60 con 0x7f889815f100 2024-05-20T22:11:13.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f889d813640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f889806cd20 con 0x7f889806c790 2024-05-20T22:11:13.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f88967fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88980b3b30 0x7f8898157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f88967fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88980b3b30 0x7f8898157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35428/0 (socket says 172.21.15.55:35428) 2024-05-20T22:11:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f88967fc640 1 -- 172.21.15.55:0/2713096984 learned_addr learned my addr 172.21.15.55:0/2713096984 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f88977fe640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f889815f100 0x7f889815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f8896ffd640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889806c790 0x7f889815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f88967fc640 1 -- 172.21.15.55:0/2713096984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889806c790 msgr2=0x7f889815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f88967fc640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889806c790 0x7f889815ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f88967fc640 1 -- 172.21.15.55:0/2713096984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f889815f100 msgr2=0x7f889815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f88967fc640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f889815f100 0x7f889815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.288+0000 7f88967fc640 1 -- 172.21.15.55:0/2713096984 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8898160770 con 0x7f88980b3b30 2024-05-20T22:11:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.289+0000 7f88967fc640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88980b3b30 0x7f8898157850 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f888c00e8a0 tx=0x7f888c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.289+0000 7f887ffff640 1 -- 172.21.15.55:0/2713096984 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c017070 con 0x7f88980b3b30 2024-05-20T22:11:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.289+0000 7f887ffff640 1 -- 172.21.15.55:0/2713096984 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888c00c940 con 0x7f88980b3b30 2024-05-20T22:11:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.289+0000 7f887ffff640 1 -- 172.21.15.55:0/2713096984 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c00cd30 con 0x7f88980b3b30 2024-05-20T22:11:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.289+0000 7f889d813640 1 -- 172.21.15.55:0/2713096984 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8898160a50 con 0x7f88980b3b30 2024-05-20T22:11:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.289+0000 7f889d813640 1 -- 172.21.15.55:0/2713096984 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8898157d90 con 0x7f88980b3b30 2024-05-20T22:11:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.290+0000 7f889d813640 1 -- 172.21.15.55:0/2713096984 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8898069090 con 0x7f88980b3b30 2024-05-20T22:11:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.291+0000 7f887ffff640 1 -- 172.21.15.55:0/2713096984 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f888c01a070 con 0x7f88980b3b30 2024-05-20T22:11:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.291+0000 7f887ffff640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f888406be30 0x7f888406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.291+0000 7f887ffff640 1 -- 172.21.15.55:0/2713096984 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f888c08aea0 con 0x7f88980b3b30 2024-05-20T22:11:13.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.291+0000 7f8896ffd640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f888406be30 0x7f888406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.292+0000 7f8896ffd640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f888406be30 0x7f888406e340 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f889806d0c0 tx=0x7f8888009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:13.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.296+0000 7f887ffff640 1 -- 172.21.15.55:0/2713096984 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f888c059320 con 0x7f88980b3b30 2024-05-20T22:11:13.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.480+0000 7f889d813640 1 -- 172.21.15.55:0/2713096984 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88981604c0 con 0x7f888406be30 2024-05-20T22:11:13.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.481+0000 7f887ffff640 1 -- 172.21.15.55:0/2713096984 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13482 (secure 0 0 0) 0x7f88981604c0 con 0x7f888406be30 2024-05-20T22:11:13.482 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:13.482 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:11:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 -- 172.21.15.55:0/2713096984 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f888406be30 msgr2=0x7f888406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f888406be30 0x7f888406e340 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f889806d0c0 tx=0x7f8888009040 comp rx=0 tx=0).stop 2024-05-20T22:11:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 -- 172.21.15.55:0/2713096984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88980b3b30 msgr2=0x7f8898157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88980b3b30 0x7f8898157850 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f888c00e8a0 tx=0x7f888c00ed60 comp rx=0 tx=0).stop 2024-05-20T22:11:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 -- 172.21.15.55:0/2713096984 shutdown_connections 2024-05-20T22:11:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f888406be30 0x7f888406e340 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f889815f100 0x7f889815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88980b3b30 0x7f8898157850 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 --2- 172.21.15.55:0/2713096984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f889806c790 0x7f889815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 -- 172.21.15.55:0/2713096984 >> 172.21.15.55:0/2713096984 conn(0x7f88980d51d0 msgr2=0x7f88980b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 -- 172.21.15.55:0/2713096984 shutdown_connections 2024-05-20T22:11:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.483+0000 7f889d813640 1 -- 172.21.15.55:0/2713096984 wait complete. 2024-05-20T22:11:13.495 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-05-20T22:11:12.374154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":244048,"kb_used_data":7088,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730672,"statfs":{"total":773094113280,"available":772844208128,"internally_reserved":0,"allocated":7258112,"data_stored":3961998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7353,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-72,"num_write_kb":-30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001236"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":259,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-20T22:11:10.767738+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:11:10.767738+0000","last_peered":"2024-05-20T22:11:10.767738+0000","last_clean":"2024-05-20T22:11:10.767738+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:11:10.767738+0000","last_undegraded":"2024-05-20T22:11:10.767738+0000","last_fullsized":"2024-05-20T22:11:10.767738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":938024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":347663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31920,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":757760,"data_stored":347656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":347663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":938024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:11:13.495 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T22:11:13.590 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:11:13.592 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:11:13.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.609+0000 7f1492837640 1 Processor -- start 2024-05-20T22:11:13.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.609+0000 7f1492837640 1 -- start start 2024-05-20T22:11:13.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f1492837640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c15d550 0x7f148c09a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f1492837640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f148c0b3080 0x7f148c166b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f1492837640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f148c0b18f0 con 0x7f148c09a7a0 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f1492837640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f148c0b1bd0 con 0x7f148c0b3080 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f1492837640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f148c0b1eb0 con 0x7f148c15d550 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f1483fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f148c09a7a0 0x7f148c0b2970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44312/0 (socket says 172.21.15.55:44312) 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f1483fff640 1 -- 172.21.15.55:0/3968094845 learned_addr learned my addr 172.21.15.55:0/3968094845 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f148bfff640 1 --2- 172.21.15.55:0/3968094845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c15d550 0x7f148c09a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f1490dd4640 1 --2- 172.21.15.55:0/3968094845 >> v2:172.21.15.138:3300/0 conn(0x7f148c0b3080 0x7f148c166b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f148b7fe640 1 -- 172.21.15.55:0/3968094845 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035780326 0 0) 0x7f148c0b18f0 con 0x7f148c09a7a0 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.611+0000 7f148b7fe640 1 -- 172.21.15.55:0/3968094845 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1478003520 con 0x7f148c09a7a0 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.612+0000 7f1490dd4640 1 -- 172.21.15.55:0/3968094845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c15d550 msgr2=0x7f148c09a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.612+0000 7f1490dd4640 1 --2- 172.21.15.55:0/3968094845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c15d550 0x7f148c09a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.612+0000 7f1490dd4640 1 -- 172.21.15.55:0/3968094845 >> v1:172.21.15.55:6789/0 conn(0x7f148c09a7a0 legacy=0x7f148c0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.612+0000 7f1490dd4640 1 -- 172.21.15.55:0/3968094845 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c0b2190 con 0x7f148c0b3080 2024-05-20T22:11:13.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.612+0000 7f148bfff640 1 --2- 172.21.15.55:0/3968094845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c15d550 0x7f148c09a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T22:11:13.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.612+0000 7f1490dd4640 1 --2- 172.21.15.55:0/3968094845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c0b3080 0x7f148c166b10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1484009d90 tx=0x7f148402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8f6f2cfb5a518fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:13.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.613+0000 7f148b7fe640 1 -- 172.21.15.55:0/3968094845 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484007740 con 0x7f148c0b3080 2024-05-20T22:11:13.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.613+0000 7f148b7fe640 1 -- 172.21.15.55:0/3968094845 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14840078e0 con 0x7f148c0b3080 2024-05-20T22:11:13.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.613+0000 7f148b7fe640 1 -- 172.21.15.55:0/3968094845 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484007cb0 con 0x7f148c0b3080 2024-05-20T22:11:13.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.614+0000 7f1492837640 1 -- 172.21.15.55:0/3968094845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c0b3080 msgr2=0x7f148c166b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.614+0000 7f1492837640 1 --2- 172.21.15.55:0/3968094845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c0b3080 0x7f148c166b10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1484009d90 tx=0x7f148402a660 comp rx=0 tx=0).stop 2024-05-20T22:11:13.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.614+0000 7f1492837640 1 -- 172.21.15.55:0/3968094845 shutdown_connections 2024-05-20T22:11:13.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.614+0000 7f1492837640 1 --2- 172.21.15.55:0/3968094845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c0b3080 0x7f148c166b10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.614+0000 7f1492837640 1 --2- 172.21.15.55:0/3968094845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c15d550 0x7f148c09a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.614+0000 7f1492837640 1 -- 172.21.15.55:0/3968094845 >> 172.21.15.55:0/3968094845 conn(0x7f148c072f20 msgr2=0x7f148c0b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:13.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.614+0000 7f1492837640 1 -- 172.21.15.55:0/3968094845 shutdown_connections 2024-05-20T22:11:13.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.614+0000 7f1492837640 1 -- 172.21.15.55:0/3968094845 wait complete. 2024-05-20T22:11:13.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.615+0000 7f1492837640 1 Processor -- start 2024-05-20T22:11:13.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.616+0000 7f1492837640 1 -- start start 2024-05-20T22:11:13.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1492837640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c09a7a0 0x7f148c143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1492837640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f148c0b3080 0x7f148c143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1492837640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c15d550 0x7f148c1357e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1492837640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f148c0b3490 con 0x7f148c0b3080 2024-05-20T22:11:13.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1492837640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f148c15d960 con 0x7f148c15d550 2024-05-20T22:11:13.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1492837640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f148c0b3610 con 0x7f148c09a7a0 2024-05-20T22:11:13.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1483fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f148c0b3080 0x7f148c143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1483fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f148c0b3080 0x7f148c143f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35438/0 (socket says 172.21.15.55:35438) 2024-05-20T22:11:13.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1483fff640 1 -- 172.21.15.55:0/1122265517 learned_addr learned my addr 172.21.15.55:0/1122265517 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:13.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1490dd4640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c15d550 0x7f148c1357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f148bfff640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c09a7a0 0x7f148c143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1483fff640 1 -- 172.21.15.55:0/1122265517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c09a7a0 msgr2=0x7f148c143a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1483fff640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c09a7a0 0x7f148c143a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1483fff640 1 -- 172.21.15.55:0/1122265517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c15d550 msgr2=0x7f148c1357e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1483fff640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c15d550 0x7f148c1357e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.617+0000 7f1483fff640 1 -- 172.21.15.55:0/1122265517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c131640 con 0x7f148c0b3080 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.618+0000 7f1483fff640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f148c0b3080 0x7f148c143f90 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f14740010e0 tx=0x7f147400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.618+0000 7f14897fa640 1 -- 172.21.15.55:0/1122265517 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474017070 con 0x7f148c0b3080 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.618+0000 7f14897fa640 1 -- 172.21.15.55:0/1122265517 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1474005d00 con 0x7f148c0b3080 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.618+0000 7f1492837640 1 -- 172.21.15.55:0/1122265517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148c131920 con 0x7f148c0b3080 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.618+0000 7f14897fa640 1 -- 172.21.15.55:0/1122265517 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474007310 con 0x7f148c0b3080 2024-05-20T22:11:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.618+0000 7f1492837640 1 -- 172.21.15.55:0/1122265517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f148c131ee0 con 0x7f148c0b3080 2024-05-20T22:11:13.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.619+0000 7f1492837640 1 -- 172.21.15.55:0/1122265517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148c1313b0 con 0x7f148c0b3080 2024-05-20T22:11:13.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.620+0000 7f14897fa640 1 -- 172.21.15.55:0/1122265517 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f147401a070 con 0x7f148c0b3080 2024-05-20T22:11:13.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.620+0000 7f14897fa640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f146c06be30 0x7f146c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:13.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.620+0000 7f14897fa640 1 -- 172.21.15.55:0/1122265517 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f147408a120 con 0x7f148c0b3080 2024-05-20T22:11:13.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.620+0000 7f148bfff640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f146c06be30 0x7f146c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:13.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.621+0000 7f148bfff640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f146c06be30 0x7f146c06e340 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f147c009560 tx=0x7f147c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:13.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.625+0000 7f14897fa640 1 -- 172.21.15.55:0/1122265517 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1474058510 con 0x7f148c0b3080 2024-05-20T22:11:13.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.812+0000 7f1492837640 1 -- 172.21.15.55:0/1122265517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f148c069090 con 0x7f148c0b3080 2024-05-20T22:11:13.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.813+0000 7f14897fa640 1 -- 172.21.15.55:0/1122265517 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+13423 (secure 0 0 0) 0x7f147405c810 con 0x7f148c0b3080 2024-05-20T22:11:13.814 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:13.814 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":64,"fsid":"de8ca569-3fd1-4cb2-8bdc-99f367745927","created":"2024-05-20T22:07:27.906762+0000","modified":"2024-05-20T22:11:10.759234+0000","last_up_change":"2024-05-20T22:07:40.003102+0000","last_in_change":"2024-05-20T22:07:30.984077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T22:07:40.373261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2a65a30c-09c8-4932-b55c-3be3667d8acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6825","nonce":2576057485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6827","nonce":2576057485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6831","nonce":2576057485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":2576057485},{"type":"v1","addr":"172.21.15.55:6829","nonce":2576057485}]},"public_addr":"172.21.15.55:6825/2576057485","cluster_addr":"172.21.15.55:6827/2576057485","heartbeat_back_addr":"172.21.15.55:6831/2576057485","heartbeat_front_addr":"172.21.15.55:6829/2576057485","state":["exists","up"]},{"osd":1,"uuid":"9087b53a-b367-45fe-bb5f-4d05def6d27c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6817","nonce":518932600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6819","nonce":518932600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6823","nonce":518932600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":518932600},{"type":"v1","addr":"172.21.15.55:6821","nonce":518932600}]},"public_addr":"172.21.15.55:6817/518932600","cluster_addr":"172.21.15.55:6819/518932600","heartbeat_back_addr":"172.21.15.55:6823/518932600","heartbeat_front_addr":"172.21.15.55:6821/518932600","state":["exists","up"]},{"osd":2,"uuid":"f32d4168-f8e0-4ef1-b75e-604198b040ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6801","nonce":3066480496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6803","nonce":3066480496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6807","nonce":3066480496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":3066480496},{"type":"v1","addr":"172.21.15.55:6805","nonce":3066480496}]},"public_addr":"172.21.15.55:6801/3066480496","cluster_addr":"172.21.15.55:6803/3066480496","heartbeat_back_addr":"172.21.15.55:6807/3066480496","heartbeat_front_addr":"172.21.15.55:6805/3066480496","state":["exists","up"]},{"osd":3,"uuid":"3f99a86a-2fe8-407c-90fa-9cec443ba0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6809","nonce":3969115886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6811","nonce":3969115886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6815","nonce":3969115886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3969115886},{"type":"v1","addr":"172.21.15.55:6813","nonce":3969115886}]},"public_addr":"172.21.15.55:6809/3969115886","cluster_addr":"172.21.15.55:6811/3969115886","heartbeat_back_addr":"172.21.15.55:6815/3969115886","heartbeat_front_addr":"172.21.15.55:6813/3969115886","state":["exists","up"]},{"osd":4,"uuid":"d8a41f58-5245-4f29-98d5-316fb50a126a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6809","nonce":3059706956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6811","nonce":3059706956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6815","nonce":3059706956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3059706956},{"type":"v1","addr":"172.21.15.138:6813","nonce":3059706956}]},"public_addr":"172.21.15.138:6809/3059706956","cluster_addr":"172.21.15.138:6811/3059706956","heartbeat_back_addr":"172.21.15.138:6815/3059706956","heartbeat_front_addr":"172.21.15.138:6813/3059706956","state":["exists","up"]},{"osd":5,"uuid":"b6744a73-c861-4ce7-a303-5281e03b687d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6801","nonce":2873568230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6803","nonce":2873568230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6807","nonce":2873568230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2873568230},{"type":"v1","addr":"172.21.15.138:6805","nonce":2873568230}]},"public_addr":"172.21.15.138:6801/2873568230","cluster_addr":"172.21.15.138:6803/2873568230","heartbeat_back_addr":"172.21.15.138:6807/2873568230","heartbeat_front_addr":"172.21.15.138:6805/2873568230","state":["exists","up"]},{"osd":6,"uuid":"6541c8c6-e672-44e2-9604-253abe988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6825","nonce":1454288856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6827","nonce":1454288856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6831","nonce":1454288856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1454288856},{"type":"v1","addr":"172.21.15.138:6829","nonce":1454288856}]},"public_addr":"172.21.15.138:6825/1454288856","cluster_addr":"172.21.15.138:6827/1454288856","heartbeat_back_addr":"172.21.15.138:6831/1454288856","heartbeat_front_addr":"172.21.15.138:6829/1454288856","state":["exists","up"]},{"osd":7,"uuid":"986dd45e-e14c-4f4a-9320-5e2adc4bd06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6817","nonce":2068592032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6819","nonce":2068592032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6823","nonce":2068592032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2068592032},{"type":"v1","addr":"172.21.15.138:6821","nonce":2068592032}]},"public_addr":"172.21.15.138:6817/2068592032","cluster_addr":"172.21.15.138:6819/2068592032","heartbeat_back_addr":"172.21.15.138:6823/2068592032","heartbeat_front_addr":"172.21.15.138:6821/2068592032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.180155+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.137850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.134257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.200335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.163340+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.111953+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.212759+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T22:07:36.187185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:6837/3463078364":"2024-05-21T22:11:07.742328+0000","172.21.15.55:6836/3463078364":"2024-05-21T22:11:07.742328+0000","172.21.15.55:6837/2445634740":"2024-05-21T22:10:38.297533+0000","172.21.15.55:6835/2445634740":"2024-05-21T22:10:38.297533+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T22:11:13.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 -- 172.21.15.55:0/1122265517 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f146c06be30 msgr2=0x7f146c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f146c06be30 0x7f146c06e340 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f147c009560 tx=0x7f147c009190 comp rx=0 tx=0).stop 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 -- 172.21.15.55:0/1122265517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f148c0b3080 msgr2=0x7f148c143f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f148c0b3080 0x7f148c143f90 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f14740010e0 tx=0x7f147400c890 comp rx=0 tx=0).stop 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 -- 172.21.15.55:0/1122265517 shutdown_connections 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f146c06be30 0x7f146c06e340 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148c15d550 0x7f148c1357e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f148c0b3080 0x7f148c143f90 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 --2- 172.21.15.55:0/1122265517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f148c09a7a0 0x7f148c143a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 -- 172.21.15.55:0/1122265517 >> 172.21.15.55:0/1122265517 conn(0x7f148c072f20 msgr2=0x7f148c06fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 -- 172.21.15.55:0/1122265517 shutdown_connections 2024-05-20T22:11:13.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:13.815+0000 7f1492837640 1 -- 172.21.15.55:0/1122265517 wait complete. 2024-05-20T22:11:14.829 INFO:tasks.ceph:Scrubbing osd.0 2024-05-20T22:11:14.829 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-20T22:11:14.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.961+0000 7f92b832a640 1 Processor -- start 2024-05-20T22:11:14.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.961+0000 7f92b832a640 1 -- start start 2024-05-20T22:11:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b832a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f92b015f150 0x7f92b00cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b832a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f92b006b500 0x7f92b006b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b832a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b00d3780 con 0x7f92b00d32a0 2024-05-20T22:11:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b832a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f92b00d3a60 con 0x7f92b006b500 2024-05-20T22:11:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b832a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f92b00d3d40 con 0x7f92b015f150 2024-05-20T22:11:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b68c7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f92b006b500 0x7f92b006b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:14.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b60c6640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f92b00d32a0 0x7f92b00cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44324/0 (socket says 172.21.15.55:44324) 2024-05-20T22:11:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b60c6640 1 -- 172.21.15.55:0/1103010470 learned_addr learned my addr 172.21.15.55:0/1103010470 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b68c7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f92b006b500 0x7f92b006b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35442/0 (socket says 172.21.15.55:35442) 2024-05-20T22:11:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b58c5640 1 --2- 172.21.15.55:0/1103010470 >> v2:172.21.15.138:3300/0 conn(0x7f92b015f150 0x7f92b00cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b68c7640 1 -- 172.21.15.55:0/1103010470 >> v2:172.21.15.138:3300/0 conn(0x7f92b015f150 msgr2=0x7f92b00cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b68c7640 1 --2- 172.21.15.55:0/1103010470 >> v2:172.21.15.138:3300/0 conn(0x7f92b015f150 0x7f92b00cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b68c7640 1 -- 172.21.15.55:0/1103010470 >> v1:172.21.15.55:6789/0 conn(0x7f92b00d32a0 legacy=0x7f92b00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b68c7640 1 -- 172.21.15.55:0/1103010470 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b00d4020 con 0x7f92b006b500 2024-05-20T22:11:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.963+0000 7f92b50c4640 1 -- 172.21.15.55:0/1103010470 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1495541782 0 0) 0x7f92b00d3780 con 0x7f92b00d32a0 2024-05-20T22:11:14.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.964+0000 7f92b68c7640 1 --2- 172.21.15.55:0/1103010470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92b006b500 0x7f92b006b910 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f92b008e150 tx=0x7f92a802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13cf8da2efae421 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:14.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.964+0000 7f92b50c4640 1 -- 172.21.15.55:0/1103010470 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8007740 con 0x7f92b006b500 2024-05-20T22:11:14.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.964+0000 7f92b50c4640 1 -- 172.21.15.55:0/1103010470 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a80078e0 con 0x7f92b006b500 2024-05-20T22:11:14.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.964+0000 7f92b50c4640 1 -- 172.21.15.55:0/1103010470 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8007cb0 con 0x7f92b006b500 2024-05-20T22:11:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.965+0000 7f92b832a640 1 -- 172.21.15.55:0/1103010470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92b006b500 msgr2=0x7f92b006b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.965+0000 7f92b832a640 1 --2- 172.21.15.55:0/1103010470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92b006b500 0x7f92b006b910 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f92b008e150 tx=0x7f92a802a680 comp rx=0 tx=0).stop 2024-05-20T22:11:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.965+0000 7f92b832a640 1 -- 172.21.15.55:0/1103010470 shutdown_connections 2024-05-20T22:11:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.965+0000 7f92b832a640 1 --2- 172.21.15.55:0/1103010470 >> v2:172.21.15.138:3300/0 conn(0x7f92b015f150 0x7f92b00cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.965+0000 7f92b832a640 1 --2- 172.21.15.55:0/1103010470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92b006b500 0x7f92b006b910 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.965+0000 7f92b832a640 1 -- 172.21.15.55:0/1103010470 >> 172.21.15.55:0/1103010470 conn(0x7f92b00b6de0 msgr2=0x7f92b00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.965+0000 7f92b832a640 1 -- 172.21.15.55:0/1103010470 shutdown_connections 2024-05-20T22:11:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.965+0000 7f92b832a640 1 -- 172.21.15.55:0/1103010470 wait complete. 2024-05-20T22:11:14.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.967+0000 7f92b832a640 1 Processor -- start 2024-05-20T22:11:14.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.967+0000 7f92b832a640 1 -- start start 2024-05-20T22:11:14.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b832a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92b006b500 0x7f92b015f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b832a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92b00d32a0 0x7f92b01f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b832a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92b015f150 0x7f92b01f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b832a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f92b00d4820 con 0x7f92b006b500 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b832a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92b015f560 con 0x7f92b015f150 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b832a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f92b014f330 con 0x7f92b00d32a0 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b60c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92b006b500 0x7f92b015f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b60c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92b006b500 0x7f92b015f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35444/0 (socket says 172.21.15.55:35444) 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b60c6640 1 -- 172.21.15.55:0/75619555 learned_addr learned my addr 172.21.15.55:0/75619555 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b58c5640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92b00d32a0 0x7f92b01f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b68c7640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92b015f150 0x7f92b01f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b60c6640 1 -- 172.21.15.55:0/75619555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92b00d32a0 msgr2=0x7f92b01f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:14.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.968+0000 7f92b60c6640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92b00d32a0 0x7f92b01f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92b60c6640 1 -- 172.21.15.55:0/75619555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92b015f150 msgr2=0x7f92b01f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92b60c6640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92b015f150 0x7f92b01f8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92b60c6640 1 -- 172.21.15.55:0/75619555 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b01f5390 con 0x7f92b006b500 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92b58c5640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92b00d32a0 0x7f92b01f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92b68c7640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92b015f150 0x7f92b01f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92b60c6640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92b006b500 0x7f92b015f770 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c430 tx=0x7f92ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92a6ffd640 1 -- 172.21.15.55:0/75619555 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0074e0 con 0x7f92b006b500 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92a6ffd640 1 -- 172.21.15.55:0/75619555 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ac0047e0 con 0x7f92b006b500 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b01602e0 con 0x7f92b006b500 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92a6ffd640 1 -- 172.21.15.55:0/75619555 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac004bd0 con 0x7f92b006b500 2024-05-20T22:11:14.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.969+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b01608a0 con 0x7f92b006b500 2024-05-20T22:11:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.970+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f92b0069090 con 0x7f92b006b500 2024-05-20T22:11:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.971+0000 7f92a6ffd640 1 -- 172.21.15.55:0/75619555 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92ac011070 con 0x7f92b006b500 2024-05-20T22:11:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.971+0000 7f92a6ffd640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f929406be60 0x7f929406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.971+0000 7f92a6ffd640 1 -- 172.21.15.55:0/75619555 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f92ac08a9e0 con 0x7f92b006b500 2024-05-20T22:11:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.971+0000 7f92b58c5640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f929406be60 0x7f929406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.971+0000 7f92a6ffd640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] conn(0x7f92940723f0 0x7f9294074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.971+0000 7f92a6ffd640 1 -- 172.21.15.55:0/75619555 --> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92ac00ee50 con 0x7f92940723f0 2024-05-20T22:11:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.971+0000 7f92a6ffd640 1 -- 172.21.15.55:0/75619555 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f92ac08f2c0 con 0x7f92b006b500 2024-05-20T22:11:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.971+0000 7f92b68c7640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] conn(0x7f92940723f0 0x7f9294074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:14.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.972+0000 7f92b58c5640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f929406be60 0x7f929406e370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f92a0006fd0 tx=0x7f92a0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:14.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.972+0000 7f92b68c7640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] conn(0x7f92940723f0 0x7f9294074880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:14.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.973+0000 7f92a6ffd640 1 -- 172.21.15.55:0/75619555 <== osd.0 v2:172.21.15.55:6824/2576057485 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92ac00ee50 con 0x7f92940723f0 2024-05-20T22:11:14.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.986+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 --> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f92b00d2e50 con 0x7f92940723f0 2024-05-20T22:11:14.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.986+0000 7f92a6ffd640 1 -- 172.21.15.55:0/75619555 <== osd.0 v2:172.21.15.55:6824/2576057485 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f92b00d2e50 con 0x7f92940723f0 2024-05-20T22:11:14.988 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:11:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T22:11:14.989 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:11:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.986+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] conn(0x7f92940723f0 msgr2=0x7f9294074880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.987+0000 7f92b832a640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:6824/2576057485,v1:172.21.15.55:6825/2576057485] conn(0x7f92940723f0 0x7f9294074880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.987+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f929406be60 msgr2=0x7f929406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.987+0000 7f92b832a640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f929406be60 0x7f929406e370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f92a0006fd0 tx=0x7f92a0008040 comp rx=0 tx=0).stop 2024-05-20T22:11:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.987+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92b006b500 msgr2=0x7f92b015f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.987+0000 7f92b832a640 1 --2- 172.21.15.55:0/75619555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92b006b500 0x7f92b015f770 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c430 tx=0x7f92ac00c8f0 comp rx=0 tx=0).stop 2024-05-20T22:11:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.987+0000 7f92b68c7640 1 -- 172.21.15.55:0/75619555 reap_dead start 2024-05-20T22:11:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.987+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 shutdown_connections 2024-05-20T22:11:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.987+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 >> 172.21.15.55:0/75619555 conn(0x7f92b00b6de0 msgr2=0x7f92b00b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.987+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 shutdown_connections 2024-05-20T22:11:14.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:14.987+0000 7f92b832a640 1 -- 172.21.15.55:0/75619555 wait complete. 2024-05-20T22:11:14.999 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-20T22:11:15.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.115+0000 7fb42e673640 1 Processor -- start 2024-05-20T22:11:15.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.115+0000 7fb42e673640 1 -- start start 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.116+0000 7fb42e673640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280ceb10 0x7fb4280d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.116+0000 7fb42e673640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4280d1ee0 con 0x7fb42806e0c0 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.116+0000 7fb42e673640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4280cef20 con 0x7fb4280d1ad0 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.116+0000 7fb42e673640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb42809a260 con 0x7fb4280ceb10 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.116+0000 7fb427fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb42806e0c0 0x7fb42815cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44338/0 (socket says 172.21.15.55:44338) 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.116+0000 7fb427fff640 1 -- 172.21.15.55:0/2247106520 learned_addr learned my addr 172.21.15.55:0/2247106520 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.116+0000 7fb4277fe640 1 --2- 172.21.15.55:0/2247106520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280ceb10 0x7fb4280d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.117+0000 7fb426ffd640 1 -- 172.21.15.55:0/2247106520 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 181967004 0 0) 0x7fb418001110 con 0x7fb42806e0c0 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.117+0000 7fb426ffd640 1 -- 172.21.15.55:0/2247106520 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb410003520 con 0x7fb42806e0c0 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.117+0000 7fb426ffd640 1 -- 172.21.15.55:0/2247106520 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2648068834 0 0) 0x7fb4280cef20 con 0x7fb4280d1ad0 2024-05-20T22:11:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.117+0000 7fb426ffd640 1 -- 172.21.15.55:0/2247106520 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb418001110 con 0x7fb4280d1ad0 2024-05-20T22:11:15.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.117+0000 7fb4277fe640 1 -- 172.21.15.55:0/2247106520 >> v1:172.21.15.138:6789/0 conn(0x7fb4280d1ad0 legacy=0x7fb4280cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.117+0000 7fb4277fe640 1 -- 172.21.15.55:0/2247106520 >> v1:172.21.15.55:6789/0 conn(0x7fb42806e0c0 legacy=0x7fb42815cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.117+0000 7fb4277fe640 1 -- 172.21.15.55:0/2247106520 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42809a540 con 0x7fb4280ceb10 2024-05-20T22:11:15.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.118+0000 7fb4277fe640 1 --2- 172.21.15.55:0/2247106520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280ceb10 0x7fb4280d1590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0097d0 tx=0x7fb41c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ff4bfd41b12284 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.118+0000 7fb426ffd640 1 -- 172.21.15.55:0/2247106520 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c037070 con 0x7fb4280ceb10 2024-05-20T22:11:15.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.118+0000 7fb426ffd640 1 -- 172.21.15.55:0/2247106520 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb41c032440 con 0x7fb4280ceb10 2024-05-20T22:11:15.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.118+0000 7fb426ffd640 1 -- 172.21.15.55:0/2247106520 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0327f0 con 0x7fb4280ceb10 2024-05-20T22:11:15.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.118+0000 7fb42e673640 1 -- 172.21.15.55:0/2247106520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280ceb10 msgr2=0x7fb4280d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.118+0000 7fb42e673640 1 --2- 172.21.15.55:0/2247106520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280ceb10 0x7fb4280d1590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0097d0 tx=0x7fb41c02a210 comp rx=0 tx=0).stop 2024-05-20T22:11:15.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.118+0000 7fb42e673640 1 -- 172.21.15.55:0/2247106520 shutdown_connections 2024-05-20T22:11:15.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.118+0000 7fb42e673640 1 --2- 172.21.15.55:0/2247106520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280ceb10 0x7fb4280d1590 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.118+0000 7fb42e673640 1 -- 172.21.15.55:0/2247106520 >> 172.21.15.55:0/2247106520 conn(0x7fb4280736f0 msgr2=0x7fb4280b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.119+0000 7fb42e673640 1 -- 172.21.15.55:0/2247106520 shutdown_connections 2024-05-20T22:11:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.119+0000 7fb42e673640 1 -- 172.21.15.55:0/2247106520 wait complete. 2024-05-20T22:11:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.120+0000 7fb42e673640 1 Processor -- start 2024-05-20T22:11:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.121+0000 7fb42e673640 1 -- start start 2024-05-20T22:11:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb42e673640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb42806e0c0 0x7fb428159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb42e673640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4280ceb10 0x7fb4280ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb42e673640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280d1ad0 0x7fb4280ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb42e673640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb42815d2a0 con 0x7fb42806e0c0 2024-05-20T22:11:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb42e673640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb42815c670 con 0x7fb4280ceb10 2024-05-20T22:11:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb42e673640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4280b2fe0 con 0x7fb4280d1ad0 2024-05-20T22:11:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb427fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb42806e0c0 0x7fb428159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb427fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb42806e0c0 0x7fb428159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35456/0 (socket says 172.21.15.55:35456) 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb427fff640 1 -- 172.21.15.55:0/500302858 learned_addr learned my addr 172.21.15.55:0/500302858 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb42cc10640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280d1ad0 0x7fb4280ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb427fff640 1 -- 172.21.15.55:0/500302858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280d1ad0 msgr2=0x7fb4280ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb427fff640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280d1ad0 0x7fb4280ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb4277fe640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4280ceb10 0x7fb4280ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb427fff640 1 -- 172.21.15.55:0/500302858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4280ceb10 msgr2=0x7fb4280ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb427fff640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4280ceb10 0x7fb4280ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb427fff640 1 -- 172.21.15.55:0/500302858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4280c5c70 con 0x7fb42806e0c0 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.122+0000 7fb427fff640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb42806e0c0 0x7fb428159610 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb41800c730 tx=0x7fb41800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.123+0000 7fb424ff9640 1 -- 172.21.15.55:0/500302858 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418017070 con 0x7fb42806e0c0 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.123+0000 7fb424ff9640 1 -- 172.21.15.55:0/500302858 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb418005ab0 con 0x7fb42806e0c0 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.123+0000 7fb42e673640 1 -- 172.21.15.55:0/500302858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4280c61a0 con 0x7fb42806e0c0 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.123+0000 7fb424ff9640 1 -- 172.21.15.55:0/500302858 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418005e60 con 0x7fb42806e0c0 2024-05-20T22:11:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.123+0000 7fb42e673640 1 -- 172.21.15.55:0/500302858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4280c6760 con 0x7fb42806e0c0 2024-05-20T22:11:15.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.124+0000 7fb42e673640 1 -- 172.21.15.55:0/500302858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb428069090 con 0x7fb42806e0c0 2024-05-20T22:11:15.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.124+0000 7fb424ff9640 1 -- 172.21.15.55:0/500302858 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb41801a070 con 0x7fb42806e0c0 2024-05-20T22:11:15.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.125+0000 7fb424ff9640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 0x7fb41406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.125+0000 7fb424ff9640 1 -- 172.21.15.55:0/500302858 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7fb41808a210 con 0x7fb42806e0c0 2024-05-20T22:11:15.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.125+0000 7fb4277fe640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 0x7fb41406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.125+0000 7fb4277fe640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 0x7fb41406e340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb41c034000 tx=0x7fb41c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.129+0000 7fb424ff9640 1 -- 172.21.15.55:0/500302858 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb4280d1ee0 con 0x7fb42806e0c0 2024-05-20T22:11:15.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.314+0000 7fb42e673640 1 -- 172.21.15.55:0/500302858 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fb4280cd260 con 0x7fb41406be30 2024-05-20T22:11:15.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.314+0000 7fb424ff9640 1 -- 172.21.15.55:0/500302858 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb4280cd260 con 0x7fb41406be30 2024-05-20T22:11:15.315 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 0 to deep-scrub 2024-05-20T22:11:15.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.316+0000 7fb42e673640 1 -- 172.21.15.55:0/500302858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 msgr2=0x7fb41406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.316+0000 7fb42e673640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 0x7fb41406e340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb41c034000 tx=0x7fb41c02e000 comp rx=0 tx=0).stop 2024-05-20T22:11:15.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.316+0000 7fb42e673640 1 -- 172.21.15.55:0/500302858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb42806e0c0 msgr2=0x7fb428159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.316+0000 7fb42e673640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb42806e0c0 0x7fb428159610 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb41800c730 tx=0x7fb41800cbf0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.316+0000 7fb42e673640 1 -- 172.21.15.55:0/500302858 shutdown_connections 2024-05-20T22:11:15.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.316+0000 7fb42e673640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb41406be30 0x7fb41406e340 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.317+0000 7fb42e673640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4280d1ad0 0x7fb4280ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.317+0000 7fb42e673640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4280ceb10 0x7fb4280ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.317+0000 7fb42e673640 1 --2- 172.21.15.55:0/500302858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb42806e0c0 0x7fb428159610 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.317+0000 7fb42e673640 1 -- 172.21.15.55:0/500302858 >> 172.21.15.55:0/500302858 conn(0x7fb4280736f0 msgr2=0x7fb42815d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:15.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.317+0000 7fb42e673640 1 -- 172.21.15.55:0/500302858 shutdown_connections 2024-05-20T22:11:15.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.317+0000 7fb42e673640 1 -- 172.21.15.55:0/500302858 wait complete. 2024-05-20T22:11:15.329 INFO:tasks.ceph:Scrubbing osd.1 2024-05-20T22:11:15.329 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-20T22:11:15.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.446+0000 7f9f69b4c640 1 Processor -- start 2024-05-20T22:11:15.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.446+0000 7f9f69b4c640 1 -- start start 2024-05-20T22:11:15.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f69b4c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f640d32a0 0x7f9f6415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f69b4c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f640cf690 0x7f9f640cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f69b4c640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f640d3780 con 0x7f9f640cffe0 2024-05-20T22:11:15.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f69b4c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9f6406c510 con 0x7f9f640d32a0 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f69b4c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f640d3a60 con 0x7f9f640cf690 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f63fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9f640cffe0 0x7f9f64162cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44346/0 (socket says 172.21.15.55:44346) 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f63fff640 1 -- 172.21.15.55:0/4154871882 learned_addr learned my addr 172.21.15.55:0/4154871882 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f62ffd640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f640cf690 0x7f9f640cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3852725299 0 0) 0x7f9f640d3780 con 0x7f9f640cffe0 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f637fe640 1 --2- 172.21.15.55:0/4154871882 >> v2:172.21.15.138:3300/0 conn(0x7f9f640d32a0 0x7f9f6415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f4c003520 con 0x7f9f640cffe0 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2432429285 0 0) 0x7f9f4c003520 con 0x7f9f640cffe0 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f640cf690 msgr2=0x7f9f640cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f640cf690 0x7f9f640cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 >> v2:172.21.15.138:3300/0 conn(0x7f9f640d32a0 msgr2=0x7f9f6415f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 --2- 172.21.15.55:0/4154871882 >> v2:172.21.15.138:3300/0 conn(0x7f9f640d32a0 0x7f9f6415f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f640d3d40 con 0x7f9f640cffe0 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f9f5c0019f0 con 0x7f9f640cffe0 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 >> v1:172.21.15.55:6789/0 conn(0x7f9f640cffe0 legacy=0x7f9f64162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f4c0039b0 0x7f9f4c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f640cf690 0x7f9f4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c004d40 0x7f9f4c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6406c510 con 0x7f9f4c004d40 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f640d3a60 con 0x7f9f4c0039b0 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.448+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4c00b9c0 con 0x7f9f640cf690 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.449+0000 7f9f63fff640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c004d40 0x7f9f4c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.449+0000 7f9f637fe640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f4c0039b0 0x7f9f4c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.449+0000 7f9f62ffd640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f640cf690 0x7f9f4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.449+0000 7f9f63fff640 1 -- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f640cf690 msgr2=0x7f9f4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.449+0000 7f9f63fff640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f640cf690 0x7f9f4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.449+0000 7f9f63fff640 1 -- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f4c0039b0 msgr2=0x7f9f4c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.449+0000 7f9f63fff640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f4c0039b0 0x7f9f4c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.449+0000 7f9f63fff640 1 -- 172.21.15.55:0/4154871882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c003520 con 0x7f9f4c004d40 2024-05-20T22:11:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.450+0000 7f9f63fff640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c004d40 0x7f9f4c005150 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9f640b7830 tx=0x7f9f4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844e20a4394baea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.450+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00a4e0 con 0x7f9f4c004d40 2024-05-20T22:11:15.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.450+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c02f070 con 0x7f9f4c004d40 2024-05-20T22:11:15.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.450+0000 7f9f627fc640 1 -- 172.21.15.55:0/4154871882 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c02bc90 con 0x7f9f4c004d40 2024-05-20T22:11:15.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.450+0000 7f9f69b4c640 1 -- 172.21.15.55:0/4154871882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c004d40 msgr2=0x7f9f4c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.450+0000 7f9f69b4c640 1 --2- 172.21.15.55:0/4154871882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c004d40 0x7f9f4c005150 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9f640b7830 tx=0x7f9f4c005830 comp rx=0 tx=0).stop 2024-05-20T22:11:15.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.450+0000 7f9f63fff640 1 -- 172.21.15.55:0/4154871882 reap_dead start 2024-05-20T22:11:15.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.450+0000 7f9f69b4c640 1 -- 172.21.15.55:0/4154871882 shutdown_connections 2024-05-20T22:11:15.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.450+0000 7f9f69b4c640 1 -- 172.21.15.55:0/4154871882 >> 172.21.15.55:0/4154871882 conn(0x7f9f640b6de0 msgr2=0x7f9f640d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:15.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.451+0000 7f9f69b4c640 1 -- 172.21.15.55:0/4154871882 shutdown_connections 2024-05-20T22:11:15.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.451+0000 7f9f69b4c640 1 -- 172.21.15.55:0/4154871882 wait complete. 2024-05-20T22:11:15.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.452+0000 7f9f69b4c640 1 Processor -- start 2024-05-20T22:11:15.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.452+0000 7f9f69b4c640 1 -- start start 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f69b4c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f6414af50 0x7f9f641f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f69b4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f640ca7e0 0x7f9f640cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f69b4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f641e9b20 0x7f9f641e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f69b4c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9f640d03f0 con 0x7f9f641e9b20 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f69b4c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f640d4300 con 0x7f9f640ca7e0 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f69b4c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f640d4820 con 0x7f9f6414af50 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f641e9b20 0x7f9f641e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f641e9b20 0x7f9f641e9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35476/0 (socket says 172.21.15.55:35476) 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f63fff640 1 -- 172.21.15.55:0/22398710 learned_addr learned my addr 172.21.15.55:0/22398710 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f637fe640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f6414af50 0x7f9f641f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f62ffd640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f640ca7e0 0x7f9f640cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f63fff640 1 -- 172.21.15.55:0/22398710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f6414af50 msgr2=0x7f9f641f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f63fff640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f6414af50 0x7f9f641f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f63fff640 1 -- 172.21.15.55:0/22398710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f640ca7e0 msgr2=0x7f9f640cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f63fff640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f640ca7e0 0x7f9f640cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f63fff640 1 -- 172.21.15.55:0/22398710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f640cb130 con 0x7f9f641e9b20 2024-05-20T22:11:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.454+0000 7f9f63fff640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f641e9b20 0x7f9f641e9f30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c000f80 tx=0x7f9f640d3f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.455+0000 7f9f4bfff640 1 -- 172.21.15.55:0/22398710 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00a4e0 con 0x7f9f641e9b20 2024-05-20T22:11:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.455+0000 7f9f4bfff640 1 -- 172.21.15.55:0/22398710 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c0019e0 con 0x7f9f641e9b20 2024-05-20T22:11:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.455+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f640cbbd0 con 0x7f9f641e9b20 2024-05-20T22:11:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.455+0000 7f9f4bfff640 1 -- 172.21.15.55:0/22398710 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c03f470 con 0x7f9f641e9b20 2024-05-20T22:11:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.455+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f640cc190 con 0x7f9f641e9b20 2024-05-20T22:11:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.455+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f64069090 con 0x7f9f641e9b20 2024-05-20T22:11:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.456+0000 7f9f4bfff640 1 -- 172.21.15.55:0/22398710 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f5c03f610 con 0x7f9f641e9b20 2024-05-20T22:11:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.457+0000 7f9f4bfff640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9f5006be30 0x7f9f5006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.457+0000 7f9f4bfff640 1 -- 172.21.15.55:0/22398710 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f9f5c0ab3f0 con 0x7f9f641e9b20 2024-05-20T22:11:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.457+0000 7f9f637fe640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9f5006be30 0x7f9f5006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.457+0000 7f9f4bfff640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] conn(0x7f9f500723c0 0x7f9f50074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.457+0000 7f9f4bfff640 1 -- 172.21.15.55:0/22398710 --> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f50074f20 con 0x7f9f500723c0 2024-05-20T22:11:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.457+0000 7f9f4bfff640 1 -- 172.21.15.55:0/22398710 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f9f5c0ab880 con 0x7f9f641e9b20 2024-05-20T22:11:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.457+0000 7f9f62ffd640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] conn(0x7f9f500723c0 0x7f9f50074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.457+0000 7f9f637fe640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9f5006be30 0x7f9f5006e340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9f6415e550 tx=0x7f9f54009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.458+0000 7f9f62ffd640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] conn(0x7f9f500723c0 0x7f9f50074850 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.458+0000 7f9f4bfff640 1 -- 172.21.15.55:0/22398710 <== osd.1 v2:172.21.15.55:6816/518932600 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f50074f20 con 0x7f9f500723c0 2024-05-20T22:11:15.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.471+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 --> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9f640d2e50 con 0x7f9f500723c0 2024-05-20T22:11:15.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.471+0000 7f9f4bfff640 1 -- 172.21.15.55:0/22398710 <== osd.1 v2:172.21.15.55:6816/518932600 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9f640d2e50 con 0x7f9f500723c0 2024-05-20T22:11:15.474 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:11:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T22:11:15.474 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:11:15.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] conn(0x7f9f500723c0 msgr2=0x7f9f50074850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f69b4c640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:6816/518932600,v1:172.21.15.55:6817/518932600] conn(0x7f9f500723c0 0x7f9f50074850 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9f5006be30 msgr2=0x7f9f5006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f69b4c640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9f5006be30 0x7f9f5006e340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9f6415e550 tx=0x7f9f54009190 comp rx=0 tx=0).stop 2024-05-20T22:11:15.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f641e9b20 msgr2=0x7f9f641e9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f69b4c640 1 --2- 172.21.15.55:0/22398710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f641e9b20 0x7f9f641e9f30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c000f80 tx=0x7f9f640d3f10 comp rx=0 tx=0).stop 2024-05-20T22:11:15.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f63fff640 1 -- 172.21.15.55:0/22398710 reap_dead start 2024-05-20T22:11:15.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 shutdown_connections 2024-05-20T22:11:15.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 >> 172.21.15.55:0/22398710 conn(0x7f9f640b6de0 msgr2=0x7f9f640b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:15.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 shutdown_connections 2024-05-20T22:11:15.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.472+0000 7f9f69b4c640 1 -- 172.21.15.55:0/22398710 wait complete. 2024-05-20T22:11:15.484 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-20T22:11:15.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.598+0000 7f9ca3875640 1 Processor -- start 2024-05-20T22:11:15.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.599+0000 7f9ca3875640 1 -- start start 2024-05-20T22:11:15.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.600+0000 7f9ca3875640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c167ff0 0x7f9c9c16a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.600+0000 7f9ca3875640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c9c0d04b0 con 0x7f9c9c15f150 2024-05-20T22:11:15.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.600+0000 7f9ca3875640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c9c0d3780 con 0x7f9c9c0d32a0 2024-05-20T22:11:15.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.600+0000 7f9ca3875640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9c06c510 con 0x7f9c9c167ff0 2024-05-20T22:11:15.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.600+0000 7f9ca0e10640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9c9c15f150 0x7f9c9c0cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44358/0 (socket says 172.21.15.55:44358) 2024-05-20T22:11:15.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.600+0000 7f9ca0e10640 1 -- 172.21.15.55:0/2816981587 learned_addr learned my addr 172.21.15.55:0/2816981587 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:15.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.600+0000 7f9ca1e12640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c167ff0 0x7f9c9c16a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.600+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688270778 0 0) 0x7f9c9c0d04b0 con 0x7f9c9c15f150 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c84003520 con 0x7f9c9c15f150 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 5284846 0 0) 0x7f9c84003520 con 0x7f9c9c15f150 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c167ff0 msgr2=0x7f9c9c16a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c167ff0 0x7f9c9c16a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 >> v1:172.21.15.138:6789/0 conn(0x7f9c9c0d32a0 legacy=0x7f9c9c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c0d3a60 con 0x7f9c9c15f150 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f9c90001670 con 0x7f9c9c15f150 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 >> v1:172.21.15.55:6789/0 conn(0x7f9c9c15f150 legacy=0x7f9c9c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 0x7f9c84003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c167ff0 0x7f9c84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c84004d40 0x7f9c84005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c06c510 con 0x7f9c840039e0 2024-05-20T22:11:15.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8400ba70 con 0x7f9c84004d40 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8400bc70 con 0x7f9c9c167ff0 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9ca1611640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 0x7f9c84003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9ca0e10640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c167ff0 0x7f9c84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9ca1e12640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c84004d40 0x7f9c84005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9ca1611640 1 -- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c167ff0 msgr2=0x7f9c84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9ca1611640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c167ff0 0x7f9c84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9ca1611640 1 -- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c84004d40 msgr2=0x7f9c84005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9ca1611640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c84004d40 0x7f9c84005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.601+0000 7f9ca1611640 1 -- 172.21.15.55:0/2816981587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c84003520 con 0x7f9c840039e0 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.602+0000 7f9ca1611640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 0x7f9c84003e10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c002010 tx=0x7f9c84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdca8f60a3ef36f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.602+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c036070 con 0x7f9c840039e0 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.602+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c031600 con 0x7f9c840039e0 2024-05-20T22:11:15.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.602+0000 7f9c9bfff640 1 -- 172.21.15.55:0/2816981587 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c001be0 con 0x7f9c840039e0 2024-05-20T22:11:15.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.603+0000 7f9ca3875640 1 -- 172.21.15.55:0/2816981587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 msgr2=0x7f9c84003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.603+0000 7f9ca3875640 1 --2- 172.21.15.55:0/2816981587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 0x7f9c84003e10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c002010 tx=0x7f9c84005830 comp rx=0 tx=0).stop 2024-05-20T22:11:15.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.603+0000 7f9ca1e12640 1 -- 172.21.15.55:0/2816981587 reap_dead start 2024-05-20T22:11:15.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.603+0000 7f9ca3875640 1 -- 172.21.15.55:0/2816981587 shutdown_connections 2024-05-20T22:11:15.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.603+0000 7f9ca3875640 1 -- 172.21.15.55:0/2816981587 >> 172.21.15.55:0/2816981587 conn(0x7f9c9c0b6de0 msgr2=0x7f9c9c0d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:15.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.603+0000 7f9ca3875640 1 -- 172.21.15.55:0/2816981587 shutdown_connections 2024-05-20T22:11:15.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.603+0000 7f9ca3875640 1 -- 172.21.15.55:0/2816981587 wait complete. 2024-05-20T22:11:15.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.605+0000 7f9ca3875640 1 Processor -- start 2024-05-20T22:11:15.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.605+0000 7f9ca3875640 1 -- start start 2024-05-20T22:11:15.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca3875640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 0x7f9c9c152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca3875640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9c0ce940 0x7f9c9c15f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca3875640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c0ce3f0 0x7f9c9c15fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca3875640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c0d4300 con 0x7f9c840039e0 2024-05-20T22:11:15.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca3875640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c0d4020 con 0x7f9c9c0ce940 2024-05-20T22:11:15.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca3875640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9c15f560 con 0x7f9c9c0ce3f0 2024-05-20T22:11:15.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca1611640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 0x7f9c9c152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca1611640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 0x7f9c9c152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35490/0 (socket says 172.21.15.55:35490) 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca1611640 1 -- 172.21.15.55:0/2424464141 learned_addr learned my addr 172.21.15.55:0/2424464141 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca1e12640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c0ce3f0 0x7f9c9c15fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.606+0000 7f9ca0e10640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9c0ce940 0x7f9c9c15f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9ca1611640 1 -- 172.21.15.55:0/2424464141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c0ce3f0 msgr2=0x7f9c9c15fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9ca1611640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c0ce3f0 0x7f9c9c15fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9ca1611640 1 -- 172.21.15.55:0/2424464141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9c0ce940 msgr2=0x7f9c9c15f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9ca1611640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9c0ce940 0x7f9c9c15f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9ca1611640 1 -- 172.21.15.55:0/2424464141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c0b8120 con 0x7f9c840039e0 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9ca1611640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 0x7f9c9c152110 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c002010 tx=0x7f9c8c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9c99ffb640 1 -- 172.21.15.55:0/2424464141 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c036040 con 0x7f9c840039e0 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9c99ffb640 1 -- 172.21.15.55:0/2424464141 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c003970 con 0x7f9c840039e0 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9ca3875640 1 -- 172.21.15.55:0/2424464141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9c161b40 con 0x7f9c840039e0 2024-05-20T22:11:15.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9c99ffb640 1 -- 172.21.15.55:0/2424464141 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c007df0 con 0x7f9c840039e0 2024-05-20T22:11:15.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.607+0000 7f9ca3875640 1 -- 172.21.15.55:0/2424464141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9c162100 con 0x7f9c840039e0 2024-05-20T22:11:15.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.608+0000 7f9ca3875640 1 -- 172.21.15.55:0/2424464141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9c1618c0 con 0x7f9c840039e0 2024-05-20T22:11:15.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.609+0000 7f9c99ffb640 1 -- 172.21.15.55:0/2424464141 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c8c005070 con 0x7f9c840039e0 2024-05-20T22:11:15.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.609+0000 7f9c99ffb640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c8806be60 0x7f9c8806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.609+0000 7f9c99ffb640 1 -- 172.21.15.55:0/2424464141 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f9c8c0ab760 con 0x7f9c840039e0 2024-05-20T22:11:15.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.610+0000 7f9ca0e10640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c8806be60 0x7f9c8806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.610+0000 7f9ca0e10640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c8806be60 0x7f9c8806e370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0ba980 tx=0x7f9c90001e20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.614+0000 7f9c99ffb640 1 -- 172.21.15.55:0/2424464141 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9c9c1618c0 con 0x7f9c840039e0 2024-05-20T22:11:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.798+0000 7f9ca3875640 1 -- 172.21.15.55:0/2424464141 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f9c9c069090 con 0x7f9c8806be60 2024-05-20T22:11:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.799+0000 7f9c99ffb640 1 -- 172.21.15.55:0/2424464141 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9c9c069090 con 0x7f9c8806be60 2024-05-20T22:11:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 1 to deep-scrub 2024-05-20T22:11:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 -- 172.21.15.55:0/2424464141 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c8806be60 msgr2=0x7f9c8806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c8806be60 0x7f9c8806e370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0ba980 tx=0x7f9c90001e20 comp rx=0 tx=0).stop 2024-05-20T22:11:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 -- 172.21.15.55:0/2424464141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 msgr2=0x7f9c9c152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 0x7f9c9c152110 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c002010 tx=0x7f9c8c001630 comp rx=0 tx=0).stop 2024-05-20T22:11:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 -- 172.21.15.55:0/2424464141 shutdown_connections 2024-05-20T22:11:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9c8806be60 0x7f9c8806e370 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c9c0ce3f0 0x7f9c9c15fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9c0ce940 0x7f9c9c15f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 --2- 172.21.15.55:0/2424464141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c840039e0 0x7f9c9c152110 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 -- 172.21.15.55:0/2424464141 >> 172.21.15.55:0/2424464141 conn(0x7f9c9c0b6de0 msgr2=0x7f9c9c0b4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 -- 172.21.15.55:0/2424464141 shutdown_connections 2024-05-20T22:11:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.801+0000 7f9ca3875640 1 -- 172.21.15.55:0/2424464141 wait complete. 2024-05-20T22:11:15.813 INFO:tasks.ceph:Scrubbing osd.2 2024-05-20T22:11:15.813 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-20T22:11:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.930+0000 7f848f679640 1 Processor -- start 2024-05-20T22:11:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.931+0000 7f848f679640 1 -- start start 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848f679640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f84880d0720 0x7f84880cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848f679640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f848806dc90 0x7f848806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848f679640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84880d0e40 0x7f84880d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848f679640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f848806e5e0 con 0x7f84880d0720 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848f679640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f84880d1790 con 0x7f848806dc90 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848f679640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f84880cfa10 con 0x7f84880d0e40 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848d415640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f84880d0720 0x7f84880cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848d415640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f84880d0720 0x7f84880cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35504/0 (socket says 172.21.15.55:35504) 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848d415640 1 -- 172.21.15.55:0/4237824714 learned_addr learned my addr 172.21.15.55:0/4237824714 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848dc16640 1 --2- 172.21.15.55:0/4237824714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84880d0e40 0x7f84880d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848cc14640 1 --2- 172.21.15.55:0/4237824714 >> v2:172.21.15.138:3300/0 conn(0x7f848806dc90 0x7f848806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848d415640 1 -- 172.21.15.55:0/4237824714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84880d0e40 msgr2=0x7f84880d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848d415640 1 --2- 172.21.15.55:0/4237824714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84880d0e40 0x7f84880d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848d415640 1 -- 172.21.15.55:0/4237824714 >> v2:172.21.15.138:3300/0 conn(0x7f848806dc90 msgr2=0x7f848806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.932+0000 7f848d415640 1 --2- 172.21.15.55:0/4237824714 >> v2:172.21.15.138:3300/0 conn(0x7f848806dc90 0x7f848806e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.933+0000 7f848d415640 1 -- 172.21.15.55:0/4237824714 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848815fa40 con 0x7f84880d0720 2024-05-20T22:11:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.933+0000 7f848dc16640 1 --2- 172.21.15.55:0/4237824714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84880d0e40 0x7f84880d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.933+0000 7f848d415640 1 --2- 172.21.15.55:0/4237824714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84880d0720 0x7f84880cf4d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8488071fc0 tx=0x7f848002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c6ca20a2e81b44 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.933+0000 7f8487fff640 1 -- 172.21.15.55:0/4237824714 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480037070 con 0x7f84880d0720 2024-05-20T22:11:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.933+0000 7f8487fff640 1 -- 172.21.15.55:0/4237824714 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8480032440 con 0x7f84880d0720 2024-05-20T22:11:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.933+0000 7f8487fff640 1 -- 172.21.15.55:0/4237824714 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480032810 con 0x7f84880d0720 2024-05-20T22:11:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.934+0000 7f848f679640 1 -- 172.21.15.55:0/4237824714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84880d0720 msgr2=0x7f84880cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.934+0000 7f848f679640 1 --2- 172.21.15.55:0/4237824714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84880d0720 0x7f84880cf4d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8488071fc0 tx=0x7f848002a1e0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.934+0000 7f848f679640 1 -- 172.21.15.55:0/4237824714 shutdown_connections 2024-05-20T22:11:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.934+0000 7f848f679640 1 --2- 172.21.15.55:0/4237824714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84880d0e40 0x7f84880d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.934+0000 7f848f679640 1 --2- 172.21.15.55:0/4237824714 >> v2:172.21.15.138:3300/0 conn(0x7f848806dc90 0x7f848806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.934+0000 7f848f679640 1 --2- 172.21.15.55:0/4237824714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84880d0720 0x7f84880cf4d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.934+0000 7f848f679640 1 -- 172.21.15.55:0/4237824714 >> 172.21.15.55:0/4237824714 conn(0x7f84880b3b60 msgr2=0x7f84880b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.934+0000 7f848f679640 1 -- 172.21.15.55:0/4237824714 shutdown_connections 2024-05-20T22:11:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.934+0000 7f848f679640 1 -- 172.21.15.55:0/4237824714 wait complete. 2024-05-20T22:11:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.936+0000 7f848f679640 1 Processor -- start 2024-05-20T22:11:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.936+0000 7f848f679640 1 -- start start 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.937+0000 7f848f679640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f848806dc90 0x7f8488157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.937+0000 7f848f679640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84880d0720 0x7f84881476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.937+0000 7f848f679640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84880d0e40 0x7f8488158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.937+0000 7f848f679640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f84881602e0 con 0x7f84880d0720 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.937+0000 7f848f679640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8488160000 con 0x7f84880d0e40 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.937+0000 7f848f679640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f84881605c0 con 0x7f848806dc90 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.937+0000 7f848cc14640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84880d0720 0x7f84881476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.937+0000 7f848cc14640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84880d0720 0x7f84881476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35518/0 (socket says 172.21.15.55:35518) 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.937+0000 7f848cc14640 1 -- 172.21.15.55:0/201787673 learned_addr learned my addr 172.21.15.55:0/201787673 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848d415640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f848806dc90 0x7f8488157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848dc16640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84880d0e40 0x7f8488158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848cc14640 1 -- 172.21.15.55:0/201787673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f848806dc90 msgr2=0x7f8488157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848cc14640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f848806dc90 0x7f8488157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848cc14640 1 -- 172.21.15.55:0/201787673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84880d0e40 msgr2=0x7f8488158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848cc14640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84880d0e40 0x7f8488158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848cc14640 1 -- 172.21.15.55:0/201787673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8488157c60 con 0x7f84880d0720 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848d415640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f848806dc90 0x7f8488157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848dc16640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84880d0e40 0x7f8488158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848cc14640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84880d0720 0x7f84881476b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f847c00d5f0 tx=0x7f847c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f8485ffb640 1 -- 172.21.15.55:0/201787673 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0125f0 con 0x7f84880d0720 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f8485ffb640 1 -- 172.21.15.55:0/201787673 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847c019070 con 0x7f84880d0720 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8488157f40 con 0x7f84880d0720 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f8485ffb640 1 -- 172.21.15.55:0/201787673 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0129d0 con 0x7f84880d0720 2024-05-20T22:11:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.938+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8488158500 con 0x7f84880d0720 2024-05-20T22:11:15.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.939+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84881588e0 con 0x7f84880d0720 2024-05-20T22:11:15.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.940+0000 7f8485ffb640 1 -- 172.21.15.55:0/201787673 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f847c012b70 con 0x7f84880d0720 2024-05-20T22:11:15.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.940+0000 7f8485ffb640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f847406be60 0x7f847406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.940+0000 7f8485ffb640 1 -- 172.21.15.55:0/201787673 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f847c08b160 con 0x7f84880d0720 2024-05-20T22:11:15.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.940+0000 7f848d415640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f847406be60 0x7f847406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.940+0000 7f8485ffb640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] conn(0x7f84740723f0 0x7f8474074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:15.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.940+0000 7f8485ffb640 1 -- 172.21.15.55:0/201787673 --> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8474074f50 con 0x7f84740723f0 2024-05-20T22:11:15.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.941+0000 7f8485ffb640 1 -- 172.21.15.55:0/201787673 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f847c092050 con 0x7f84880d0720 2024-05-20T22:11:15.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.941+0000 7f848dc16640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] conn(0x7f84740723f0 0x7f8474074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:15.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.941+0000 7f848d415640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f847406be60 0x7f847406e370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8480034000 tx=0x7f848002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.941+0000 7f848dc16640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] conn(0x7f84740723f0 0x7f8474074880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:15.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.942+0000 7f8485ffb640 1 -- 172.21.15.55:0/201787673 <== osd.2 v2:172.21.15.55:6800/3066480496 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8474074f50 con 0x7f84740723f0 2024-05-20T22:11:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.955+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 --> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8488157a10 con 0x7f84740723f0 2024-05-20T22:11:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.955+0000 7f8485ffb640 1 -- 172.21.15.55:0/201787673 <== osd.2 v2:172.21.15.55:6800/3066480496 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8488157a10 con 0x7f84740723f0 2024-05-20T22:11:15.956 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:11:15.956 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T22:11:15.956 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:11:15.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.955+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] conn(0x7f84740723f0 msgr2=0x7f8474074880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.955+0000 7f848f679640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:6800/3066480496,v1:172.21.15.55:6801/3066480496] conn(0x7f84740723f0 0x7f8474074880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.956+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f847406be60 msgr2=0x7f847406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.956+0000 7f848f679640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f847406be60 0x7f847406e370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8480034000 tx=0x7f848002e000 comp rx=0 tx=0).stop 2024-05-20T22:11:15.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.956+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84880d0720 msgr2=0x7f84881476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:15.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.956+0000 7f848f679640 1 --2- 172.21.15.55:0/201787673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84880d0720 0x7f84881476b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f847c00d5f0 tx=0x7f847c00dab0 comp rx=0 tx=0).stop 2024-05-20T22:11:15.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.956+0000 7f848dc16640 1 -- 172.21.15.55:0/201787673 reap_dead start 2024-05-20T22:11:15.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.956+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 shutdown_connections 2024-05-20T22:11:15.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.956+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 >> 172.21.15.55:0/201787673 conn(0x7f84880b3b60 msgr2=0x7f848815c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:15.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.956+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 shutdown_connections 2024-05-20T22:11:15.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:15.956+0000 7f848f679640 1 -- 172.21.15.55:0/201787673 wait complete. 2024-05-20T22:11:15.967 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-20T22:11:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.082+0000 7f2e8b102640 1 Processor -- start 2024-05-20T22:11:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.082+0000 7f2e8b102640 1 -- start start 2024-05-20T22:11:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e8b102640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2e8415c390 0x7f2e841664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e8b102640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8415f120 con 0x7f2e84072da0 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e8b102640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8415c9b0 con 0x7f2e84155cf0 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e8b102640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2e840b0350 con 0x7f2e8415c390 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e83fff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2e84072da0 0x7f2e8415bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44362/0 (socket says 172.21.15.55:44362) 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e83fff640 1 -- 172.21.15.55:0/305800142 learned_addr learned my addr 172.21.15.55:0/305800142 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e8969f640 1 --2- 172.21.15.55:0/305800142 >> v2:172.21.15.138:3300/0 conn(0x7f2e8415c390 0x7f2e841664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1205377497 0 0) 0x7f2e8415f120 con 0x7f2e84072da0 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e6c003520 con 0x7f2e84072da0 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1388902780 0 0) 0x7f2e6c003520 con 0x7f2e84072da0 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 >> v2:172.21.15.138:3300/0 conn(0x7f2e8415c390 msgr2=0x7f2e841664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 --2- 172.21.15.55:0/305800142 >> v2:172.21.15.138:3300/0 conn(0x7f2e8415c390 0x7f2e841664b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 >> v1:172.21.15.138:6789/0 conn(0x7f2e84155cf0 legacy=0x7f2e84072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8415f3c0 con 0x7f2e84072da0 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f2e78001670 con 0x7f2e84072da0 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 >> v1:172.21.15.55:6789/0 conn(0x7f2e84072da0 legacy=0x7f2e8415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c0039b0 0x7f2e6c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.084+0000 7f2e837fe640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e6c004800 0x7f2e6c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e837fe640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6c0051c0 0x7f2e6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2e840b0350 con 0x7f2e6c0051c0 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c00c160 con 0x7f2e6c0039b0 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6c00c360 con 0x7f2e6c004800 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e8969f640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6c0051c0 0x7f2e6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e88e9e640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c0039b0 0x7f2e6c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e83fff640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e6c004800 0x7f2e6c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e8969f640 1 -- 172.21.15.55:0/305800142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e6c004800 msgr2=0x7f2e6c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e8969f640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e6c004800 0x7f2e6c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e8969f640 1 -- 172.21.15.55:0/305800142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c0039b0 msgr2=0x7f2e6c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e8969f640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c0039b0 0x7f2e6c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.085+0000 7f2e8969f640 1 -- 172.21.15.55:0/305800142 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c003520 con 0x7f2e6c0051c0 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.086+0000 7f2e8969f640 1 -- 172.21.15.55:0/305800142 reap_dead start 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.086+0000 7f2e8969f640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6c0051c0 0x7f2e6c00b9c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2e740098f0 tx=0x7f2e6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7483aa890e88b1c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.086+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7400f040 con 0x7f2e6c0051c0 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.086+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e74004220 con 0x7f2e6c0051c0 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.086+0000 7f2e837fe640 1 -- 172.21.15.55:0/305800142 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74003610 con 0x7f2e6c0051c0 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.086+0000 7f2e8b102640 1 -- 172.21.15.55:0/305800142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6c0051c0 msgr2=0x7f2e6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.086+0000 7f2e8b102640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6c0051c0 0x7f2e6c00b9c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2e740098f0 tx=0x7f2e6c005830 comp rx=0 tx=0).stop 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.087+0000 7f2e8b102640 1 -- 172.21.15.55:0/305800142 shutdown_connections 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.087+0000 7f2e8b102640 1 --2- 172.21.15.55:0/305800142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6c0051c0 0x7f2e6c00b9c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.087+0000 7f2e8b102640 1 -- 172.21.15.55:0/305800142 >> 172.21.15.55:0/305800142 conn(0x7f2e840b3980 msgr2=0x7f2e840b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.087+0000 7f2e8b102640 1 -- 172.21.15.55:0/305800142 shutdown_connections 2024-05-20T22:11:16.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.087+0000 7f2e8b102640 1 -- 172.21.15.55:0/305800142 wait complete. 2024-05-20T22:11:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.088+0000 7f2e8b102640 1 Processor -- start 2024-05-20T22:11:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.088+0000 7f2e8b102640 1 -- start start 2024-05-20T22:11:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e8b102640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c0051c0 0x7f2e840b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e8b102640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8414c860 0x7f2e841f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e8b102640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e841f5900 0x7f2e8415a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e8b102640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8415fea0 con 0x7f2e8414c860 2024-05-20T22:11:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e8b102640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8415fa10 con 0x7f2e6c0051c0 2024-05-20T22:11:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e8b102640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e84160020 con 0x7f2e841f5900 2024-05-20T22:11:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8414c860 0x7f2e841f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8414c860 0x7f2e841f53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35530/0 (socket says 172.21.15.55:35530) 2024-05-20T22:11:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e83fff640 1 -- 172.21.15.55:0/2148187435 learned_addr learned my addr 172.21.15.55:0/2148187435 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e88e9e640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c0051c0 0x7f2e840b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e8969f640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e841f5900 0x7f2e8415a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e83fff640 1 -- 172.21.15.55:0/2148187435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e841f5900 msgr2=0x7f2e8415a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e83fff640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e841f5900 0x7f2e8415a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e83fff640 1 -- 172.21.15.55:0/2148187435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c0051c0 msgr2=0x7f2e840b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e83fff640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c0051c0 0x7f2e840b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e83fff640 1 -- 172.21.15.55:0/2148187435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e841f6180 con 0x7f2e8414c860 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e88e9e640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c0051c0 0x7f2e840b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.090+0000 7f2e8969f640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e841f5900 0x7f2e8415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.091+0000 7f2e83fff640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8414c860 0x7f2e841f53c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2e78004fa0 tx=0x7f2e780074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.091+0000 7f2e817fa640 1 -- 172.21.15.55:0/2148187435 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78007740 con 0x7f2e8414c860 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.091+0000 7f2e817fa640 1 -- 172.21.15.55:0/2148187435 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e78018070 con 0x7f2e8414c860 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.091+0000 7f2e8b102640 1 -- 172.21.15.55:0/2148187435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e840b04d0 con 0x7f2e8414c860 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.091+0000 7f2e817fa640 1 -- 172.21.15.55:0/2148187435 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800fb70 con 0x7f2e8414c860 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.091+0000 7f2e8b102640 1 -- 172.21.15.55:0/2148187435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e840b0a90 con 0x7f2e8414c860 2024-05-20T22:11:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.092+0000 7f2e8b102640 1 -- 172.21.15.55:0/2148187435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e84069090 con 0x7f2e8414c860 2024-05-20T22:11:16.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.092+0000 7f2e817fa640 1 -- 172.21.15.55:0/2148187435 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e7800fd10 con 0x7f2e8414c860 2024-05-20T22:11:16.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.093+0000 7f2e817fa640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e7006c000 0x7f2e7006e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.093+0000 7f2e817fa640 1 -- 172.21.15.55:0/2148187435 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f2e7808aef0 con 0x7f2e8414c860 2024-05-20T22:11:16.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.093+0000 7f2e88e9e640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e7006c000 0x7f2e7006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.093+0000 7f2e88e9e640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e7006c000 0x7f2e7006e510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000f80 tx=0x7f2e7c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.098+0000 7f2e817fa640 1 -- 172.21.15.55:0/2148187435 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2e78059890 con 0x7f2e8414c860 2024-05-20T22:11:16.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.286+0000 7f2e8b102640 1 -- 172.21.15.55:0/2148187435 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f2e841f5f10 con 0x7f2e7006c000 2024-05-20T22:11:16.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.286+0000 7f2e817fa640 1 -- 172.21.15.55:0/2148187435 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2e841f5f10 con 0x7f2e7006c000 2024-05-20T22:11:16.287 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 2 to deep-scrub 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.288+0000 7f2e8b102640 1 -- 172.21.15.55:0/2148187435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e7006c000 msgr2=0x7f2e7006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.288+0000 7f2e8b102640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e7006c000 0x7f2e7006e510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000f80 tx=0x7f2e7c009040 comp rx=0 tx=0).stop 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.288+0000 7f2e8b102640 1 -- 172.21.15.55:0/2148187435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8414c860 msgr2=0x7f2e841f53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.288+0000 7f2e8b102640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8414c860 0x7f2e841f53c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2e78004fa0 tx=0x7f2e780074f0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.289+0000 7f2e8b102640 1 -- 172.21.15.55:0/2148187435 shutdown_connections 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.289+0000 7f2e8b102640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f2e7006c000 0x7f2e7006e510 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.289+0000 7f2e8b102640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e841f5900 0x7f2e8415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.289+0000 7f2e8b102640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e8414c860 0x7f2e841f53c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.289+0000 7f2e8b102640 1 --2- 172.21.15.55:0/2148187435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c0051c0 0x7f2e840b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.289+0000 7f2e8b102640 1 -- 172.21.15.55:0/2148187435 >> 172.21.15.55:0/2148187435 conn(0x7f2e840b3980 msgr2=0x7f2e8406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:16.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.289+0000 7f2e8b102640 1 -- 172.21.15.55:0/2148187435 shutdown_connections 2024-05-20T22:11:16.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.289+0000 7f2e8b102640 1 -- 172.21.15.55:0/2148187435 wait complete. 2024-05-20T22:11:16.301 INFO:tasks.ceph:Scrubbing osd.3 2024-05-20T22:11:16.301 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-20T22:11:16.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.419+0000 7f16ca3c3640 1 Processor -- start 2024-05-20T22:11:16.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.419+0000 7f16ca3c3640 1 -- start start 2024-05-20T22:11:16.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16ca3c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16c40cd520 0x7f16c408c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16ca3c3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f16c40d0860 0x7f16c40b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16ca3c3640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f16c40cf270 0x7f16c40cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16ca3c3640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f16c40cfbc0 con 0x7f16c40cf270 2024-05-20T22:11:16.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16ca3c3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f16c40b1f30 con 0x7f16c40d0860 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16ca3c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f16c40d1020 con 0x7f16c40cd520 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16c8960640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f16c40cf270 0x7f16c40cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16c8960640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f16c40cf270 0x7f16c40cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35536/0 (socket says 172.21.15.55:35536) 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16c8960640 1 -- 172.21.15.55:0/1579816630 learned_addr learned my addr 172.21.15.55:0/1579816630 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16c37fe640 1 --2- 172.21.15.55:0/1579816630 >> v2:172.21.15.138:3300/0 conn(0x7f16c40d0860 0x7f16c40b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.420+0000 7f16c3fff640 1 --2- 172.21.15.55:0/1579816630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16c40cd520 0x7f16c408c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.421+0000 7f16c8960640 1 -- 172.21.15.55:0/1579816630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16c40cd520 msgr2=0x7f16c408c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.421+0000 7f16c8960640 1 --2- 172.21.15.55:0/1579816630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16c40cd520 0x7f16c408c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.421+0000 7f16c8960640 1 -- 172.21.15.55:0/1579816630 >> v2:172.21.15.138:3300/0 conn(0x7f16c40d0860 msgr2=0x7f16c40b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.421+0000 7f16c8960640 1 --2- 172.21.15.55:0/1579816630 >> v2:172.21.15.138:3300/0 conn(0x7f16c40d0860 0x7f16c40b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.421+0000 7f16c8960640 1 -- 172.21.15.55:0/1579816630 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c40d1300 con 0x7f16c40cf270 2024-05-20T22:11:16.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.421+0000 7f16c3fff640 1 --2- 172.21.15.55:0/1579816630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16c40cd520 0x7f16c408c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:16.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.421+0000 7f16c8960640 1 --2- 172.21.15.55:0/1579816630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16c40cf270 0x7f16c40cf680 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f16c408cf50 tx=0x7f16bc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfacc2becf9e712 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16c2ffd640 1 -- 172.21.15.55:0/1579816630 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc036070 con 0x7f16c40cf270 2024-05-20T22:11:16.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16c2ffd640 1 -- 172.21.15.55:0/1579816630 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc031600 con 0x7f16c40cf270 2024-05-20T22:11:16.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16c2ffd640 1 -- 172.21.15.55:0/1579816630 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0319d0 con 0x7f16c40cf270 2024-05-20T22:11:16.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16ca3c3640 1 -- 172.21.15.55:0/1579816630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16c40cf270 msgr2=0x7f16c40cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16ca3c3640 1 --2- 172.21.15.55:0/1579816630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16c40cf270 0x7f16c40cf680 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f16c408cf50 tx=0x7f16bc02a610 comp rx=0 tx=0).stop 2024-05-20T22:11:16.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16ca3c3640 1 -- 172.21.15.55:0/1579816630 shutdown_connections 2024-05-20T22:11:16.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16ca3c3640 1 --2- 172.21.15.55:0/1579816630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16c40cf270 0x7f16c40cf680 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16ca3c3640 1 --2- 172.21.15.55:0/1579816630 >> v2:172.21.15.138:3300/0 conn(0x7f16c40d0860 0x7f16c40b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16ca3c3640 1 --2- 172.21.15.55:0/1579816630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16c40cd520 0x7f16c408c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16ca3c3640 1 -- 172.21.15.55:0/1579816630 >> 172.21.15.55:0/1579816630 conn(0x7f16c40d2860 msgr2=0x7f16c40d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:16.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16ca3c3640 1 -- 172.21.15.55:0/1579816630 shutdown_connections 2024-05-20T22:11:16.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.422+0000 7f16ca3c3640 1 -- 172.21.15.55:0/1579816630 wait complete. 2024-05-20T22:11:16.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.424+0000 7f16ca3c3640 1 Processor -- start 2024-05-20T22:11:16.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.424+0000 7f16ca3c3640 1 -- start start 2024-05-20T22:11:16.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16ca3c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16c40cd520 0x7f16c409fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16ca3c3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16c40cf270 0x7f16c414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16ca3c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c40d0860 0x7f16c415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16ca3c3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f16c40b2cb0 con 0x7f16c40cf270 2024-05-20T22:11:16.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16ca3c3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16c40d18c0 con 0x7f16c40d0860 2024-05-20T22:11:16.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16ca3c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f16c40d0c70 con 0x7f16c40cd520 2024-05-20T22:11:16.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16c40cf270 0x7f16c414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16c40cf270 0x7f16c414ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35540/0 (socket says 172.21.15.55:35540) 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c37fe640 1 -- 172.21.15.55:0/548103815 learned_addr learned my addr 172.21.15.55:0/548103815 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c8960640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c40d0860 0x7f16c415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c3fff640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16c40cd520 0x7f16c409fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c37fe640 1 -- 172.21.15.55:0/548103815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16c40cd520 msgr2=0x7f16c409fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c37fe640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f16c40cd520 0x7f16c409fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c37fe640 1 -- 172.21.15.55:0/548103815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c40d0860 msgr2=0x7f16c415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c37fe640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c40d0860 0x7f16c415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c37fe640 1 -- 172.21.15.55:0/548103815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c40a0350 con 0x7f16c40cf270 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c37fe640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16c40cf270 0x7f16c414ee10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f16b800e8a0 tx=0x7f16b800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.426+0000 7f16c0ff9640 1 -- 172.21.15.55:0/548103815 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8017070 con 0x7f16c40cf270 2024-05-20T22:11:16.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.427+0000 7f16c0ff9640 1 -- 172.21.15.55:0/548103815 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b800ee90 con 0x7f16c40cf270 2024-05-20T22:11:16.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.427+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c414fb40 con 0x7f16c40cf270 2024-05-20T22:11:16.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.427+0000 7f16c0ff9640 1 -- 172.21.15.55:0/548103815 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b800cb60 con 0x7f16c40cf270 2024-05-20T22:11:16.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.427+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c414cf40 con 0x7f16c40cf270 2024-05-20T22:11:16.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.428+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16c414d600 con 0x7f16c40cf270 2024-05-20T22:11:16.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.428+0000 7f16c0ff9640 1 -- 172.21.15.55:0/548103815 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f16b801a070 con 0x7f16c40cf270 2024-05-20T22:11:16.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.429+0000 7f16c0ff9640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f16b006be60 0x7f16b006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.429+0000 7f16c0ff9640 1 -- 172.21.15.55:0/548103815 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f16b808aaf0 con 0x7f16c40cf270 2024-05-20T22:11:16.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.429+0000 7f16c3fff640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f16b006be60 0x7f16b006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.429+0000 7f16c0ff9640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x7f16b00723f0 0x7f16b0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.429+0000 7f16c0ff9640 1 -- 172.21.15.55:0/548103815 --> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16b8022e50 con 0x7f16b00723f0 2024-05-20T22:11:16.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.429+0000 7f16c0ff9640 1 -- 172.21.15.55:0/548103815 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f16b808f2c0 con 0x7f16c40cf270 2024-05-20T22:11:16.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.429+0000 7f16c8960640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x7f16b00723f0 0x7f16b0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.429+0000 7f16c3fff640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f16b006be60 0x7f16b006e370 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f16b4003610 tx=0x7f16b4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.430+0000 7f16c8960640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x7f16b00723f0 0x7f16b0074880 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.430+0000 7f16c0ff9640 1 -- 172.21.15.55:0/548103815 <== osd.3 v2:172.21.15.55:6808/3969115886 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16b8022e50 con 0x7f16b00723f0 2024-05-20T22:11:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 --> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f16c414f8f0 con 0x7f16b00723f0 2024-05-20T22:11:16.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16c0ff9640 1 -- 172.21.15.55:0/548103815 <== osd.3 v2:172.21.15.55:6808/3969115886 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f16c414f8f0 con 0x7f16b00723f0 2024-05-20T22:11:16.445 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:11:16.445 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T22:11:16.445 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:11:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x7f16b00723f0 msgr2=0x7f16b0074880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:6808/3969115886,v1:172.21.15.55:6809/3969115886] conn(0x7f16b00723f0 0x7f16b0074880 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f16b006be60 msgr2=0x7f16b006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f16b006be60 0x7f16b006e370 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f16b4003610 tx=0x7f16b4008040 comp rx=0 tx=0).stop 2024-05-20T22:11:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16c40cf270 msgr2=0x7f16c414ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 --2- 172.21.15.55:0/548103815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16c40cf270 0x7f16c414ee10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f16b800e8a0 tx=0x7f16b800ed60 comp rx=0 tx=0).stop 2024-05-20T22:11:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16c8960640 1 -- 172.21.15.55:0/548103815 reap_dead start 2024-05-20T22:11:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 shutdown_connections 2024-05-20T22:11:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 >> 172.21.15.55:0/548103815 conn(0x7f16c40d2860 msgr2=0x7f16c415a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 shutdown_connections 2024-05-20T22:11:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.444+0000 7f16ca3c3640 1 -- 172.21.15.55:0/548103815 wait complete. 2024-05-20T22:11:16.456 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-20T22:11:16.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.570+0000 7f5c2b6b8640 1 Processor -- start 2024-05-20T22:11:16.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.570+0000 7f5c2b6b8640 1 -- start start 2024-05-20T22:11:16.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.571+0000 7f5c2b6b8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c240b4080 0x7f5c2415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.571+0000 7f5c2b6b8640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c240b35e0 0x7f5c2415f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.571+0000 7f5c2b6b8640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c2406b500 con 0x7f5c2415fd60 2024-05-20T22:11:16.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.571+0000 7f5c2b6b8640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f5c24163960 con 0x7f5c240b35e0 2024-05-20T22:11:16.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.571+0000 7f5c2b6b8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5c240b3e10 con 0x7f5c240b4080 2024-05-20T22:11:16.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.571+0000 7f5c28c53640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c240b35e0 0x7f5c2415f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.571+0000 7f5c29c55640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5c2415fd60 0x7f5c240cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44368/0 (socket says 172.21.15.55:44368) 2024-05-20T22:11:16.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.571+0000 7f5c29c55640 1 -- 172.21.15.55:0/3317025120 learned_addr learned my addr 172.21.15.55:0/3317025120 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:16.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.572+0000 7f5c29454640 1 --2- 172.21.15.55:0/3317025120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c240b4080 0x7f5c2415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.572+0000 7f5c28c53640 1 -- 172.21.15.55:0/3317025120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c240b4080 msgr2=0x7f5c2415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:16.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.572+0000 7f5c28c53640 1 --2- 172.21.15.55:0/3317025120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c240b4080 0x7f5c2415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.572+0000 7f5c28c53640 1 -- 172.21.15.55:0/3317025120 >> v1:172.21.15.55:6789/0 conn(0x7f5c2415fd60 legacy=0x7f5c240cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.572+0000 7f5c28c53640 1 -- 172.21.15.55:0/3317025120 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c240b3b30 con 0x7f5c240b35e0 2024-05-20T22:11:16.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.572+0000 7f5c1bfff640 1 -- 172.21.15.55:0/3317025120 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693736934 0 0) 0x7f5c1c001490 con 0x7f5c2415fd60 2024-05-20T22:11:16.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.573+0000 7f5c28c53640 1 --2- 172.21.15.55:0/3317025120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c240b35e0 0x7f5c2415f820 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5c140097a0 tx=0x7f5c1402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290de667faec7f79 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.573+0000 7f5c1bfff640 1 -- 172.21.15.55:0/3317025120 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14037070 con 0x7f5c240b35e0 2024-05-20T22:11:16.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.573+0000 7f5c1bfff640 1 -- 172.21.15.55:0/3317025120 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c14032440 con 0x7f5c240b35e0 2024-05-20T22:11:16.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.573+0000 7f5c1bfff640 1 -- 172.21.15.55:0/3317025120 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14032810 con 0x7f5c240b35e0 2024-05-20T22:11:16.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.573+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3317025120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c240b35e0 msgr2=0x7f5c2415f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.573+0000 7f5c2b6b8640 1 --2- 172.21.15.55:0/3317025120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c240b35e0 0x7f5c2415f820 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5c140097a0 tx=0x7f5c1402a180 comp rx=0 tx=0).stop 2024-05-20T22:11:16.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.573+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3317025120 shutdown_connections 2024-05-20T22:11:16.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.573+0000 7f5c2b6b8640 1 --2- 172.21.15.55:0/3317025120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c240b35e0 0x7f5c2415f820 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.573+0000 7f5c2b6b8640 1 --2- 172.21.15.55:0/3317025120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c240b4080 0x7f5c2415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.573+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3317025120 >> 172.21.15.55:0/3317025120 conn(0x7f5c240d51b0 msgr2=0x7f5c240b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:16.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.574+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3317025120 shutdown_connections 2024-05-20T22:11:16.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.574+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3317025120 wait complete. 2024-05-20T22:11:16.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.575+0000 7f5c2b6b8640 1 Processor -- start 2024-05-20T22:11:16.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.575+0000 7f5c2b6b8640 1 -- start start 2024-05-20T22:11:16.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c2b6b8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c240b35e0 0x7f5c2414cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c2b6b8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c240b4080 0x7f5c240ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c2b6b8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c2415fd60 0x7f5c24161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c2b6b8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5c241635f0 con 0x7f5c2415fd60 2024-05-20T22:11:16.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c2b6b8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c241604f0 con 0x7f5c240b35e0 2024-05-20T22:11:16.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c2b6b8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5c240d42a0 con 0x7f5c240b4080 2024-05-20T22:11:16.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c29c55640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c2415fd60 0x7f5c24161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c29c55640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c2415fd60 0x7f5c24161cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35560/0 (socket says 172.21.15.55:35560) 2024-05-20T22:11:16.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c29c55640 1 -- 172.21.15.55:0/3182090115 learned_addr learned my addr 172.21.15.55:0/3182090115 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:16.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c28c53640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c240b4080 0x7f5c240ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c29454640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c240b35e0 0x7f5c2414cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c29c55640 1 -- 172.21.15.55:0/3182090115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c240b4080 msgr2=0x7f5c240ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c29c55640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c240b4080 0x7f5c240ba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c29c55640 1 -- 172.21.15.55:0/3182090115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c240b35e0 msgr2=0x7f5c2414cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c29c55640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c240b35e0 0x7f5c2414cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c29c55640 1 -- 172.21.15.55:0/3182090115 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c240c8130 con 0x7f5c2415fd60 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.577+0000 7f5c29c55640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c2415fd60 0x7f5c24161cf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c6d0 tx=0x7f5c1c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.578+0000 7f5c19ffb640 1 -- 172.21.15.55:0/3182090115 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c018070 con 0x7f5c2415fd60 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.578+0000 7f5c19ffb640 1 -- 172.21.15.55:0/3182090115 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c1c004d60 con 0x7f5c2415fd60 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.578+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3182090115 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c240b8810 con 0x7f5c2415fd60 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.578+0000 7f5c19ffb640 1 -- 172.21.15.55:0/3182090115 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00a880 con 0x7f5c2415fd60 2024-05-20T22:11:16.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.578+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3182090115 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c240b8dd0 con 0x7f5c2415fd60 2024-05-20T22:11:16.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.579+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3182090115 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c24069090 con 0x7f5c2415fd60 2024-05-20T22:11:16.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.580+0000 7f5c19ffb640 1 -- 172.21.15.55:0/3182090115 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c1c01b070 con 0x7f5c2415fd60 2024-05-20T22:11:16.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.580+0000 7f5c19ffb640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5c0806be30 0x7f5c0806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.580+0000 7f5c19ffb640 1 -- 172.21.15.55:0/3182090115 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f5c1c08a460 con 0x7f5c2415fd60 2024-05-20T22:11:16.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.580+0000 7f5c29454640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5c0806be30 0x7f5c0806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.581+0000 7f5c29454640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5c0806be30 0x7f5c0806e340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5c240d0980 tx=0x7f5c20009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.585+0000 7f5c19ffb640 1 -- 172.21.15.55:0/3182090115 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5c2406b500 con 0x7f5c2415fd60 2024-05-20T22:11:16.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.766+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3182090115 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f5c2415eb20 con 0x7f5c0806be30 2024-05-20T22:11:16.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.767+0000 7f5c19ffb640 1 -- 172.21.15.55:0/3182090115 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5c2415eb20 con 0x7f5c0806be30 2024-05-20T22:11:16.768 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 3 to deep-scrub 2024-05-20T22:11:16.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3182090115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5c0806be30 msgr2=0x7f5c0806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5c0806be30 0x7f5c0806e340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5c240d0980 tx=0x7f5c20009190 comp rx=0 tx=0).stop 2024-05-20T22:11:16.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3182090115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c2415fd60 msgr2=0x7f5c24161cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c2415fd60 0x7f5c24161cf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c6d0 tx=0x7f5c1c00cb90 comp rx=0 tx=0).stop 2024-05-20T22:11:16.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3182090115 shutdown_connections 2024-05-20T22:11:16.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f5c0806be30 0x7f5c0806e340 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c2415fd60 0x7f5c24161cf0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c240b4080 0x7f5c240ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 --2- 172.21.15.55:0/3182090115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c240b35e0 0x7f5c2414cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3182090115 >> 172.21.15.55:0/3182090115 conn(0x7f5c240d51b0 msgr2=0x7f5c24157080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:16.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3182090115 shutdown_connections 2024-05-20T22:11:16.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.769+0000 7f5c2b6b8640 1 -- 172.21.15.55:0/3182090115 wait complete. 2024-05-20T22:11:16.781 INFO:tasks.ceph:Scrubbing osd.4 2024-05-20T22:11:16.781 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-20T22:11:16.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.899+0000 7f98ccd3e640 1 Processor -- start 2024-05-20T22:11:16.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.899+0000 7f98ccd3e640 1 -- start start 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.900+0000 7f98ccd3e640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98c806e480 0x7f98c80b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.900+0000 7f98ccd3e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f98c80d1d40 0x7f98c80b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.900+0000 7f98ccd3e640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c8160c50 con 0x7f98c80b2dd0 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.900+0000 7f98ccd3e640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f98c81617f0 con 0x7f98c806e480 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.900+0000 7f98ccd3e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f98c8160ec0 con 0x7f98c80d1d40 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.901+0000 7f98c659c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98c806e480 0x7f98c80b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.901+0000 7f98bffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f98c80b2dd0 0x7f98c80d1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44372/0 (socket says 172.21.15.55:44372) 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.901+0000 7f98bffff640 1 -- 172.21.15.55:0/3707427763 learned_addr learned my addr 172.21.15.55:0/3707427763 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.901+0000 7f98c6d9d640 1 --2- 172.21.15.55:0/3707427763 >> v2:172.21.15.138:3300/0 conn(0x7f98c80d1d40 0x7f98c80b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.901+0000 7f98c659c640 1 -- 172.21.15.55:0/3707427763 >> v2:172.21.15.138:3300/0 conn(0x7f98c80d1d40 msgr2=0x7f98c80b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.901+0000 7f98c659c640 1 --2- 172.21.15.55:0/3707427763 >> v2:172.21.15.138:3300/0 conn(0x7f98c80d1d40 0x7f98c80b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.901+0000 7f98c659c640 1 -- 172.21.15.55:0/3707427763 >> v1:172.21.15.55:6789/0 conn(0x7f98c80b2dd0 legacy=0x7f98c80d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.901+0000 7f98c659c640 1 -- 172.21.15.55:0/3707427763 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c81611a0 con 0x7f98c806e480 2024-05-20T22:11:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.901+0000 7f98c6d9d640 1 --2- 172.21.15.55:0/3707427763 >> v2:172.21.15.138:3300/0 conn(0x7f98c80d1d40 0x7f98c80b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.902+0000 7f98c659c640 1 --2- 172.21.15.55:0/3707427763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98c806e480 0x7f98c80b2890 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f98c80d2860 tx=0x7f98b802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d856057c8dafb9e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.902+0000 7f98c5d9b640 1 -- 172.21.15.55:0/3707427763 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8037070 con 0x7f98c806e480 2024-05-20T22:11:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.902+0000 7f98c5d9b640 1 -- 172.21.15.55:0/3707427763 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b8032440 con 0x7f98c806e480 2024-05-20T22:11:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.902+0000 7f98c5d9b640 1 -- 172.21.15.55:0/3707427763 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8032810 con 0x7f98c806e480 2024-05-20T22:11:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.902+0000 7f98ccd3e640 1 -- 172.21.15.55:0/3707427763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98c806e480 msgr2=0x7f98c80b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.902+0000 7f98ccd3e640 1 --2- 172.21.15.55:0/3707427763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98c806e480 0x7f98c80b2890 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f98c80d2860 tx=0x7f98b802a1a0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.903+0000 7f98ccd3e640 1 -- 172.21.15.55:0/3707427763 shutdown_connections 2024-05-20T22:11:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.903+0000 7f98ccd3e640 1 --2- 172.21.15.55:0/3707427763 >> v2:172.21.15.138:3300/0 conn(0x7f98c80d1d40 0x7f98c80b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.903+0000 7f98ccd3e640 1 --2- 172.21.15.55:0/3707427763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98c806e480 0x7f98c80b2890 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.903+0000 7f98ccd3e640 1 -- 172.21.15.55:0/3707427763 >> 172.21.15.55:0/3707427763 conn(0x7f98c8073830 msgr2=0x7f98c80b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:16.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.903+0000 7f98ccd3e640 1 -- 172.21.15.55:0/3707427763 shutdown_connections 2024-05-20T22:11:16.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.903+0000 7f98ccd3e640 1 -- 172.21.15.55:0/3707427763 wait complete. 2024-05-20T22:11:16.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.904+0000 7f98ccd3e640 1 Processor -- start 2024-05-20T22:11:16.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.905+0000 7f98ccd3e640 1 -- start start 2024-05-20T22:11:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98ccd3e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98c806e480 0x7f98c81f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98ccd3e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c80b2dd0 0x7f98c814ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98ccd3e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c80b06f0 0x7f98c806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98ccd3e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f98c80cde20 con 0x7f98c806e480 2024-05-20T22:11:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98ccd3e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98c806e890 con 0x7f98c80b06f0 2024-05-20T22:11:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98ccd3e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98c81f4480 con 0x7f98c80b2dd0 2024-05-20T22:11:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98c806e480 0x7f98c81f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98c806e480 0x7f98c81f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35578/0 (socket says 172.21.15.55:35578) 2024-05-20T22:11:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c659c640 1 -- 172.21.15.55:0/4005520102 learned_addr learned my addr 172.21.15.55:0/4005520102 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c6d9d640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c80b06f0 0x7f98c806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98bffff640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c80b2dd0 0x7f98c814ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c659c640 1 -- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c80b2dd0 msgr2=0x7f98c814ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c659c640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c80b2dd0 0x7f98c814ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c659c640 1 -- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c80b06f0 msgr2=0x7f98c806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c659c640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c80b06f0 0x7f98c806d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c659c640 1 -- 172.21.15.55:0/4005520102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c814b280 con 0x7f98c806e480 2024-05-20T22:11:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c6d9d640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c80b06f0 0x7f98c806d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T22:11:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.906+0000 7f98c659c640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98c806e480 0x7f98c81f3230 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f98b8000c00 tx=0x7f98b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.907+0000 7f98beffd640 1 -- 172.21.15.55:0/4005520102 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8037040 con 0x7f98c806e480 2024-05-20T22:11:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.907+0000 7f98beffd640 1 -- 172.21.15.55:0/4005520102 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b8042070 con 0x7f98c806e480 2024-05-20T22:11:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.907+0000 7f98beffd640 1 -- 172.21.15.55:0/4005520102 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8030a50 con 0x7f98c806e480 2024-05-20T22:11:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.907+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c815d6b0 con 0x7f98c806e480 2024-05-20T22:11:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.907+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c81f3af0 con 0x7f98c806e480 2024-05-20T22:11:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.908+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98c8069090 con 0x7f98c806e480 2024-05-20T22:11:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.909+0000 7f98beffd640 1 -- 172.21.15.55:0/4005520102 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98b8030bf0 con 0x7f98c806e480 2024-05-20T22:11:16.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.909+0000 7f98beffd640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98a806be30 0x7f98a806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.909+0000 7f98beffd640 1 -- 172.21.15.55:0/4005520102 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f98b80ac3b0 con 0x7f98c806e480 2024-05-20T22:11:16.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.909+0000 7f98bffff640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98a806be30 0x7f98a806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.909+0000 7f98beffd640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] conn(0x7f98a80723c0 0x7f98a8074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.909+0000 7f98beffd640 1 -- 172.21.15.55:0/4005520102 --> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98b8004490 con 0x7f98a80723c0 2024-05-20T22:11:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.909+0000 7f98beffd640 1 -- 172.21.15.55:0/4005520102 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f98b80ac840 con 0x7f98c806e480 2024-05-20T22:11:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.910+0000 7f98bffff640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98a806be30 0x7f98a806e340 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f98c8070090 tx=0x7f98b0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.910+0000 7f98c6d9d640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] conn(0x7f98a80723c0 0x7f98a8074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.910+0000 7f98c6d9d640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] conn(0x7f98a80723c0 0x7f98a8074850 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.911+0000 7f98beffd640 1 -- 172.21.15.55:0/4005520102 <== osd.4 v2:172.21.15.138:6808/3059706956 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98b8004490 con 0x7f98a80723c0 2024-05-20T22:11:16.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.925+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 --> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f98c809a3a0 con 0x7f98a80723c0 2024-05-20T22:11:16.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.925+0000 7f98beffd640 1 -- 172.21.15.55:0/4005520102 <== osd.4 v2:172.21.15.138:6808/3059706956 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f98c809a3a0 con 0x7f98a80723c0 2024-05-20T22:11:16.926 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:11:16.926 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T22:11:16.926 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.925+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] conn(0x7f98a80723c0 msgr2=0x7f98a8074850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.925+0000 7f98ccd3e640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.138:6808/3059706956,v1:172.21.15.138:6809/3059706956] conn(0x7f98a80723c0 0x7f98a8074850 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.926+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98a806be30 msgr2=0x7f98a806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.926+0000 7f98ccd3e640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f98a806be30 0x7f98a806e340 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f98c8070090 tx=0x7f98b0009040 comp rx=0 tx=0).stop 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.926+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98c806e480 msgr2=0x7f98c81f3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.926+0000 7f98ccd3e640 1 --2- 172.21.15.55:0/4005520102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98c806e480 0x7f98c81f3230 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f98b8000c00 tx=0x7f98b80046c0 comp rx=0 tx=0).stop 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.926+0000 7f98c6d9d640 1 -- 172.21.15.55:0/4005520102 reap_dead start 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.926+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 shutdown_connections 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.926+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 >> 172.21.15.55:0/4005520102 conn(0x7f98c8073830 msgr2=0x7f98c815c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.926+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 shutdown_connections 2024-05-20T22:11:16.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:16.926+0000 7f98ccd3e640 1 -- 172.21.15.55:0/4005520102 wait complete. 2024-05-20T22:11:16.938 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-20T22:11:17.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.053+0000 7f8e29887640 1 Processor -- start 2024-05-20T22:11:17.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.053+0000 7f8e29887640 1 -- start start 2024-05-20T22:11:17.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.054+0000 7f8e29887640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8e240b4080 0x7f8e2415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.054+0000 7f8e29887640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e240b35e0 0x7f8e2415f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.054+0000 7f8e29887640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e2415fd60 0x7f8e240cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.054+0000 7f8e29887640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f8e24163960 con 0x7f8e2415fd60 2024-05-20T22:11:17.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.054+0000 7f8e29887640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8e2406b500 con 0x7f8e240b4080 2024-05-20T22:11:17.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.054+0000 7f8e29887640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e240b3e10 con 0x7f8e240b35e0 2024-05-20T22:11:17.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e237fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e2415fd60 0x7f8e240cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e237fe640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e2415fd60 0x7f8e240cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35594/0 (socket says 172.21.15.55:35594) 2024-05-20T22:11:17.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e237fe640 1 -- 172.21.15.55:0/2412660069 learned_addr learned my addr 172.21.15.55:0/2412660069 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:17.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e22ffd640 1 --2- 172.21.15.55:0/2412660069 >> v2:172.21.15.138:3300/0 conn(0x7f8e240b4080 0x7f8e2415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e1bfff640 1 --2- 172.21.15.55:0/2412660069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e240b35e0 0x7f8e2415f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e237fe640 1 -- 172.21.15.55:0/2412660069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e240b35e0 msgr2=0x7f8e2415f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:17.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e237fe640 1 --2- 172.21.15.55:0/2412660069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e240b35e0 0x7f8e2415f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e237fe640 1 -- 172.21.15.55:0/2412660069 >> v2:172.21.15.138:3300/0 conn(0x7f8e240b4080 msgr2=0x7f8e2415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:17.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e237fe640 1 --2- 172.21.15.55:0/2412660069 >> v2:172.21.15.138:3300/0 conn(0x7f8e240b4080 0x7f8e2415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e237fe640 1 -- 172.21.15.55:0/2412660069 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e240b3b30 con 0x7f8e2415fd60 2024-05-20T22:11:17.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.055+0000 7f8e22ffd640 1 --2- 172.21.15.55:0/2412660069 >> v2:172.21.15.138:3300/0 conn(0x7f8e240b4080 0x7f8e2415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.056+0000 7f8e237fe640 1 --2- 172.21.15.55:0/2412660069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e2415fd60 0x7f8e240cf620 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8e2408d7c0 tx=0x7f8e1c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bf6064efa8a1c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.056+0000 7f8e227fc640 1 -- 172.21.15.55:0/2412660069 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c036070 con 0x7f8e2415fd60 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.056+0000 7f8e227fc640 1 -- 172.21.15.55:0/2412660069 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1c031600 con 0x7f8e2415fd60 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.056+0000 7f8e227fc640 1 -- 172.21.15.55:0/2412660069 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0319d0 con 0x7f8e2415fd60 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.057+0000 7f8e29887640 1 -- 172.21.15.55:0/2412660069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e2415fd60 msgr2=0x7f8e240cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.057+0000 7f8e29887640 1 --2- 172.21.15.55:0/2412660069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e2415fd60 0x7f8e240cf620 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8e2408d7c0 tx=0x7f8e1c02a6a0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.057+0000 7f8e29887640 1 -- 172.21.15.55:0/2412660069 shutdown_connections 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.057+0000 7f8e29887640 1 --2- 172.21.15.55:0/2412660069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e2415fd60 0x7f8e240cf620 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.057+0000 7f8e29887640 1 --2- 172.21.15.55:0/2412660069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e240b35e0 0x7f8e2415f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.057+0000 7f8e29887640 1 --2- 172.21.15.55:0/2412660069 >> v2:172.21.15.138:3300/0 conn(0x7f8e240b4080 0x7f8e2415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.057+0000 7f8e29887640 1 -- 172.21.15.55:0/2412660069 >> 172.21.15.55:0/2412660069 conn(0x7f8e240d51b0 msgr2=0x7f8e240b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.057+0000 7f8e29887640 1 -- 172.21.15.55:0/2412660069 shutdown_connections 2024-05-20T22:11:17.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.057+0000 7f8e29887640 1 -- 172.21.15.55:0/2412660069 wait complete. 2024-05-20T22:11:17.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.058+0000 7f8e29887640 1 Processor -- start 2024-05-20T22:11:17.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.058+0000 7f8e29887640 1 -- start start 2024-05-20T22:11:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e29887640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e240b35e0 0x7f8e240cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e29887640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e240b4080 0x7f8e241f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e29887640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e2415fd60 0x7f8e241f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e29887640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241635f0 con 0x7f8e240b4080 2024-05-20T22:11:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e29887640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241604f0 con 0x7f8e240b35e0 2024-05-20T22:11:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e29887640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e240cfb60 con 0x7f8e2415fd60 2024-05-20T22:11:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e1bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e240b4080 0x7f8e241f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e1bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e240b4080 0x7f8e241f2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35610/0 (socket says 172.21.15.55:35610) 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e1bfff640 1 -- 172.21.15.55:0/3113327877 learned_addr learned my addr 172.21.15.55:0/3113327877 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e22ffd640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e240b35e0 0x7f8e240cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e237fe640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e2415fd60 0x7f8e241f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e1bfff640 1 -- 172.21.15.55:0/3113327877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e2415fd60 msgr2=0x7f8e241f8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e1bfff640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e2415fd60 0x7f8e241f8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e1bfff640 1 -- 172.21.15.55:0/3113327877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e240b35e0 msgr2=0x7f8e240cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e1bfff640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e240b35e0 0x7f8e240cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e1bfff640 1 -- 172.21.15.55:0/3113327877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e241f9880 con 0x7f8e240b4080 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.060+0000 7f8e1bfff640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e240b4080 0x7f8e241f2840 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00d800 tx=0x7f8e0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.061+0000 7f8e20ff9640 1 -- 172.21.15.55:0/3113327877 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c018030 con 0x7f8e240b4080 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.061+0000 7f8e20ff9640 1 -- 172.21.15.55:0/3113327877 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e0c0096a0 con 0x7f8e240b4080 2024-05-20T22:11:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.061+0000 7f8e29887640 1 -- 172.21.15.55:0/3113327877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e240cd720 con 0x7f8e240b4080 2024-05-20T22:11:17.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.061+0000 7f8e20ff9640 1 -- 172.21.15.55:0/3113327877 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c009870 con 0x7f8e240b4080 2024-05-20T22:11:17.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.061+0000 7f8e29887640 1 -- 172.21.15.55:0/3113327877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e240cdce0 con 0x7f8e240b4080 2024-05-20T22:11:17.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.062+0000 7f8e29887640 1 -- 172.21.15.55:0/3113327877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e240cd490 con 0x7f8e240b4080 2024-05-20T22:11:17.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.062+0000 7f8e20ff9640 1 -- 172.21.15.55:0/3113327877 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e0c01b070 con 0x7f8e240b4080 2024-05-20T22:11:17.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.063+0000 7f8e20ff9640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8e0406be30 0x7f8e0406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.063+0000 7f8e22ffd640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8e0406be30 0x7f8e0406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.063+0000 7f8e20ff9640 1 -- 172.21.15.55:0/3113327877 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f8e0c08ae90 con 0x7f8e240b4080 2024-05-20T22:11:17.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.064+0000 7f8e22ffd640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8e0406be30 0x7f8e0406e340 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e14006fd0 tx=0x7f8e14008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.068+0000 7f8e20ff9640 1 -- 172.21.15.55:0/3113327877 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8e0c059310 con 0x7f8e240b4080 2024-05-20T22:11:17.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.252+0000 7f8e29887640 1 -- 172.21.15.55:0/3113327877 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f8e24069090 con 0x7f8e0406be30 2024-05-20T22:11:17.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.252+0000 7f8e20ff9640 1 -- 172.21.15.55:0/3113327877 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8e24069090 con 0x7f8e0406be30 2024-05-20T22:11:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 4 to deep-scrub 2024-05-20T22:11:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 -- 172.21.15.55:0/3113327877 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8e0406be30 msgr2=0x7f8e0406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8e0406be30 0x7f8e0406e340 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e14006fd0 tx=0x7f8e14008040 comp rx=0 tx=0).stop 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 -- 172.21.15.55:0/3113327877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e240b4080 msgr2=0x7f8e241f2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e240b4080 0x7f8e241f2840 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00d800 tx=0x7f8e0c00dcc0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 -- 172.21.15.55:0/3113327877 shutdown_connections 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f8e0406be30 0x7f8e0406e340 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e2415fd60 0x7f8e241f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e240b4080 0x7f8e241f2840 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 --2- 172.21.15.55:0/3113327877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e240b35e0 0x7f8e240cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 -- 172.21.15.55:0/3113327877 >> 172.21.15.55:0/3113327877 conn(0x7f8e240d51b0 msgr2=0x7f8e2406b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.254+0000 7f8e29887640 1 -- 172.21.15.55:0/3113327877 shutdown_connections 2024-05-20T22:11:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.255+0000 7f8e29887640 1 -- 172.21.15.55:0/3113327877 wait complete. 2024-05-20T22:11:17.267 INFO:tasks.ceph:Scrubbing osd.5 2024-05-20T22:11:17.267 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-20T22:11:17.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.384+0000 7fe708965640 1 Processor -- start 2024-05-20T22:11:17.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.384+0000 7fe708965640 1 -- start start 2024-05-20T22:11:17.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe708965640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe7040d32a0 0x7fe70415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe708965640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe7040d01b0 0x7fe70406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe708965640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7040d3780 con 0x7fe7040cf690 2024-05-20T22:11:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe708965640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fe7040d3a60 con 0x7fe7040d01b0 2024-05-20T22:11:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe708965640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe7040d3d40 con 0x7fe7040d32a0 2024-05-20T22:11:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe702d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe7040d01b0 0x7fe70406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe702d9d640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe7040d01b0 0x7fe70406b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35614/0 (socket says 172.21.15.55:35614) 2024-05-20T22:11:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe702d9d640 1 -- 172.21.15.55:0/994225232 learned_addr learned my addr 172.21.15.55:0/994225232 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe70259c640 1 --2- 172.21.15.55:0/994225232 >> v2:172.21.15.138:3300/0 conn(0x7fe7040d32a0 0x7fe70415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe702d9d640 1 -- 172.21.15.55:0/994225232 >> v2:172.21.15.138:3300/0 conn(0x7fe7040d32a0 msgr2=0x7fe70415f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe702d9d640 1 --2- 172.21.15.55:0/994225232 >> v2:172.21.15.138:3300/0 conn(0x7fe7040d32a0 0x7fe70415f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe702d9d640 1 -- 172.21.15.55:0/994225232 >> v1:172.21.15.138:6789/0 conn(0x7fe7040cf690 legacy=0x7fe7040cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.386+0000 7fe702d9d640 1 -- 172.21.15.55:0/994225232 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7040d4020 con 0x7fe7040d01b0 2024-05-20T22:11:17.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.387+0000 7fe702d9d640 1 --2- 172.21.15.55:0/994225232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7040d01b0 0x7fe70406b500 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe7040d36b0 tx=0x7fe6f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbdf53483642467 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.387+0000 7fe70159a640 1 -- 172.21.15.55:0/994225232 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4037070 con 0x7fe7040d01b0 2024-05-20T22:11:17.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.387+0000 7fe70159a640 1 -- 172.21.15.55:0/994225232 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f4032440 con 0x7fe7040d01b0 2024-05-20T22:11:17.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.387+0000 7fe70159a640 1 -- 172.21.15.55:0/994225232 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4032810 con 0x7fe7040d01b0 2024-05-20T22:11:17.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.388+0000 7fe708965640 1 -- 172.21.15.55:0/994225232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7040d01b0 msgr2=0x7fe70406b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.388+0000 7fe708965640 1 --2- 172.21.15.55:0/994225232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7040d01b0 0x7fe70406b500 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe7040d36b0 tx=0x7fe6f402a270 comp rx=0 tx=0).stop 2024-05-20T22:11:17.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.388+0000 7fe708965640 1 -- 172.21.15.55:0/994225232 shutdown_connections 2024-05-20T22:11:17.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.388+0000 7fe708965640 1 --2- 172.21.15.55:0/994225232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7040d01b0 0x7fe70406b500 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.388+0000 7fe708965640 1 --2- 172.21.15.55:0/994225232 >> v2:172.21.15.138:3300/0 conn(0x7fe7040d32a0 0x7fe70415f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.388+0000 7fe708965640 1 -- 172.21.15.55:0/994225232 >> 172.21.15.55:0/994225232 conn(0x7fe7040b6de0 msgr2=0x7fe7040d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:17.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.388+0000 7fe708965640 1 -- 172.21.15.55:0/994225232 shutdown_connections 2024-05-20T22:11:17.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.388+0000 7fe708965640 1 -- 172.21.15.55:0/994225232 wait complete. 2024-05-20T22:11:17.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.390+0000 7fe708965640 1 Processor -- start 2024-05-20T22:11:17.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.390+0000 7fe708965640 1 -- start start 2024-05-20T22:11:17.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.391+0000 7fe708965640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040cf690 0x7fe70414a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.391+0000 7fe708965640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7040d01b0 0x7fe704160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.391+0000 7fe708965640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7040d32a0 0x7fe7041e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.391+0000 7fe708965640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe7040d4820 con 0x7fe7040d32a0 2024-05-20T22:11:17.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.391+0000 7fe708965640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7040b4350 con 0x7fe7040cf690 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.391+0000 7fe708965640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe704161740 con 0x7fe7040d01b0 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.391+0000 7fe702d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7040d32a0 0x7fe7041e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.391+0000 7fe702d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7040d32a0 0x7fe7041e78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35622/0 (socket says 172.21.15.55:35622) 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.391+0000 7fe702d9d640 1 -- 172.21.15.55:0/2450034281 learned_addr learned my addr 172.21.15.55:0/2450034281 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe70259c640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040cf690 0x7fe70414a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe702d9d640 1 -- 172.21.15.55:0/2450034281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7040d01b0 msgr2=0x7fe704160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe702d9d640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7040d01b0 0x7fe704160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe702d9d640 1 -- 172.21.15.55:0/2450034281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040cf690 msgr2=0x7fe70414a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe702d9d640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040cf690 0x7fe70414a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe702d9d640 1 -- 172.21.15.55:0/2450034281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe704161990 con 0x7fe7040d32a0 2024-05-20T22:11:17.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe702d9d640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7040d32a0 0x7fe7041e78b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40090b0 tx=0x7fe6f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe6faffd640 1 -- 172.21.15.55:0/2450034281 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4037040 con 0x7fe7040d32a0 2024-05-20T22:11:17.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe6faffd640 1 -- 172.21.15.55:0/2450034281 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f4042070 con 0x7fe7040d32a0 2024-05-20T22:11:17.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70414b550 con 0x7fe7040d32a0 2024-05-20T22:11:17.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe6faffd640 1 -- 172.21.15.55:0/2450034281 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4030920 con 0x7fe7040d32a0 2024-05-20T22:11:17.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.392+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70414bb10 con 0x7fe7040d32a0 2024-05-20T22:11:17.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.393+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe704069090 con 0x7fe7040d32a0 2024-05-20T22:11:17.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.394+0000 7fe6faffd640 1 -- 172.21.15.55:0/2450034281 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6f4030ac0 con 0x7fe7040d32a0 2024-05-20T22:11:17.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.394+0000 7fe6faffd640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe6e406be30 0x7fe6e406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.394+0000 7fe6faffd640 1 -- 172.21.15.55:0/2450034281 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7fe6f40ab670 con 0x7fe7040d32a0 2024-05-20T22:11:17.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.394+0000 7fe70259c640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe6e406be30 0x7fe6e406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.394+0000 7fe6faffd640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] conn(0x7fe6e40723c0 0x7fe6e4074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.394+0000 7fe6faffd640 1 -- 172.21.15.55:0/2450034281 --> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6e4074f20 con 0x7fe6e40723c0 2024-05-20T22:11:17.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.394+0000 7fe6faffd640 1 -- 172.21.15.55:0/2450034281 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe6f40b2050 con 0x7fe7040d32a0 2024-05-20T22:11:17.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.394+0000 7fe701d9b640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] conn(0x7fe6e40723c0 0x7fe6e4074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.395+0000 7fe70259c640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe6e406be30 0x7fe6e406e340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009e90 tx=0x7fe6fc00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.395+0000 7fe701d9b640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] conn(0x7fe6e40723c0 0x7fe6e4074850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.396+0000 7fe6faffd640 1 -- 172.21.15.55:0/2450034281 <== osd.5 v2:172.21.15.138:6800/2873568230 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6e4074f20 con 0x7fe6e40723c0 2024-05-20T22:11:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.409+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 --> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe7040d2e50 con 0x7fe6e40723c0 2024-05-20T22:11:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.409+0000 7fe6faffd640 1 -- 172.21.15.55:0/2450034281 <== osd.5 v2:172.21.15.138:6800/2873568230 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe7040d2e50 con 0x7fe6e40723c0 2024-05-20T22:11:17.410 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:11:17.411 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T22:11:17.411 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:11:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 >> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] conn(0x7fe6e40723c0 msgr2=0x7fe6e4074850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe708965640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.138:6800/2873568230,v1:172.21.15.138:6801/2873568230] conn(0x7fe6e40723c0 0x7fe6e4074850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe6e406be30 msgr2=0x7fe6e406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe708965640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fe6e406be30 0x7fe6e406e340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009e90 tx=0x7fe6fc00b040 comp rx=0 tx=0).stop 2024-05-20T22:11:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7040d32a0 msgr2=0x7fe7041e78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe708965640 1 --2- 172.21.15.55:0/2450034281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7040d32a0 0x7fe7041e78b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40090b0 tx=0x7fe6f40046c0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe702d9d640 1 -- 172.21.15.55:0/2450034281 reap_dead start 2024-05-20T22:11:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 shutdown_connections 2024-05-20T22:11:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 >> 172.21.15.55:0/2450034281 conn(0x7fe7040b6de0 msgr2=0x7fe70406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 shutdown_connections 2024-05-20T22:11:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.410+0000 7fe708965640 1 -- 172.21.15.55:0/2450034281 wait complete. 2024-05-20T22:11:17.422 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-20T22:11:17.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.537+0000 7fb75dad8640 1 Processor -- start 2024-05-20T22:11:17.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.537+0000 7fb75dad8640 1 -- start start 2024-05-20T22:11:17.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb75dad8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 0x7fb7580cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb75dad8640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7580d3780 con 0x7fb7580d32a0 2024-05-20T22:11:17.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb75dad8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7580d3a60 con 0x7fb75806b500 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb75dad8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7580d3d40 con 0x7fb75815f150 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb7577fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb7580d32a0 0x7fb7580cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44378/0 (socket says 172.21.15.55:44378) 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb7577fe640 1 -- 172.21.15.55:0/2534111057 learned_addr learned my addr 172.21.15.55:0/2534111057 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb756ffd640 1 --2- 172.21.15.55:0/2534111057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 0x7fb7580cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb7567fc640 1 -- 172.21.15.55:0/2534111057 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3096150500 0 0) 0x7fb7580d3780 con 0x7fb7580d32a0 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb7567fc640 1 -- 172.21.15.55:0/2534111057 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb740003520 con 0x7fb7580d32a0 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb7567fc640 1 -- 172.21.15.55:0/2534111057 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189384487 0 0) 0x7fb7580d3a60 con 0x7fb75806b500 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb7567fc640 1 -- 172.21.15.55:0/2534111057 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7580d3780 con 0x7fb75806b500 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb756ffd640 1 -- 172.21.15.55:0/2534111057 >> v1:172.21.15.138:6789/0 conn(0x7fb75806b500 legacy=0x7fb75806b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb756ffd640 1 -- 172.21.15.55:0/2534111057 >> v1:172.21.15.55:6789/0 conn(0x7fb7580d32a0 legacy=0x7fb7580cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.539+0000 7fb756ffd640 1 -- 172.21.15.55:0/2534111057 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7580d4020 con 0x7fb75815f150 2024-05-20T22:11:17.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.540+0000 7fb756ffd640 1 --2- 172.21.15.55:0/2534111057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 0x7fb7580cfda0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0097d0 tx=0x7fb74c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8f9006e238245 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.540+0000 7fb7567fc640 1 -- 172.21.15.55:0/2534111057 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c037070 con 0x7fb75815f150 2024-05-20T22:11:17.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.540+0000 7fb7567fc640 1 -- 172.21.15.55:0/2534111057 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c032440 con 0x7fb75815f150 2024-05-20T22:11:17.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.540+0000 7fb7567fc640 1 -- 172.21.15.55:0/2534111057 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0327f0 con 0x7fb75815f150 2024-05-20T22:11:17.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.541+0000 7fb75dad8640 1 -- 172.21.15.55:0/2534111057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 msgr2=0x7fb7580cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.541+0000 7fb75dad8640 1 --2- 172.21.15.55:0/2534111057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 0x7fb7580cfda0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0097d0 tx=0x7fb74c02a210 comp rx=0 tx=0).stop 2024-05-20T22:11:17.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.541+0000 7fb75dad8640 1 -- 172.21.15.55:0/2534111057 shutdown_connections 2024-05-20T22:11:17.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.541+0000 7fb75dad8640 1 --2- 172.21.15.55:0/2534111057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 0x7fb7580cfda0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.541+0000 7fb75dad8640 1 -- 172.21.15.55:0/2534111057 >> 172.21.15.55:0/2534111057 conn(0x7fb7580b6de0 msgr2=0x7fb7580d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:17.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.541+0000 7fb75dad8640 1 -- 172.21.15.55:0/2534111057 shutdown_connections 2024-05-20T22:11:17.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.541+0000 7fb75dad8640 1 -- 172.21.15.55:0/2534111057 wait complete. 2024-05-20T22:11:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.543+0000 7fb75dad8640 1 Processor -- start 2024-05-20T22:11:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.543+0000 7fb75dad8640 1 -- start start 2024-05-20T22:11:17.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.544+0000 7fb75dad8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb75806b500 0x7fb758155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.544+0000 7fb75dad8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7580d32a0 0x7fb7580c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.544+0000 7fb75dad8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 0x7fb7580caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.544+0000 7fb75dad8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb7580d4820 con 0x7fb7580d32a0 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.544+0000 7fb75dad8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb75815f560 con 0x7fb75806b500 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.544+0000 7fb75dad8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7580cd390 con 0x7fb75815f150 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.544+0000 7fb756ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7580d32a0 0x7fb7580c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.544+0000 7fb756ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7580d32a0 0x7fb7580c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35638/0 (socket says 172.21.15.55:35638) 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.544+0000 7fb756ffd640 1 -- 172.21.15.55:0/2938619593 learned_addr learned my addr 172.21.15.55:0/2938619593 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb7577fe640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb75806b500 0x7fb758155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb757fff640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 0x7fb7580caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb756ffd640 1 -- 172.21.15.55:0/2938619593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 msgr2=0x7fb7580caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb756ffd640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 0x7fb7580caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb756ffd640 1 -- 172.21.15.55:0/2938619593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb75806b500 msgr2=0x7fb758155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb756ffd640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb75806b500 0x7fb758155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb756ffd640 1 -- 172.21.15.55:0/2938619593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7580c9250 con 0x7fb7580d32a0 2024-05-20T22:11:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb756ffd640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7580d32a0 0x7fb7580c8790 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb74c035040 tx=0x7fb74c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb73ffff640 1 -- 172.21.15.55:0/2938619593 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c037050 con 0x7fb7580d32a0 2024-05-20T22:11:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb73ffff640 1 -- 172.21.15.55:0/2938619593 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c043070 con 0x7fb7580d32a0 2024-05-20T22:11:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb73ffff640 1 -- 172.21.15.55:0/2938619593 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c03ca10 con 0x7fb7580d32a0 2024-05-20T22:11:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.545+0000 7fb75dad8640 1 -- 172.21.15.55:0/2938619593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7580c9530 con 0x7fb7580d32a0 2024-05-20T22:11:17.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.546+0000 7fb75dad8640 1 -- 172.21.15.55:0/2938619593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7580c9af0 con 0x7fb7580d32a0 2024-05-20T22:11:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.547+0000 7fb75dad8640 1 -- 172.21.15.55:0/2938619593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7580c8fc0 con 0x7fb7580d32a0 2024-05-20T22:11:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.547+0000 7fb73ffff640 1 -- 172.21.15.55:0/2938619593 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb74c03cbb0 con 0x7fb7580d32a0 2024-05-20T22:11:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.548+0000 7fb73ffff640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb74406be60 0x7fb74406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.548+0000 7fb73ffff640 1 -- 172.21.15.55:0/2938619593 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7fb74c0ab1d0 con 0x7fb7580d32a0 2024-05-20T22:11:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.548+0000 7fb7577fe640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb74406be60 0x7fb74406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.548+0000 7fb7577fe640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb74406be60 0x7fb74406e370 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb748000f60 tx=0x7fb748007d70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.552+0000 7fb73ffff640 1 -- 172.21.15.55:0/2938619593 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb74c04c060 con 0x7fb7580d32a0 2024-05-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.738+0000 7fb75dad8640 1 -- 172.21.15.55:0/2938619593 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fb758069090 con 0x7fb74406be60 2024-05-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.738+0000 7fb73ffff640 1 -- 172.21.15.55:0/2938619593 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb758069090 con 0x7fb74406be60 2024-05-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 5 to deep-scrub 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 -- 172.21.15.55:0/2938619593 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb74406be60 msgr2=0x7fb74406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb74406be60 0x7fb74406e370 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb748000f60 tx=0x7fb748007d70 comp rx=0 tx=0).stop 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 -- 172.21.15.55:0/2938619593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7580d32a0 msgr2=0x7fb7580c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7580d32a0 0x7fb7580c8790 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb74c035040 tx=0x7fb74c0046c0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 -- 172.21.15.55:0/2938619593 shutdown_connections 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb74406be60 0x7fb74406e370 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb75815f150 0x7fb7580caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7580d32a0 0x7fb7580c8790 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 --2- 172.21.15.55:0/2938619593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb75806b500 0x7fb758155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 -- 172.21.15.55:0/2938619593 >> 172.21.15.55:0/2938619593 conn(0x7fb7580b6de0 msgr2=0x7fb7580d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.740+0000 7fb75dad8640 1 -- 172.21.15.55:0/2938619593 shutdown_connections 2024-05-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.741+0000 7fb75dad8640 1 -- 172.21.15.55:0/2938619593 wait complete. 2024-05-20T22:11:17.753 INFO:tasks.ceph:Scrubbing osd.6 2024-05-20T22:11:17.753 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-20T22:11:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.871+0000 7fb4e4daa640 1 Processor -- start 2024-05-20T22:11:17.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.871+0000 7fb4e4daa640 1 -- start start 2024-05-20T22:11:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.872+0000 7fb4e4daa640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb4e00b29e0 0x7fb4e015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.872+0000 7fb4e4daa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4e00cf7f0 0x7fb4e00d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.872+0000 7fb4e4daa640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4e008c690 con 0x7fb4e008bcb0 2024-05-20T22:11:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.872+0000 7fb4e4daa640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e00b2df0 con 0x7fb4e00b29e0 2024-05-20T22:11:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.872+0000 7fb4e4daa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e008c970 con 0x7fb4e00cf7f0 2024-05-20T22:11:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.872+0000 7fb4de59c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb4e00b29e0 0x7fb4e015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.872+0000 7fb4de59c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb4e00b29e0 0x7fb4e015f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35640/0 (socket says 172.21.15.55:35640) 2024-05-20T22:11:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.872+0000 7fb4de59c640 1 -- 172.21.15.55:0/2662899179 learned_addr learned my addr 172.21.15.55:0/2662899179 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.873+0000 7fb4ded9d640 1 --2- 172.21.15.55:0/2662899179 >> v2:172.21.15.138:3300/0 conn(0x7fb4e00cf7f0 0x7fb4e00d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.873+0000 7fb4de59c640 1 -- 172.21.15.55:0/2662899179 >> v2:172.21.15.138:3300/0 conn(0x7fb4e00cf7f0 msgr2=0x7fb4e00d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.873+0000 7fb4de59c640 1 --2- 172.21.15.55:0/2662899179 >> v2:172.21.15.138:3300/0 conn(0x7fb4e00cf7f0 0x7fb4e00d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.873+0000 7fb4de59c640 1 -- 172.21.15.55:0/2662899179 >> v1:172.21.15.138:6789/0 conn(0x7fb4e008bcb0 legacy=0x7fb4e00cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.873+0000 7fb4de59c640 1 -- 172.21.15.55:0/2662899179 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e00d0de0 con 0x7fb4e00b29e0 2024-05-20T22:11:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.873+0000 7fb4de59c640 1 --2- 172.21.15.55:0/2662899179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e00b29e0 0x7fb4e015f0e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00b88e0 tx=0x7fb4d802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404992d03b6628aa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4dd59a640 1 -- 172.21.15.55:0/2662899179 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8036070 con 0x7fb4e00b29e0 2024-05-20T22:11:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4dd59a640 1 -- 172.21.15.55:0/2662899179 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d8031600 con 0x7fb4e00b29e0 2024-05-20T22:11:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4dd59a640 1 -- 172.21.15.55:0/2662899179 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80319d0 con 0x7fb4e00b29e0 2024-05-20T22:11:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4e4daa640 1 -- 172.21.15.55:0/2662899179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e00b29e0 msgr2=0x7fb4e015f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4e4daa640 1 --2- 172.21.15.55:0/2662899179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e00b29e0 0x7fb4e015f0e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00b88e0 tx=0x7fb4d802a730 comp rx=0 tx=0).stop 2024-05-20T22:11:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4e4daa640 1 -- 172.21.15.55:0/2662899179 shutdown_connections 2024-05-20T22:11:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4e4daa640 1 --2- 172.21.15.55:0/2662899179 >> v2:172.21.15.138:3300/0 conn(0x7fb4e00cf7f0 0x7fb4e00d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4e4daa640 1 --2- 172.21.15.55:0/2662899179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e00b29e0 0x7fb4e015f0e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4e4daa640 1 -- 172.21.15.55:0/2662899179 >> 172.21.15.55:0/2662899179 conn(0x7fb4e00d2860 msgr2=0x7fb4e00d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:17.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4e4daa640 1 -- 172.21.15.55:0/2662899179 shutdown_connections 2024-05-20T22:11:17.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.874+0000 7fb4e4daa640 1 -- 172.21.15.55:0/2662899179 wait complete. 2024-05-20T22:11:17.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.876+0000 7fb4e4daa640 1 Processor -- start 2024-05-20T22:11:17.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.876+0000 7fb4e4daa640 1 -- start start 2024-05-20T22:11:17.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4e4daa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4e008bcb0 0x7fb4e01552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4e4daa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4e00b29e0 0x7fb4e00afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4e4daa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e0157dd0 0x7fb4e00b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4e4daa640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e008cc50 con 0x7fb4e0157dd0 2024-05-20T22:11:17.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4e4daa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e015f620 con 0x7fb4e008bcb0 2024-05-20T22:11:17.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4e4daa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e00b80e0 con 0x7fb4e00b29e0 2024-05-20T22:11:17.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ded9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e0157dd0 0x7fb4e00b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ded9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e0157dd0 0x7fb4e00b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35646/0 (socket says 172.21.15.55:35646) 2024-05-20T22:11:17.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ded9d640 1 -- 172.21.15.55:0/3312944395 learned_addr learned my addr 172.21.15.55:0/3312944395 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4de59c640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4e008bcb0 0x7fb4e01552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ddd9b640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4e00b29e0 0x7fb4e00afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ded9d640 1 -- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4e00b29e0 msgr2=0x7fb4e00afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ded9d640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4e00b29e0 0x7fb4e00afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ded9d640 1 -- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4e008bcb0 msgr2=0x7fb4e01552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ded9d640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4e008bcb0 0x7fb4e01552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ded9d640 1 -- 172.21.15.55:0/3312944395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e00b56b0 con 0x7fb4e0157dd0 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ddd9b640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4e00b29e0 0x7fb4e00afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4ded9d640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e0157dd0 0x7fb4e00b0200 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000d600 tx=0x7fb4d000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.878+0000 7fb4d6ffd640 1 -- 172.21.15.55:0/3312944395 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00125f0 con 0x7fb4e0157dd0 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.879+0000 7fb4d6ffd640 1 -- 172.21.15.55:0/3312944395 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d0019070 con 0x7fb4e0157dd0 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.879+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e01584b0 con 0x7fb4e0157dd0 2024-05-20T22:11:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.879+0000 7fb4d6ffd640 1 -- 172.21.15.55:0/3312944395 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00129f0 con 0x7fb4e0157dd0 2024-05-20T22:11:17.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.879+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e015c9b0 con 0x7fb4e0157dd0 2024-05-20T22:11:17.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.879+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4e01563b0 con 0x7fb4e0157dd0 2024-05-20T22:11:17.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.880+0000 7fb4d6ffd640 1 -- 172.21.15.55:0/3312944395 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4d0012b90 con 0x7fb4e0157dd0 2024-05-20T22:11:17.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.881+0000 7fb4d6ffd640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be30 0x7fb4c006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.881+0000 7fb4d6ffd640 1 -- 172.21.15.55:0/3312944395 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7fb4d008bac0 con 0x7fb4e0157dd0 2024-05-20T22:11:17.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.881+0000 7fb4de59c640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be30 0x7fb4c006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.881+0000 7fb4d6ffd640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] conn(0x7fb4c00723c0 0x7fb4c0074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:17.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.881+0000 7fb4d6ffd640 1 -- 172.21.15.55:0/3312944395 --> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4c0074f20 con 0x7fb4c00723c0 2024-05-20T22:11:17.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.881+0000 7fb4d6ffd640 1 -- 172.21.15.55:0/3312944395 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb4d008bf20 con 0x7fb4e0157dd0 2024-05-20T22:11:17.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.881+0000 7fb4ddd9b640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] conn(0x7fb4c00723c0 0x7fb4c0074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:17.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.881+0000 7fb4de59c640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be30 0x7fb4c006e340 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8033000 tx=0x7fb4d8009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.882+0000 7fb4ddd9b640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] conn(0x7fb4c00723c0 0x7fb4c0074850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:17.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.883+0000 7fb4d6ffd640 1 -- 172.21.15.55:0/3312944395 <== osd.6 v2:172.21.15.138:6824/1454288856 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4c0074f20 con 0x7fb4c00723c0 2024-05-20T22:11:17.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.896+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 --> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb4e0158240 con 0x7fb4c00723c0 2024-05-20T22:11:17.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4d6ffd640 1 -- 172.21.15.55:0/3312944395 <== osd.6 v2:172.21.15.138:6824/1454288856 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb4e0158240 con 0x7fb4c00723c0 2024-05-20T22:11:17.898 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:11:17.898 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T22:11:17.898 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:11:17.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] conn(0x7fb4c00723c0 msgr2=0x7fb4c0074850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4e4daa640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.138:6824/1454288856,v1:172.21.15.138:6825/1454288856] conn(0x7fb4c00723c0 0x7fb4c0074850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be30 msgr2=0x7fb4c006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4e4daa640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fb4c006be30 0x7fb4c006e340 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8033000 tx=0x7fb4d8009000 comp rx=0 tx=0).stop 2024-05-20T22:11:17.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e0157dd0 msgr2=0x7fb4e00b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:17.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4e4daa640 1 --2- 172.21.15.55:0/3312944395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e0157dd0 0x7fb4e00b0200 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000d600 tx=0x7fb4d000dac0 comp rx=0 tx=0).stop 2024-05-20T22:11:17.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4ded9d640 1 -- 172.21.15.55:0/3312944395 reap_dead start 2024-05-20T22:11:17.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 shutdown_connections 2024-05-20T22:11:17.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 >> 172.21.15.55:0/3312944395 conn(0x7fb4e00d2860 msgr2=0x7fb4e01593d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:17.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 shutdown_connections 2024-05-20T22:11:17.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:17.897+0000 7fb4e4daa640 1 -- 172.21.15.55:0/3312944395 wait complete. 2024-05-20T22:11:17.909 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-20T22:11:18.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.023+0000 7f9b76964640 1 Processor -- start 2024-05-20T22:11:18.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.023+0000 7f9b76964640 1 -- start start 2024-05-20T22:11:18.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.024+0000 7f9b76964640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b70161140 0x7f9b70072690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.024+0000 7f9b76964640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b70157870 con 0x7f9b7015e550 2024-05-20T22:11:18.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.024+0000 7f9b76964640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b7015ed70 con 0x7f9b7015e960 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.024+0000 7f9b76964640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9b70073310 con 0x7f9b70161140 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6ffff640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9b7015e550 0x7f9b7006bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44382/0 (socket says 172.21.15.55:44382) 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6ffff640 1 -- 172.21.15.55:0/4291041794 learned_addr learned my addr 172.21.15.55:0/4291041794 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b74f01640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b70161140 0x7f9b70072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 576751181 0 0) 0x7f9b70157870 con 0x7f9b7015e550 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b58003520 con 0x7f9b7015e550 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3188135610 0 0) 0x7f9b58003520 con 0x7f9b7015e550 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b70161140 msgr2=0x7f9b70072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b70161140 0x7f9b70072690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 >> v1:172.21.15.138:6789/0 conn(0x7f9b7015e960 legacy=0x7f9b70160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b700735f0 con 0x7f9b7015e550 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7f9b60001690 con 0x7f9b7015e550 2024-05-20T22:11:18.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 >> v1:172.21.15.55:6789/0 conn(0x7f9b7015e550 legacy=0x7f9b7006bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 0x7f9b58003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b58004800 0x7f9b58004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b58005150 0x7f9b5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9b70073310 con 0x7f9b580039e0 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5800bf00 con 0x7f9b58005150 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9b5800c080 con 0x7f9b58004800 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6ffff640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 0x7f9b58003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.025+0000 7f9b6f7fe640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b58004800 0x7f9b58004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.026+0000 7f9b74f01640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b58005150 0x7f9b5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.026+0000 7f9b6ffff640 1 -- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b58004800 msgr2=0x7f9b58004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.026+0000 7f9b6ffff640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b58004800 0x7f9b58004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.026+0000 7f9b6ffff640 1 -- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b58005150 msgr2=0x7f9b5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.026+0000 7f9b6ffff640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b58005150 0x7f9b5800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.026+0000 7f9b6ffff640 1 -- 172.21.15.55:0/4291041794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b58003520 con 0x7f9b580039e0 2024-05-20T22:11:18.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.026+0000 7f9b6ffff640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 0x7f9b58003e10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9b700707a0 tx=0x7f9b58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a3e1805bc606b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.027+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b60007ce0 con 0x7f9b580039e0 2024-05-20T22:11:18.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.027+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b60005af0 con 0x7f9b580039e0 2024-05-20T22:11:18.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.027+0000 7f9b6effd640 1 -- 172.21.15.55:0/4291041794 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b60007070 con 0x7f9b580039e0 2024-05-20T22:11:18.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.027+0000 7f9b76964640 1 -- 172.21.15.55:0/4291041794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 msgr2=0x7f9b58003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.027+0000 7f9b76964640 1 --2- 172.21.15.55:0/4291041794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 0x7f9b58003e10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9b700707a0 tx=0x7f9b58005830 comp rx=0 tx=0).stop 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.027+0000 7f9b74f01640 1 -- 172.21.15.55:0/4291041794 reap_dead start 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.027+0000 7f9b76964640 1 -- 172.21.15.55:0/4291041794 shutdown_connections 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.027+0000 7f9b76964640 1 -- 172.21.15.55:0/4291041794 >> 172.21.15.55:0/4291041794 conn(0x7f9b700b3980 msgr2=0x7f9b700b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.027+0000 7f9b76964640 1 -- 172.21.15.55:0/4291041794 shutdown_connections 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.027+0000 7f9b76964640 1 -- 172.21.15.55:0/4291041794 wait complete. 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.029+0000 7f9b76964640 1 Processor -- start 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.029+0000 7f9b76964640 1 -- start start 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.030+0000 7f9b76964640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 0x7f9b7013e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b76964640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b7013db70 0x7f9b701401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b76964640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7014f670 0x7f9b7015be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b76964640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7006bac0 con 0x7f9b580039e0 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b76964640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7006b7e0 con 0x7f9b7014f670 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b76964640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7006c6f0 con 0x7f9b7013db70 2024-05-20T22:11:18.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 0x7f9b7013e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 0x7f9b7013e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35670/0 (socket says 172.21.15.55:35670) 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6ffff640 1 -- 172.21.15.55:0/2615717221 learned_addr learned my addr 172.21.15.55:0/2615717221 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6f7fe640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b7013db70 0x7f9b701401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b74f01640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7014f670 0x7f9b7015be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6ffff640 1 -- 172.21.15.55:0/2615717221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b7013db70 msgr2=0x7f9b701401f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6ffff640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b7013db70 0x7f9b701401f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6ffff640 1 -- 172.21.15.55:0/2615717221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7014f670 msgr2=0x7f9b7015be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6ffff640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7014f670 0x7f9b7015be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6ffff640 1 -- 172.21.15.55:0/2615717221 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b70140730 con 0x7f9b580039e0 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6ffff640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 0x7f9b7013e1a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9b6002ad50 tx=0x7f9b600019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6cff9640 1 -- 172.21.15.55:0/2615717221 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b60007ce0 con 0x7f9b580039e0 2024-05-20T22:11:18.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b6cff9640 1 -- 172.21.15.55:0/2615717221 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b60042070 con 0x7f9b580039e0 2024-05-20T22:11:18.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.031+0000 7f9b76964640 1 -- 172.21.15.55:0/2615717221 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b70140c10 con 0x7f9b580039e0 2024-05-20T22:11:18.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.032+0000 7f9b6cff9640 1 -- 172.21.15.55:0/2615717221 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b60007270 con 0x7f9b580039e0 2024-05-20T22:11:18.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.032+0000 7f9b76964640 1 -- 172.21.15.55:0/2615717221 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7014ee70 con 0x7f9b580039e0 2024-05-20T22:11:18.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.033+0000 7f9b76964640 1 -- 172.21.15.55:0/2615717221 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b70141130 con 0x7f9b580039e0 2024-05-20T22:11:18.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.033+0000 7f9b6cff9640 1 -- 172.21.15.55:0/2615717221 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b60007410 con 0x7f9b580039e0 2024-05-20T22:11:18.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.034+0000 7f9b6cff9640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b5c06be60 0x7f9b5c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.034+0000 7f9b6cff9640 1 -- 172.21.15.55:0/2615717221 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f9b600aac70 con 0x7f9b580039e0 2024-05-20T22:11:18.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.034+0000 7f9b6f7fe640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b5c06be60 0x7f9b5c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.034+0000 7f9b6f7fe640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b5c06be60 0x7f9b5c06e370 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9b701535c0 tx=0x7f9b64009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.039+0000 7f9b6cff9640 1 -- 172.21.15.55:0/2615717221 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9b70141130 con 0x7f9b580039e0 2024-05-20T22:11:18.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.223+0000 7f9b76964640 1 -- 172.21.15.55:0/2615717221 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f9b70069090 con 0x7f9b5c06be60 2024-05-20T22:11:18.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.223+0000 7f9b6cff9640 1 -- 172.21.15.55:0/2615717221 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9b70069090 con 0x7f9b5c06be60 2024-05-20T22:11:18.224 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 6 to deep-scrub 2024-05-20T22:11:18.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 -- 172.21.15.55:0/2615717221 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b5c06be60 msgr2=0x7f9b5c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b5c06be60 0x7f9b5c06e370 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9b701535c0 tx=0x7f9b64009040 comp rx=0 tx=0).stop 2024-05-20T22:11:18.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 -- 172.21.15.55:0/2615717221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 msgr2=0x7f9b7013e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 0x7f9b7013e1a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9b6002ad50 tx=0x7f9b600019e0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 -- 172.21.15.55:0/2615717221 shutdown_connections 2024-05-20T22:11:18.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f9b5c06be60 0x7f9b5c06e370 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b7014f670 0x7f9b7015be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b7013db70 0x7f9b701401f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 --2- 172.21.15.55:0/2615717221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b580039e0 0x7f9b7013e1a0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 -- 172.21.15.55:0/2615717221 >> 172.21.15.55:0/2615717221 conn(0x7f9b700b3980 msgr2=0x7f9b700b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:18.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 -- 172.21.15.55:0/2615717221 shutdown_connections 2024-05-20T22:11:18.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.225+0000 7f9b76964640 1 -- 172.21.15.55:0/2615717221 wait complete. 2024-05-20T22:11:18.238 INFO:tasks.ceph:Scrubbing osd.7 2024-05-20T22:11:18.238 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-20T22:11:18.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.355+0000 7f7c2985a640 1 Processor -- start 2024-05-20T22:11:18.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.355+0000 7f7c2985a640 1 -- start start 2024-05-20T22:11:18.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c2985a640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7c240d32a0 0x7f7c2415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c2985a640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c240d3780 con 0x7f7c240d01b0 2024-05-20T22:11:18.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c2985a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c240d3a60 con 0x7f7c240cf690 2024-05-20T22:11:18.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c2985a640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f7c240d3d40 con 0x7f7c240d32a0 2024-05-20T22:11:18.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c22ffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7c240d32a0 0x7f7c2415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c237fe640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7c240d01b0 0x7f7c2406b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44396/0 (socket says 172.21.15.55:44396) 2024-05-20T22:11:18.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c237fe640 1 -- 172.21.15.55:0/1755732300 learned_addr learned my addr 172.21.15.55:0/1755732300 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:18.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c22ffd640 1 -- 172.21.15.55:0/1755732300 >> v1:172.21.15.138:6789/0 conn(0x7f7c240cf690 legacy=0x7f7c240cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c22ffd640 1 -- 172.21.15.55:0/1755732300 >> v1:172.21.15.55:6789/0 conn(0x7f7c240d01b0 legacy=0x7f7c2406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c21ffb640 1 -- 172.21.15.55:0/1755732300 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 284640934 0 0) 0x7f7c240d3780 con 0x7f7c240d01b0 2024-05-20T22:11:18.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.357+0000 7f7c22ffd640 1 -- 172.21.15.55:0/1755732300 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c240d4020 con 0x7f7c240d32a0 2024-05-20T22:11:18.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.358+0000 7f7c22ffd640 1 --2- 172.21.15.55:0/1755732300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c240d32a0 0x7f7c2415f150 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7c140097a0 tx=0x7f7c1402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b4dd8529a5cd95 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.358+0000 7f7c21ffb640 1 -- 172.21.15.55:0/1755732300 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14037070 con 0x7f7c240d32a0 2024-05-20T22:11:18.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.358+0000 7f7c21ffb640 1 -- 172.21.15.55:0/1755732300 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c14032440 con 0x7f7c240d32a0 2024-05-20T22:11:18.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.358+0000 7f7c21ffb640 1 -- 172.21.15.55:0/1755732300 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14032810 con 0x7f7c240d32a0 2024-05-20T22:11:18.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.359+0000 7f7c2985a640 1 -- 172.21.15.55:0/1755732300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c240d32a0 msgr2=0x7f7c2415f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.359+0000 7f7c2985a640 1 --2- 172.21.15.55:0/1755732300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c240d32a0 0x7f7c2415f150 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7c140097a0 tx=0x7f7c1402a230 comp rx=0 tx=0).stop 2024-05-20T22:11:18.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.359+0000 7f7c2985a640 1 -- 172.21.15.55:0/1755732300 shutdown_connections 2024-05-20T22:11:18.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.359+0000 7f7c2985a640 1 --2- 172.21.15.55:0/1755732300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c240d32a0 0x7f7c2415f150 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.359+0000 7f7c2985a640 1 -- 172.21.15.55:0/1755732300 >> 172.21.15.55:0/1755732300 conn(0x7f7c240b6de0 msgr2=0x7f7c240d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:18.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.359+0000 7f7c2985a640 1 -- 172.21.15.55:0/1755732300 shutdown_connections 2024-05-20T22:11:18.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.359+0000 7f7c2985a640 1 -- 172.21.15.55:0/1755732300 wait complete. 2024-05-20T22:11:18.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.361+0000 7f7c2985a640 1 Processor -- start 2024-05-20T22:11:18.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.361+0000 7f7c2985a640 1 -- start start 2024-05-20T22:11:18.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c2985a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c240cf690 0x7f7c2415a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c2985a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c240d01b0 0x7f7c240cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c2985a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c240d32a0 0x7f7c240ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c2985a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c240d4820 con 0x7f7c240d32a0 2024-05-20T22:11:18.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c2985a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c240b4350 con 0x7f7c240d01b0 2024-05-20T22:11:18.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c2985a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7c240ccd70 con 0x7f7c240cf690 2024-05-20T22:11:18.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c237fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c240d32a0 0x7f7c240ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c237fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c240d32a0 0x7f7c240ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35676/0 (socket says 172.21.15.55:35676) 2024-05-20T22:11:18.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c237fe640 1 -- 172.21.15.55:0/3594186514 learned_addr learned my addr 172.21.15.55:0/3594186514 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:18.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c22ffd640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c240cf690 0x7f7c2415a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.362+0000 7f7c227fc640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c240d01b0 0x7f7c240cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c237fe640 1 -- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c240cf690 msgr2=0x7f7c2415a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c237fe640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c240cf690 0x7f7c2415a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c237fe640 1 -- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c240d01b0 msgr2=0x7f7c240cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c237fe640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c240d01b0 0x7f7c240cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c237fe640 1 -- 172.21.15.55:0/3594186514 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2415f7e0 con 0x7f7c240d32a0 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c237fe640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c240d32a0 0x7f7c240ce2f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00c700 tx=0x7f7c1c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c0bfff640 1 -- 172.21.15.55:0/3594186514 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c018070 con 0x7f7c240d32a0 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c0bfff640 1 -- 172.21.15.55:0/3594186514 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1c00ce10 con 0x7f7c240d32a0 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2415fac0 con 0x7f7c240d32a0 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c0bfff640 1 -- 172.21.15.55:0/3594186514 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00a880 con 0x7f7c240d32a0 2024-05-20T22:11:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.363+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c24160080 con 0x7f7c240d32a0 2024-05-20T22:11:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.364+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c24160460 con 0x7f7c240d32a0 2024-05-20T22:11:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.365+0000 7f7c0bfff640 1 -- 172.21.15.55:0/3594186514 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c1c01b070 con 0x7f7c240d32a0 2024-05-20T22:11:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.365+0000 7f7c0bfff640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c1006be30 0x7f7c1006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.366+0000 7f7c0bfff640 1 -- 172.21.15.55:0/3594186514 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f7c1c08a290 con 0x7f7c240d32a0 2024-05-20T22:11:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.366+0000 7f7c22ffd640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c1006be30 0x7f7c1006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.366+0000 7f7c0bfff640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] conn(0x7f7c100723c0 0x7f7c10074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.366+0000 7f7c0bfff640 1 -- 172.21.15.55:0/3594186514 --> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c10074f20 con 0x7f7c100723c0 2024-05-20T22:11:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.366+0000 7f7c0bfff640 1 -- 172.21.15.55:0/3594186514 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7c1c091050 con 0x7f7c240d32a0 2024-05-20T22:11:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.366+0000 7f7c227fc640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] conn(0x7f7c100723c0 0x7f7c10074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.366+0000 7f7c22ffd640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c1006be30 0x7f7c1006e340 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7c14034000 tx=0x7f7c1402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.366+0000 7f7c227fc640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] conn(0x7f7c100723c0 0x7f7c10074850 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.367+0000 7f7c0bfff640 1 -- 172.21.15.55:0/3594186514 <== osd.7 v2:172.21.15.138:6816/2068592032 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c10074f20 con 0x7f7c100723c0 2024-05-20T22:11:18.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.381+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 --> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7c24069090 con 0x7f7c100723c0 2024-05-20T22:11:18.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.381+0000 7f7c0bfff640 1 -- 172.21.15.55:0/3594186514 <== osd.7 v2:172.21.15.138:6816/2068592032 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7c24069090 con 0x7f7c100723c0 2024-05-20T22:11:18.382 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-20T22:11:18.382 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T22:11:18.382 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-20T22:11:18.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.381+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] conn(0x7f7c100723c0 msgr2=0x7f7c10074850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.382+0000 7f7c2985a640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.138:6816/2068592032,v1:172.21.15.138:6817/2068592032] conn(0x7f7c100723c0 0x7f7c10074850 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.382+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c1006be30 msgr2=0x7f7c1006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.382+0000 7f7c2985a640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f7c1006be30 0x7f7c1006e340 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7c14034000 tx=0x7f7c1402e000 comp rx=0 tx=0).stop 2024-05-20T22:11:18.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.382+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c240d32a0 msgr2=0x7f7c240ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.382+0000 7f7c2985a640 1 --2- 172.21.15.55:0/3594186514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c240d32a0 0x7f7c240ce2f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00c700 tx=0x7f7c1c00cbc0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.382+0000 7f7c237fe640 1 -- 172.21.15.55:0/3594186514 reap_dead start 2024-05-20T22:11:18.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.382+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 shutdown_connections 2024-05-20T22:11:18.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.382+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 >> 172.21.15.55:0/3594186514 conn(0x7f7c240b6de0 msgr2=0x7f7c2415e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:18.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.382+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 shutdown_connections 2024-05-20T22:11:18.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.382+0000 7f7c2985a640 1 -- 172.21.15.55:0/3594186514 wait complete. 2024-05-20T22:11:18.393 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-20T22:11:18.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.507+0000 7f6ba4ede640 1 Processor -- start 2024-05-20T22:11:18.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.508+0000 7f6ba4ede640 1 -- start start 2024-05-20T22:11:18.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.509+0000 7f6ba4ede640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6ba00d32a0 0x7f6ba015f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.509+0000 7f6ba4ede640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ba00cf690 0x7f6ba00cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.509+0000 7f6ba4ede640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba00d3780 con 0x7f6ba00cffe0 2024-05-20T22:11:18.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.509+0000 7f6ba4ede640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba006c510 con 0x7f6ba00d32a0 2024-05-20T22:11:18.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.509+0000 7f6ba4ede640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba00d3a60 con 0x7f6ba00cf690 2024-05-20T22:11:18.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.509+0000 7f6b9e59c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6ba00d32a0 0x7f6ba015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.509+0000 7f6b9e59c640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6ba00d32a0 0x7f6ba015f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35690/0 (socket says 172.21.15.55:35690) 2024-05-20T22:11:18.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.509+0000 7f6b9e59c640 1 -- 172.21.15.55:0/1201988365 learned_addr learned my addr 172.21.15.55:0/1201988365 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:18.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.509+0000 7f6b9dd9b640 1 --2- 172.21.15.55:0/1201988365 >> v2:172.21.15.138:3300/0 conn(0x7f6ba00cf690 0x7f6ba00cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.510+0000 7f6b9e59c640 1 -- 172.21.15.55:0/1201988365 >> v2:172.21.15.138:3300/0 conn(0x7f6ba00cf690 msgr2=0x7f6ba00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:18.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.510+0000 7f6b9e59c640 1 --2- 172.21.15.55:0/1201988365 >> v2:172.21.15.138:3300/0 conn(0x7f6ba00cf690 0x7f6ba00cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.510+0000 7f6b9e59c640 1 -- 172.21.15.55:0/1201988365 >> v1:172.21.15.138:6789/0 conn(0x7f6ba00cffe0 legacy=0x7f6ba0162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.510+0000 7f6b9e59c640 1 -- 172.21.15.55:0/1201988365 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba00d3d40 con 0x7f6ba00d32a0 2024-05-20T22:11:18.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.510+0000 7f6b9dd9b640 1 --2- 172.21.15.55:0/1201988365 >> v2:172.21.15.138:3300/0 conn(0x7f6ba00cf690 0x7f6ba00cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:18.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.510+0000 7f6b9e59c640 1 --2- 172.21.15.55:0/1201988365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 0x7f6ba015f150 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6ba008e150 tx=0x7f6b9002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f469b1f2bdf5e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6b9d59a640 1 -- 172.21.15.55:0/1201988365 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90036070 con 0x7f6ba00d32a0 2024-05-20T22:11:18.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6b9d59a640 1 -- 172.21.15.55:0/1201988365 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b90031600 con 0x7f6ba00d32a0 2024-05-20T22:11:18.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6b9d59a640 1 -- 172.21.15.55:0/1201988365 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b900319d0 con 0x7f6ba00d32a0 2024-05-20T22:11:18.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6ba4ede640 1 -- 172.21.15.55:0/1201988365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 msgr2=0x7f6ba015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6ba4ede640 1 --2- 172.21.15.55:0/1201988365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 0x7f6ba015f150 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6ba008e150 tx=0x7f6b9002a730 comp rx=0 tx=0).stop 2024-05-20T22:11:18.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6ba4ede640 1 -- 172.21.15.55:0/1201988365 shutdown_connections 2024-05-20T22:11:18.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6ba4ede640 1 --2- 172.21.15.55:0/1201988365 >> v2:172.21.15.138:3300/0 conn(0x7f6ba00cf690 0x7f6ba00cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6ba4ede640 1 --2- 172.21.15.55:0/1201988365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 0x7f6ba015f150 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6ba4ede640 1 -- 172.21.15.55:0/1201988365 >> 172.21.15.55:0/1201988365 conn(0x7f6ba00b6de0 msgr2=0x7f6ba00d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:18.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6ba4ede640 1 -- 172.21.15.55:0/1201988365 shutdown_connections 2024-05-20T22:11:18.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.511+0000 7f6ba4ede640 1 -- 172.21.15.55:0/1201988365 wait complete. 2024-05-20T22:11:18.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.513+0000 7f6ba4ede640 1 Processor -- start 2024-05-20T22:11:18.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.513+0000 7f6ba4ede640 1 -- start start 2024-05-20T22:11:18.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6ba4ede640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ba00cf690 0x7f6ba015a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6ba4ede640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba00cffe0 0x7f6ba01609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6ba4ede640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 0x7f6ba00c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6ba4ede640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba00d03f0 con 0x7f6ba00d32a0 2024-05-20T22:11:18.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6ba4ede640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba00d4300 con 0x7f6ba00cffe0 2024-05-20T22:11:18.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6ba4ede640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba00d4820 con 0x7f6ba00cf690 2024-05-20T22:11:18.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 0x7f6ba00c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 0x7f6ba00c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35700/0 (socket says 172.21.15.55:35700) 2024-05-20T22:11:18.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9ed9d640 1 -- 172.21.15.55:0/3062347521 learned_addr learned my addr 172.21.15.55:0/3062347521 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:18.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9e59c640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ba00cf690 0x7f6ba015a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9dd9b640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba00cffe0 0x7f6ba01609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9ed9d640 1 -- 172.21.15.55:0/3062347521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ba00cf690 msgr2=0x7f6ba015a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9ed9d640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ba00cf690 0x7f6ba015a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9ed9d640 1 -- 172.21.15.55:0/3062347521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba00cffe0 msgr2=0x7f6ba01609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9ed9d640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba00cffe0 0x7f6ba01609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9ed9d640 1 -- 172.21.15.55:0/3062347521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba015a740 con 0x7f6ba00d32a0 2024-05-20T22:11:18.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.515+0000 7f6b9ed9d640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 0x7f6ba00c9d90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800c3f0 tx=0x7f6b9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.516+0000 7f6b8effd640 1 -- 172.21.15.55:0/3062347521 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980052d0 con 0x7f6ba00d32a0 2024-05-20T22:11:18.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.516+0000 7f6b8effd640 1 -- 172.21.15.55:0/3062347521 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98005470 con 0x7f6ba00d32a0 2024-05-20T22:11:18.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.516+0000 7f6ba4ede640 1 -- 172.21.15.55:0/3062347521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba00c8cf0 con 0x7f6ba00d32a0 2024-05-20T22:11:18.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.516+0000 7f6b8effd640 1 -- 172.21.15.55:0/3062347521 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98007260 con 0x7f6ba00d32a0 2024-05-20T22:11:18.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.516+0000 7f6ba4ede640 1 -- 172.21.15.55:0/3062347521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba00c92b0 con 0x7f6ba00d32a0 2024-05-20T22:11:18.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.517+0000 7f6ba4ede640 1 -- 172.21.15.55:0/3062347521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba00cd360 con 0x7f6ba00d32a0 2024-05-20T22:11:18.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.517+0000 7f6b8effd640 1 -- 172.21.15.55:0/3062347521 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b98007400 con 0x7f6ba00d32a0 2024-05-20T22:11:18.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.518+0000 7f6b8effd640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b8006be60 0x7f6b8006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.518+0000 7f6b8effd640 1 -- 172.21.15.55:0/3062347521 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f6b9808a020 con 0x7f6ba00d32a0 2024-05-20T22:11:18.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.518+0000 7f6b9e59c640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b8006be60 0x7f6b8006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.518+0000 7f6b9e59c640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b8006be60 0x7f6b8006e370 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b90033000 tx=0x7f6b90009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.522+0000 7f6b8effd640 1 -- 172.21.15.55:0/3062347521 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6b980584a0 con 0x7f6ba00d32a0 2024-05-20T22:11:18.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.705+0000 7f6ba4ede640 1 -- 172.21.15.55:0/3062347521 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f6ba0069090 con 0x7f6b8006be60 2024-05-20T22:11:18.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.705+0000 7f6b8effd640 1 -- 172.21.15.55:0/3062347521 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6ba0069090 con 0x7f6b8006be60 2024-05-20T22:11:18.707 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 7 to deep-scrub 2024-05-20T22:11:18.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 -- 172.21.15.55:0/3062347521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b8006be60 msgr2=0x7f6b8006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b8006be60 0x7f6b8006e370 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b90033000 tx=0x7f6b90009000 comp rx=0 tx=0).stop 2024-05-20T22:11:18.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 -- 172.21.15.55:0/3062347521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 msgr2=0x7f6ba00c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 0x7f6ba00c9d90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800c3f0 tx=0x7f6b9800c8b0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 -- 172.21.15.55:0/3062347521 shutdown_connections 2024-05-20T22:11:18.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f6b8006be60 0x7f6b8006e370 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ba00d32a0 0x7f6ba00c9d90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba00cffe0 0x7f6ba01609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 --2- 172.21.15.55:0/3062347521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ba00cf690 0x7f6ba015a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 -- 172.21.15.55:0/3062347521 >> 172.21.15.55:0/3062347521 conn(0x7f6ba00b6de0 msgr2=0x7f6ba015e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:18.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 -- 172.21.15.55:0/3062347521 shutdown_connections 2024-05-20T22:11:18.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.708+0000 7f6ba4ede640 1 -- 172.21.15.55:0/3062347521 wait complete. 2024-05-20T22:11:18.720 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:11:18.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.834+0000 7fa92f8ca640 1 Processor -- start 2024-05-20T22:11:18.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.834+0000 7fa92f8ca640 1 -- start start 2024-05-20T22:11:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92f8ca640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa92815f100 0x7fa9280b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92f8ca640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa92806c790 0x7fa9280cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92f8ca640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9280d3dd0 con 0x7fa9280b3b30 2024-05-20T22:11:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92f8ca640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa9280d40b0 con 0x7fa92815f100 2024-05-20T22:11:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92f8ca640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa9280d4390 con 0x7fa92806c790 2024-05-20T22:11:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92ce65640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa92815f100 0x7fa9280b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92d666640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa9280b3b30 0x7fa92806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58870/0 (socket says 172.21.15.55:58870) 2024-05-20T22:11:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92d666640 1 -- 172.21.15.55:0/3807319016 learned_addr learned my addr 172.21.15.55:0/3807319016 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92ce65640 1 -- 172.21.15.55:0/3807319016 >> v2:172.21.15.138:3300/0 conn(0x7fa92806c790 msgr2=0x7fa9280cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92de67640 1 --2- 172.21.15.55:0/3807319016 >> v2:172.21.15.138:3300/0 conn(0x7fa92806c790 0x7fa9280cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92ce65640 1 --2- 172.21.15.55:0/3807319016 >> v2:172.21.15.138:3300/0 conn(0x7fa92806c790 0x7fa9280cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92ce65640 1 -- 172.21.15.55:0/3807319016 >> v1:172.21.15.55:6789/0 conn(0x7fa9280b3b30 legacy=0x7fa92806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa91ffff640 1 -- 172.21.15.55:0/3807319016 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2451738726 0 0) 0x7fa9280d3dd0 con 0x7fa9280b3b30 2024-05-20T22:11:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.836+0000 7fa92ce65640 1 -- 172.21.15.55:0/3807319016 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92806b5a0 con 0x7fa92815f100 2024-05-20T22:11:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.837+0000 7fa92ce65640 1 --2- 172.21.15.55:0/3807319016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92815f100 0x7fa9280b7fe0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa9280d12f0 tx=0x7fa91802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136488480ccc925b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.837+0000 7fa91ffff640 1 -- 172.21.15.55:0/3807319016 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918037070 con 0x7fa92815f100 2024-05-20T22:11:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.837+0000 7fa91ffff640 1 -- 172.21.15.55:0/3807319016 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa918032440 con 0x7fa92815f100 2024-05-20T22:11:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.837+0000 7fa91ffff640 1 -- 172.21.15.55:0/3807319016 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918032810 con 0x7fa92815f100 2024-05-20T22:11:18.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.838+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3807319016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92815f100 msgr2=0x7fa9280b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.838+0000 7fa92f8ca640 1 --2- 172.21.15.55:0/3807319016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92815f100 0x7fa9280b7fe0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa9280d12f0 tx=0x7fa91802a1a0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.838+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3807319016 shutdown_connections 2024-05-20T22:11:18.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.838+0000 7fa92f8ca640 1 --2- 172.21.15.55:0/3807319016 >> v2:172.21.15.138:3300/0 conn(0x7fa92806c790 0x7fa9280cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.838+0000 7fa92f8ca640 1 --2- 172.21.15.55:0/3807319016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92815f100 0x7fa9280b7fe0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.838+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3807319016 >> 172.21.15.55:0/3807319016 conn(0x7fa9280d51d0 msgr2=0x7fa9280b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:18.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.838+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3807319016 shutdown_connections 2024-05-20T22:11:18.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.838+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3807319016 wait complete. 2024-05-20T22:11:18.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.840+0000 7fa92f8ca640 1 Processor -- start 2024-05-20T22:11:18.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.840+0000 7fa92f8ca640 1 -- start start 2024-05-20T22:11:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92f8ca640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92806c790 0x7fa928157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92f8ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9280b3b30 0x7fa928159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92f8ca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa92815f100 0x7fa928159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92f8ca640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa92806cba0 con 0x7fa92806c790 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92f8ca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa92806bb60 con 0x7fa9280b3b30 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92f8ca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa92806cd20 con 0x7fa92815f100 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92d666640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92806c790 0x7fa928157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92d666640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92806c790 0x7fa928157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52536/0 (socket says 172.21.15.55:52536) 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92d666640 1 -- 172.21.15.55:0/3113587227 learned_addr learned my addr 172.21.15.55:0/3113587227 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92ce65640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9280b3b30 0x7fa928159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92de67640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa92815f100 0x7fa928159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92d666640 1 -- 172.21.15.55:0/3113587227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa92815f100 msgr2=0x7fa928159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92d666640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa92815f100 0x7fa928159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92d666640 1 -- 172.21.15.55:0/3113587227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9280b3b30 msgr2=0x7fa928159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.841+0000 7fa92d666640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9280b3b30 0x7fa928159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.842+0000 7fa92d666640 1 -- 172.21.15.55:0/3113587227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9281578d0 con 0x7fa92806c790 2024-05-20T22:11:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.842+0000 7fa92d666640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92806c790 0x7fa928157390 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c410 tx=0x7fa92400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.842+0000 7fa91dffb640 1 -- 172.21.15.55:0/3113587227 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240074e0 con 0x7fa92806c790 2024-05-20T22:11:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.842+0000 7fa91dffb640 1 -- 172.21.15.55:0/3113587227 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9240047e0 con 0x7fa92806c790 2024-05-20T22:11:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.842+0000 7fa91dffb640 1 -- 172.21.15.55:0/3113587227 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924004bd0 con 0x7fa92806c790 2024-05-20T22:11:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.842+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3113587227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92815bb80 con 0x7fa92806c790 2024-05-20T22:11:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.842+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3113587227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92815c140 con 0x7fa92806c790 2024-05-20T22:11:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.843+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3113587227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa928069090 con 0x7fa92806c790 2024-05-20T22:11:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.844+0000 7fa91dffb640 1 -- 172.21.15.55:0/3113587227 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa924011070 con 0x7fa92806c790 2024-05-20T22:11:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.844+0000 7fa91dffb640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90c06be60 0x7fa90c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.844+0000 7fa91dffb640 1 -- 172.21.15.55:0/3113587227 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7fa92408a9d0 con 0x7fa92806c790 2024-05-20T22:11:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.844+0000 7fa92ce65640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90c06be60 0x7fa90c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.845+0000 7fa92ce65640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90c06be60 0x7fa90c06e370 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa918034000 tx=0x7fa91802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:18.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:18.849+0000 7fa91dffb640 1 -- 172.21.15.55:0/3113587227 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa92400e280 con 0x7fa92806c790 2024-05-20T22:11:19.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.030+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3113587227 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9280d2ee0 con 0x7fa90c06be60 2024-05-20T22:11:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.031+0000 7fa91dffb640 1 -- 172.21.15.55:0/3113587227 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13430 (secure 0 0 0) 0x7fa9280d2ee0 con 0x7fa90c06be60 2024-05-20T22:11:19.032 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:11:19.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3113587227 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90c06be60 msgr2=0x7fa90c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:19.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90c06be60 0x7fa90c06e370 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa918034000 tx=0x7fa91802e000 comp rx=0 tx=0).stop 2024-05-20T22:11:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3113587227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92806c790 msgr2=0x7fa928157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92806c790 0x7fa928157390 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c410 tx=0x7fa92400c8d0 comp rx=0 tx=0).stop 2024-05-20T22:11:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3113587227 shutdown_connections 2024-05-20T22:11:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa90c06be60 0x7fa90c06e370 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa92815f100 0x7fa928159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9280b3b30 0x7fa928159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 --2- 172.21.15.55:0/3113587227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa92806c790 0x7fa928157390 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3113587227 >> 172.21.15.55:0/3113587227 conn(0x7fa9280d51d0 msgr2=0x7fa9280b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3113587227 shutdown_connections 2024-05-20T22:11:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:19.033+0000 7fa92f8ca640 1 -- 172.21.15.55:0/3113587227 wait complete. 2024-05-20T22:11:19.045 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-05-20T22:11:18.375373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":244200,"kb_used_data":7208,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730520,"statfs":{"total":773094113280,"available":772844052480,"internally_reserved":0,"allocated":7380992,"data_stored":4010600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.616103"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":259,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-20T22:11:10.767738+0000","last_change":"2024-05-20T22:07:42.080164+0000","last_active":"2024-05-20T22:11:10.767738+0000","last_peered":"2024-05-20T22:11:10.767738+0000","last_clean":"2024-05-20T22:11:10.767738+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:11:10.767738+0000","last_undegraded":"2024-05-20T22:11:10.767738+0000","last_fullsized":"2024-05-20T22:11:10.767738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T22:07:41.005354+0000","last_clean_scrub_stamp":"2024-05-20T22:07:41.005354+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:11:42.013349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":944101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":944101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:11:19.048 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-20T22:07:41.005354+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=20, tm_hour=22, tm_min=7, tm_sec=41, tm_wday=0, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=20, tm_hour=22, tm_min=11, tm_sec=13, tm_wday=0, tm_yday=141, tm_isdst=0) 2024-05-20T22:11:19.048 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-20T22:11:39.050 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T22:11:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.180+0000 7fc552284640 1 Processor -- start 2024-05-20T22:11:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.180+0000 7fc552284640 1 -- start start 2024-05-20T22:11:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc552284640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c06d8f0 0x7fc54c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc552284640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc54c163510 0x7fc54c163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc552284640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc54c06e370 con 0x7fc54c163e60 2024-05-20T22:11:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc552284640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc54c0d2000 con 0x7fc54c163510 2024-05-20T22:11:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc552284640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c0b7e20 con 0x7fc54c06d8f0 2024-05-20T22:11:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc550821640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc54c163e60 0x7fc54c0b2bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52402/0 (socket says 172.21.15.55:52402) 2024-05-20T22:11:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc550821640 1 -- 172.21.15.55:0/3546627455 learned_addr learned my addr 172.21.15.55:0/3546627455 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc54bfff640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c06d8f0 0x7fc54c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc54b7fe640 1 --2- 172.21.15.55:0/3546627455 >> v2:172.21.15.138:3300/0 conn(0x7fc54c163510 0x7fc54c163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3393646652 0 0) 0x7fc54c06e370 con 0x7fc54c163e60 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.181+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc534003520 con 0x7fc54c163e60 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2519868970 0 0) 0x7fc534003520 con 0x7fc54c163e60 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c06d8f0 msgr2=0x7fc54c06dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c06d8f0 0x7fc54c06dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 >> v2:172.21.15.138:3300/0 conn(0x7fc54c163510 msgr2=0x7fc54c163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 --2- 172.21.15.55:0/3546627455 >> v2:172.21.15.138:3300/0 conn(0x7fc54c163510 0x7fc54c163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c164270 con 0x7fc54c163e60 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3937779901 0 0) 0x7fc5440019f0 con 0x7fc54c163e60 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 >> v1:172.21.15.55:6789/0 conn(0x7fc54c163e60 legacy=0x7fc54c0b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5340039b0 0x7fc534003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c06d8f0 0x7fc534004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc534004d40 0x7fc534005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0d2000 con 0x7fc534004d40 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0b7e20 con 0x7fc5340039b0 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc53400ba40 con 0x7fc54c06d8f0 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc550821640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc534004d40 0x7fc534005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54bfff640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5340039b0 0x7fc534003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc54b7fe640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c06d8f0 0x7fc534004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc550821640 1 -- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c06d8f0 msgr2=0x7fc534004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc550821640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c06d8f0 0x7fc534004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc550821640 1 -- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5340039b0 msgr2=0x7fc534003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc550821640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5340039b0 0x7fc534003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.182+0000 7fc550821640 1 -- 172.21.15.55:0/3546627455 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc534003520 con 0x7fc534004d40 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.183+0000 7fc550821640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc534004d40 0x7fc534005150 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc5440013d0 tx=0x7fc534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ca6660cce84fa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:39.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.183+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54400a4e0 con 0x7fc534004d40 2024-05-20T22:11:39.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.183+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc54402f070 con 0x7fc534004d40 2024-05-20T22:11:39.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.183+0000 7fc54affd640 1 -- 172.21.15.55:0/3546627455 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54402bce0 con 0x7fc534004d40 2024-05-20T22:11:39.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.184+0000 7fc552284640 1 -- 172.21.15.55:0/3546627455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc534004d40 msgr2=0x7fc534005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.184+0000 7fc552284640 1 --2- 172.21.15.55:0/3546627455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc534004d40 0x7fc534005150 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc5440013d0 tx=0x7fc534005830 comp rx=0 tx=0).stop 2024-05-20T22:11:39.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.184+0000 7fc550821640 1 -- 172.21.15.55:0/3546627455 reap_dead start 2024-05-20T22:11:39.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.184+0000 7fc552284640 1 -- 172.21.15.55:0/3546627455 shutdown_connections 2024-05-20T22:11:39.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.184+0000 7fc552284640 1 -- 172.21.15.55:0/3546627455 >> 172.21.15.55:0/3546627455 conn(0x7fc54c0b4340 msgr2=0x7fc54c0b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:39.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.184+0000 7fc552284640 1 -- 172.21.15.55:0/3546627455 shutdown_connections 2024-05-20T22:11:39.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.184+0000 7fc552284640 1 -- 172.21.15.55:0/3546627455 wait complete. 2024-05-20T22:11:39.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.186+0000 7fc552284640 1 Processor -- start 2024-05-20T22:11:39.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.186+0000 7fc552284640 1 -- start start 2024-05-20T22:11:39.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.187+0000 7fc552284640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54c0c9890 0x7fc54c0b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc552284640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54c15deb0 0x7fc54c0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc552284640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0b0490 0x7fc54c0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc552284640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c09c1e0 con 0x7fc54c15deb0 2024-05-20T22:11:39.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc552284640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c09c060 con 0x7fc54c0c9890 2024-05-20T22:11:39.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc552284640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c06d1c0 con 0x7fc54c0b0490 2024-05-20T22:11:39.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc54b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54c15deb0 0x7fc54c0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc54b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54c15deb0 0x7fc54c0cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45888/0 (socket says 172.21.15.55:45888) 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc54b7fe640 1 -- 172.21.15.55:0/1136807342 learned_addr learned my addr 172.21.15.55:0/1136807342 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc54bfff640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54c0c9890 0x7fc54c0b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc550821640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0b0490 0x7fc54c0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc54b7fe640 1 -- 172.21.15.55:0/1136807342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0b0490 msgr2=0x7fc54c0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc54b7fe640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0b0490 0x7fc54c0b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc54b7fe640 1 -- 172.21.15.55:0/1136807342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54c0c9890 msgr2=0x7fc54c0b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc54b7fe640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54c0c9890 0x7fc54c0b6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc54b7fe640 1 -- 172.21.15.55:0/1136807342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c0b0de0 con 0x7fc54c15deb0 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc54b7fe640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54c15deb0 0x7fc54c0cc1e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc540011aa0 tx=0x7fc540011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc548ff9640 1 -- 172.21.15.55:0/1136807342 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54000fac0 con 0x7fc54c15deb0 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc548ff9640 1 -- 172.21.15.55:0/1136807342 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc54000fc60 con 0x7fc54c15deb0 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc552284640 1 -- 172.21.15.55:0/1136807342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c0b7500 con 0x7fc54c15deb0 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.188+0000 7fc548ff9640 1 -- 172.21.15.55:0/1136807342 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540014440 con 0x7fc54c15deb0 2024-05-20T22:11:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.189+0000 7fc552284640 1 -- 172.21.15.55:0/1136807342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c0cb9e0 con 0x7fc54c15deb0 2024-05-20T22:11:39.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.190+0000 7fc552284640 1 -- 172.21.15.55:0/1136807342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c069090 con 0x7fc54c15deb0 2024-05-20T22:11:39.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.190+0000 7fc548ff9640 1 -- 172.21.15.55:0/1136807342 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc540012040 con 0x7fc54c15deb0 2024-05-20T22:11:39.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.191+0000 7fc548ff9640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc53806be30 0x7fc53806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.191+0000 7fc548ff9640 1 -- 172.21.15.55:0/1136807342 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7fc54008b370 con 0x7fc54c15deb0 2024-05-20T22:11:39.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.191+0000 7fc54bfff640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc53806be30 0x7fc53806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.191+0000 7fc54bfff640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc53806be30 0x7fc53806e340 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0d0570 tx=0x7fc53c009110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.196+0000 7fc548ff9640 1 -- 172.21.15.55:0/1136807342 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc54c0b0de0 con 0x7fc54c15deb0 2024-05-20T22:11:39.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.376+0000 7fc552284640 1 -- 172.21.15.55:0/1136807342 --> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc54c0b7b10 con 0x7fc53806be30 2024-05-20T22:11:39.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.376+0000 7fc548ff9640 1 -- 172.21.15.55:0/1136807342 <== mgr.4098 v2:172.21.15.55:6832/115760224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7fc54c0b7b10 con 0x7fc53806be30 2024-05-20T22:11:39.377 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:39.378 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-20T22:11:39.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.378+0000 7fc552284640 1 -- 172.21.15.55:0/1136807342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc53806be30 msgr2=0x7fc53806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.378+0000 7fc552284640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc53806be30 0x7fc53806e340 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0d0570 tx=0x7fc53c009110 comp rx=0 tx=0).stop 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.378+0000 7fc552284640 1 -- 172.21.15.55:0/1136807342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54c15deb0 msgr2=0x7fc54c0cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.378+0000 7fc552284640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54c15deb0 0x7fc54c0cc1e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc540011aa0 tx=0x7fc540011ea0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.378+0000 7fc552284640 1 -- 172.21.15.55:0/1136807342 shutdown_connections 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.378+0000 7fc552284640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fc53806be30 0x7fc53806e340 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.378+0000 7fc552284640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0b0490 0x7fc54c0b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.378+0000 7fc552284640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc54c15deb0 0x7fc54c0cc1e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.378+0000 7fc552284640 1 --2- 172.21.15.55:0/1136807342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54c0c9890 0x7fc54c0b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.378+0000 7fc552284640 1 -- 172.21.15.55:0/1136807342 >> 172.21.15.55:0/1136807342 conn(0x7fc54c0b4340 msgr2=0x7fc54c1e9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.379+0000 7fc552284640 1 -- 172.21.15.55:0/1136807342 shutdown_connections 2024-05-20T22:11:39.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.379+0000 7fc552284640 1 -- 172.21.15.55:0/1136807342 wait complete. 2024-05-20T22:11:39.390 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-05-20T22:11:38.380107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":244184,"kb_used_data":7208,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730536,"statfs":{"total":773094113280,"available":772844068864,"internally_reserved":0,"allocated":7380992,"data_stored":4010600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002738"},"pg_stats":[{"pgid":"1.0","version":"64'119","reported_seq":270,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-20T22:11:18.944483+0000","last_change":"2024-05-20T22:11:18.944546+0000","last_active":"2024-05-20T22:11:18.944483+0000","last_peered":"2024-05-20T22:11:18.944483+0000","last_clean":"2024-05-20T22:11:18.944483+0000","last_became_active":"2024-05-20T22:07:42.079224+0000","last_became_peered":"2024-05-20T22:07:42.079224+0000","last_unstale":"2024-05-20T22:11:18.944483+0000","last_undegraded":"2024-05-20T22:11:18.944483+0000","last_fullsized":"2024-05-20T22:11:18.944483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'119","last_scrub_stamp":"2024-05-20T22:11:18.944334+0000","last_deep_scrub":"64'119","last_deep_scrub_stamp":"2024-05-20T22:11:18.944334+0000","last_clean_scrub_stamp":"2024-05-20T22:11:18.944334+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:51:21.470057+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1368064,"data_stored":944101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":353733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1368064,"data_stored":944101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T22:11:39.392 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-20T22:11:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.505+0000 7f60eae1b640 1 Processor -- start 2024-05-20T22:11:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.505+0000 7f60eae1b640 1 -- start start 2024-05-20T22:11:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60eae1b640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f60e40d32a0 0x7f60e415f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60eae1b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f60e40d01b0 0x7f60e406b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60eae1b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60e40d3780 con 0x7f60e40cf690 2024-05-20T22:11:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60eae1b640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7f60e40d3a60 con 0x7f60e40d32a0 2024-05-20T22:11:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60eae1b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f60e40d3d40 con 0x7f60e40d01b0 2024-05-20T22:11:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60e8bb7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f60e40d32a0 0x7f60e415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60e8bb7640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f60e40d32a0 0x7f60e415f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45896/0 (socket says 172.21.15.55:45896) 2024-05-20T22:11:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60e8bb7640 1 -- 172.21.15.55:0/1175034597 learned_addr learned my addr 172.21.15.55:0/1175034597 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60e93b8640 1 --2- 172.21.15.55:0/1175034597 >> v2:172.21.15.138:3300/0 conn(0x7f60e40d01b0 0x7f60e406b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60e8bb7640 1 -- 172.21.15.55:0/1175034597 >> v2:172.21.15.138:3300/0 conn(0x7f60e40d01b0 msgr2=0x7f60e406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60e8bb7640 1 --2- 172.21.15.55:0/1175034597 >> v2:172.21.15.138:3300/0 conn(0x7f60e40d01b0 0x7f60e406b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60e8bb7640 1 -- 172.21.15.55:0/1175034597 >> v1:172.21.15.138:6789/0 conn(0x7f60e40cf690 legacy=0x7f60e40cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.507+0000 7f60e8bb7640 1 -- 172.21.15.55:0/1175034597 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e40d4020 con 0x7f60e40d32a0 2024-05-20T22:11:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.508+0000 7f60e8bb7640 1 --2- 172.21.15.55:0/1175034597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d32a0 0x7f60e415f150 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f60e4163d70 tx=0x7f60d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70be1065d956ab60 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.508+0000 7f60e37fe640 1 -- 172.21.15.55:0/1175034597 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4037070 con 0x7f60e40d32a0 2024-05-20T22:11:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.508+0000 7f60e37fe640 1 -- 172.21.15.55:0/1175034597 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d4032440 con 0x7f60e40d32a0 2024-05-20T22:11:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.508+0000 7f60e37fe640 1 -- 172.21.15.55:0/1175034597 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4032810 con 0x7f60e40d32a0 2024-05-20T22:11:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.509+0000 7f60eae1b640 1 -- 172.21.15.55:0/1175034597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d32a0 msgr2=0x7f60e415f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.509+0000 7f60eae1b640 1 --2- 172.21.15.55:0/1175034597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d32a0 0x7f60e415f150 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f60e4163d70 tx=0x7f60d402a270 comp rx=0 tx=0).stop 2024-05-20T22:11:39.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.509+0000 7f60eae1b640 1 -- 172.21.15.55:0/1175034597 shutdown_connections 2024-05-20T22:11:39.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.509+0000 7f60eae1b640 1 --2- 172.21.15.55:0/1175034597 >> v2:172.21.15.138:3300/0 conn(0x7f60e40d01b0 0x7f60e406b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.509+0000 7f60eae1b640 1 --2- 172.21.15.55:0/1175034597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d32a0 0x7f60e415f150 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.509+0000 7f60eae1b640 1 -- 172.21.15.55:0/1175034597 >> 172.21.15.55:0/1175034597 conn(0x7f60e40b6de0 msgr2=0x7f60e40d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:39.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.509+0000 7f60eae1b640 1 -- 172.21.15.55:0/1175034597 shutdown_connections 2024-05-20T22:11:39.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.509+0000 7f60eae1b640 1 -- 172.21.15.55:0/1175034597 wait complete. 2024-05-20T22:11:39.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.511+0000 7f60eae1b640 1 Processor -- start 2024-05-20T22:11:39.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.511+0000 7f60eae1b640 1 -- start start 2024-05-20T22:11:39.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.512+0000 7f60eae1b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60e40cf690 0x7f60e41f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.512+0000 7f60eae1b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d01b0 0x7f60e40c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.512+0000 7f60eae1b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60e40d32a0 0x7f60e40b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.512+0000 7f60eae1b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f60e40d4820 con 0x7f60e40d01b0 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.512+0000 7f60eae1b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60e40b4350 con 0x7f60e40cf690 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.512+0000 7f60eae1b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f60e41578b0 con 0x7f60e40d32a0 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d01b0 0x7f60e40c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d01b0 0x7f60e40c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45904/0 (socket says 172.21.15.55:45904) 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e3fff640 1 -- 172.21.15.55:0/4128357347 learned_addr learned my addr 172.21.15.55:0/4128357347 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e93b8640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60e40d32a0 0x7f60e40b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e8bb7640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60e40cf690 0x7f60e41f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e3fff640 1 -- 172.21.15.55:0/4128357347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60e40d32a0 msgr2=0x7f60e40b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e3fff640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60e40d32a0 0x7f60e40b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e3fff640 1 -- 172.21.15.55:0/4128357347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60e40cf690 msgr2=0x7f60e41f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e3fff640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60e40cf690 0x7f60e41f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e3fff640 1 -- 172.21.15.55:0/4128357347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e4155030 con 0x7f60e40d01b0 2024-05-20T22:11:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e3fff640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d01b0 0x7f60e40c9880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f60d800c3f0 tx=0x7f60d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e17fa640 1 -- 172.21.15.55:0/4128357347 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d80052d0 con 0x7f60e40d01b0 2024-05-20T22:11:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e17fa640 1 -- 172.21.15.55:0/4128357347 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d8005490 con 0x7f60e40d01b0 2024-05-20T22:11:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60eae1b640 1 -- 172.21.15.55:0/4128357347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e4155310 con 0x7f60e40d01b0 2024-05-20T22:11:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60e17fa640 1 -- 172.21.15.55:0/4128357347 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8007260 con 0x7f60e40d01b0 2024-05-20T22:11:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.513+0000 7f60eae1b640 1 -- 172.21.15.55:0/4128357347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e41558d0 con 0x7f60e40d01b0 2024-05-20T22:11:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.514+0000 7f60eae1b640 1 -- 172.21.15.55:0/4128357347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e40c9f80 con 0x7f60e40d01b0 2024-05-20T22:11:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.515+0000 7f60e17fa640 1 -- 172.21.15.55:0/4128357347 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60d8007400 con 0x7f60e40d01b0 2024-05-20T22:11:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.515+0000 7f60e17fa640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f60d006be60 0x7f60d006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.516+0000 7f60e17fa640 1 -- 172.21.15.55:0/4128357347 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7f60d808a200 con 0x7f60e40d01b0 2024-05-20T22:11:39.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.516+0000 7f60e8bb7640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f60d006be60 0x7f60d006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.516+0000 7f60e8bb7640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f60d006be60 0x7f60d006e370 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f60d4034000 tx=0x7f60d402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:39.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.520+0000 7f60e17fa640 1 -- 172.21.15.55:0/4128357347 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f60d80585e0 con 0x7f60e40d01b0 2024-05-20T22:11:39.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.704+0000 7f60eae1b640 1 -- 172.21.15.55:0/4128357347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f60e4069090 con 0x7f60e40d01b0 2024-05-20T22:11:39.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.712+0000 7f60e17fa640 1 -- 172.21.15.55:0/4128357347 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f60d805c8e0 con 0x7f60e40d01b0 2024-05-20T22:11:39.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.714+0000 7f60e17fa640 1 -- 172.21.15.55:0/4128357347 <== mon.0 v2:172.21.15.55:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f60d8061110 con 0x7f60e40d01b0 2024-05-20T22:11:39.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 -- 172.21.15.55:0/4128357347 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f60d006be60 msgr2=0x7f60d006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f60d006be60 0x7f60d006e370 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f60d4034000 tx=0x7f60d402e000 comp rx=0 tx=0).stop 2024-05-20T22:11:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 -- 172.21.15.55:0/4128357347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d01b0 msgr2=0x7f60e40c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d01b0 0x7f60e40c9880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f60d800c3f0 tx=0x7f60d800c8b0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 -- 172.21.15.55:0/4128357347 shutdown_connections 2024-05-20T22:11:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7f60d006be60 0x7f60d006e370 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60e40d32a0 0x7f60e40b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60e40d01b0 0x7f60e40c9880 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 --2- 172.21.15.55:0/4128357347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60e40cf690 0x7f60e41f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 -- 172.21.15.55:0/4128357347 >> 172.21.15.55:0/4128357347 conn(0x7f60e40b6de0 msgr2=0x7f60e40b9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 -- 172.21.15.55:0/4128357347 shutdown_connections 2024-05-20T22:11:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.715+0000 7f60eae1b640 1 -- 172.21.15.55:0/4128357347 wait complete. 2024-05-20T22:11:39.728 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T22:11:39.729 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T22:11:39.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.843+0000 7fa2c13f5640 1 Processor -- start 2024-05-20T22:11:39.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.843+0000 7fa2c13f5640 1 -- start start 2024-05-20T22:11:39.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2c13f5640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa2bc0b3b30 0x7fa2bc0b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2c13f5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2bc15f100 0x7fa2bc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2c13f5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc06c5c0 0x7fa2bc06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2c13f5640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 -- 0x7fa2bc0d3dd0 con 0x7fa2bc0b3b30 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2c13f5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa2bc0d40b0 con 0x7fa2bc15f100 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2c13f5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2bc0d4390 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2baffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa2bc0b3b30 0x7fa2bc0b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2baffd640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa2bc0b3b30 0x7fa2bc0b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45906/0 (socket says 172.21.15.55:45906) 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2baffd640 1 -- 172.21.15.55:0/524722159 learned_addr learned my addr 172.21.15.55:0/524722159 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2ba7fc640 1 --2- 172.21.15.55:0/524722159 >> v2:172.21.15.138:3300/0 conn(0x7fa2bc15f100 0x7fa2bc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2bb7fe640 1 --2- 172.21.15.55:0/524722159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc06c5c0 0x7fa2bc06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2baffd640 1 -- 172.21.15.55:0/524722159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc06c5c0 msgr2=0x7fa2bc06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:39.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2baffd640 1 --2- 172.21.15.55:0/524722159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc06c5c0 0x7fa2bc06c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2baffd640 1 -- 172.21.15.55:0/524722159 >> v2:172.21.15.138:3300/0 conn(0x7fa2bc15f100 msgr2=0x7fa2bc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T22:11:39.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.844+0000 7fa2baffd640 1 --2- 172.21.15.55:0/524722159 >> v2:172.21.15.138:3300/0 conn(0x7fa2bc15f100 0x7fa2bc06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.845+0000 7fa2baffd640 1 -- 172.21.15.55:0/524722159 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc06b5a0 con 0x7fa2bc0b3b30 2024-05-20T22:11:39.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.845+0000 7fa2baffd640 1 --2- 172.21.15.55:0/524722159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc0b3b30 0x7fa2bc0b7fe0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0d2eb0 tx=0x7fa2ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87e60c1400e6afe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:39.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.845+0000 7fa2b9ffb640 1 -- 172.21.15.55:0/524722159 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac037070 con 0x7fa2bc0b3b30 2024-05-20T22:11:39.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2b9ffb640 1 -- 172.21.15.55:0/524722159 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa2ac032440 con 0x7fa2bc0b3b30 2024-05-20T22:11:39.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2b9ffb640 1 -- 172.21.15.55:0/524722159 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac032830 con 0x7fa2bc0b3b30 2024-05-20T22:11:39.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2c13f5640 1 -- 172.21.15.55:0/524722159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc0b3b30 msgr2=0x7fa2bc0b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2c13f5640 1 --2- 172.21.15.55:0/524722159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc0b3b30 0x7fa2bc0b7fe0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0d2eb0 tx=0x7fa2ac02a1e0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2c13f5640 1 -- 172.21.15.55:0/524722159 shutdown_connections 2024-05-20T22:11:39.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2c13f5640 1 --2- 172.21.15.55:0/524722159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc06c5c0 0x7fa2bc06c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2c13f5640 1 --2- 172.21.15.55:0/524722159 >> v2:172.21.15.138:3300/0 conn(0x7fa2bc15f100 0x7fa2bc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2c13f5640 1 --2- 172.21.15.55:0/524722159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc0b3b30 0x7fa2bc0b7fe0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2c13f5640 1 -- 172.21.15.55:0/524722159 >> 172.21.15.55:0/524722159 conn(0x7fa2bc0d51d0 msgr2=0x7fa2bc0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:39.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2c13f5640 1 -- 172.21.15.55:0/524722159 shutdown_connections 2024-05-20T22:11:39.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.846+0000 7fa2c13f5640 1 -- 172.21.15.55:0/524722159 wait complete. 2024-05-20T22:11:39.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.848+0000 7fa2c13f5640 1 Processor -- start 2024-05-20T22:11:39.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.848+0000 7fa2c13f5640 1 -- start start 2024-05-20T22:11:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2c13f5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc06c5c0 0x7fa2bc14f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2c13f5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc0b3b30 0x7fa2bc148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2c13f5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc15f100 0x7fa2bc1f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2c13f5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc162b70 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2c13f5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc06bb60 con 0x7fa2bc15f100 2024-05-20T22:11:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2c13f5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2bc162cf0 con 0x7fa2bc0b3b30 2024-05-20T22:11:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2baffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc06c5c0 0x7fa2bc14f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2baffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc06c5c0 0x7fa2bc14f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45916/0 (socket says 172.21.15.55:45916) 2024-05-20T22:11:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2baffd640 1 -- 172.21.15.55:0/410597301 learned_addr learned my addr 172.21.15.55:0/410597301 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-20T22:11:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2bb7fe640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc15f100 0x7fa2bc1f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2ba7fc640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc0b3b30 0x7fa2bc148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2baffd640 1 -- 172.21.15.55:0/410597301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc0b3b30 msgr2=0x7fa2bc148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2baffd640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc0b3b30 0x7fa2bc148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2baffd640 1 -- 172.21.15.55:0/410597301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc15f100 msgr2=0x7fa2bc1f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2baffd640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc15f100 0x7fa2bc1f8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2baffd640 1 -- 172.21.15.55:0/410597301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc0c8490 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2bb7fe640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc15f100 0x7fa2bc1f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2ba7fc640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc0b3b30 0x7fa2bc148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2baffd640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc06c5c0 0x7fa2bc14f380 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac000c00 tx=0x7fa2ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.850+0000 7fa2a37fe640 1 -- 172.21.15.55:0/410597301 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac037040 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.851+0000 7fa2a37fe640 1 -- 172.21.15.55:0/410597301 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa2ac042070 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.851+0000 7fa2a37fe640 1 -- 172.21.15.55:0/410597301 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac030bd0 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.851+0000 7fa2c13f5640 1 -- 172.21.15.55:0/410597301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc0c8700 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.851+0000 7fa2c13f5640 1 -- 172.21.15.55:0/410597301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc161d10 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.851+0000 7fa2c13f5640 1 -- 172.21.15.55:0/410597301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2bc0c8c20 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.852+0000 7fa2a37fe640 1 -- 172.21.15.55:0/410597301 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2ac047460 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.853+0000 7fa2a37fe640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2a806be60 0x7fa2a806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T22:11:39.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.853+0000 7fa2ba7fc640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2a806be60 0x7fa2a806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T22:11:39.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.853+0000 7fa2a37fe640 1 -- 172.21.15.55:0/410597301 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5421+0+0 (secure 0 0 0) 0x7fa2ac0aba00 con 0x7fa2bc06c5c0 2024-05-20T22:11:39.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.853+0000 7fa2ba7fc640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2a806be60 0x7fa2a806e370 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc15d9e0 tx=0x7fa2b0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T22:11:39.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:39.857+0000 7fa2a37fe640 1 -- 172.21.15.55:0/410597301 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa2ac07a440 con 0x7fa2bc06c5c0 2024-05-20T22:11:40.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.079+0000 7fa2c13f5640 1 -- 172.21.15.55:0/410597301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2bc069090 con 0x7fa2bc06c5c0 2024-05-20T22:11:40.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.079+0000 7fa2a37fe640 1 -- 172.21.15.55:0/410597301 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+3875 (secure 0 0 0) 0x7fa2ac079d40 con 0x7fa2bc06c5c0 2024-05-20T22:11:40.080 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-20T22:11:40.080 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5758,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2604573376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2604573376},{"type":"v1","addr":"172.21.15.138:6834","nonce":2604573376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6171,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1232775482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1232775482},{"type":"v1","addr":"172.21.15.138:6835","nonce":1232775482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2351961135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2351961135},{"type":"v1","addr":"172.21.15.55:6835","nonce":2351961135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/2059202619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":2059202619},{"type":"v1","addr":"172.21.15.55:6837","nonce":2059202619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[]} 2024-05-20T22:11:40.081 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 45 2024-05-20T22:11:40.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 -- 172.21.15.55:0/410597301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2a806be60 msgr2=0x7fa2a806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:40.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2a806be60 0x7fa2a806e370 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc15d9e0 tx=0x7fa2b0008040 comp rx=0 tx=0).stop 2024-05-20T22:11:40.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 -- 172.21.15.55:0/410597301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc06c5c0 msgr2=0x7fa2bc14f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T22:11:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc06c5c0 0x7fa2bc14f380 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac000c00 tx=0x7fa2ac0046c0 comp rx=0 tx=0).stop 2024-05-20T22:11:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 -- 172.21.15.55:0/410597301 shutdown_connections 2024-05-20T22:11:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.55:6832/115760224,v1:172.21.15.55:6833/115760224] conn(0x7fa2a806be60 0x7fa2a806e370 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc15f100 0x7fa2bc1f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2bc0b3b30 0x7fa2bc148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 --2- 172.21.15.55:0/410597301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2bc06c5c0 0x7fa2bc14f380 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T22:11:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 -- 172.21.15.55:0/410597301 >> 172.21.15.55:0/410597301 conn(0x7fa2bc0d51d0 msgr2=0x7fa2bc0b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T22:11:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 -- 172.21.15.55:0/410597301 shutdown_connections 2024-05-20T22:11:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-20T22:11:40.081+0000 7fa2c13f5640 1 -- 172.21.15.55:0/410597301 wait complete. 2024-05-20T22:11:40.095 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-20T22:11:40.095 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-20T22:11:40.095 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T22:11:40.096 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.108 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T22:11:40.108 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T22:11:40.108 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.159 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T22:11:40.159 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T22:11:40.159 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.171 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T22:11:40.171 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T22:11:40.171 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.224 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T22:11:40.224 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-20T22:11:40.224 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-20T22:11:40.224 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.261 INFO:tasks.ceph.osd.0:Stopped 2024-05-20T22:11:40.261 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-20T22:11:40.261 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.336 INFO:tasks.ceph.osd.1:Stopped 2024-05-20T22:11:40.337 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-20T22:11:40.337 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.417 INFO:tasks.ceph.osd.2:Stopped 2024-05-20T22:11:40.418 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-20T22:11:40.418 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.486 INFO:tasks.ceph.osd.3:Stopped 2024-05-20T22:11:40.486 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-20T22:11:40.486 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.517 INFO:tasks.ceph.osd.4:Stopped 2024-05-20T22:11:40.517 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-20T22:11:40.517 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.591 INFO:tasks.ceph.osd.5:Stopped 2024-05-20T22:11:40.591 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-20T22:11:40.591 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.665 INFO:tasks.ceph.osd.6:Stopped 2024-05-20T22:11:40.666 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-20T22:11:40.666 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.732 INFO:tasks.ceph.osd.7:Stopped 2024-05-20T22:11:40.733 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-20T22:11:40.733 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-20T22:11:40.733 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.760 INFO:tasks.ceph.mgr.y:Stopped 2024-05-20T22:11:40.760 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-20T22:11:40.761 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.801 INFO:tasks.ceph.mgr.x:Stopped 2024-05-20T22:11:40.802 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-20T22:11:40.802 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-20T22:11:40.802 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.814 INFO:tasks.ceph.mon.a:Stopped 2024-05-20T22:11:40.814 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-20T22:11:40.814 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.854 INFO:tasks.ceph.mon.b:Stopped 2024-05-20T22:11:40.855 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-20T22:11:40.855 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T22:11:40.907 INFO:tasks.ceph.mon.c:Stopped 2024-05-20T22:11:40.908 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-20T22:11:40.908 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-20T22:11:40.945 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:11:40.946 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-20T22:11:41.154 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:11:41.155 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-20T22:11:41.235 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:11:41.235 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-20T22:11:41.291 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:11:41.291 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-20T22:11:41.344 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:11:41.344 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-20T22:11:41.519 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:11:41.520 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-20T22:11:41.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:11:41.584 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-20T22:11:41.642 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:11:41.642 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-20T22:11:41.700 INFO:tasks.ceph:Archiving mon data... 2024-05-20T22:11:41.701 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767/data/mon.a.tgz 2024-05-20T22:11:41.701 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2024-05-20T22:11:41.718 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.r4CSharV1W 2024-05-20T22:11:41.718 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.r4CSharV1W 2024-05-20T22:11:41.867 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.r4CSharV1W 2024-05-20T22:11:41.953 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.r4CSharV1W is 381KB 2024-05-20T22:11:42.051 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.r4CSharV1W 2024-05-20T22:11:42.067 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767/data/mon.b.tgz 2024-05-20T22:11:42.067 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-05-20T22:11:42.085 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.EXGlCq0tTl 2024-05-20T22:11:42.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.EXGlCq0tTl 2024-05-20T22:11:42.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.EXGlCq0tTl 2024-05-20T22:11:42.345 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.EXGlCq0tTl is 499KB 2024-05-20T22:11:42.443 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.EXGlCq0tTl 2024-05-20T22:11:42.458 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767/data/mon.c.tgz 2024-05-20T22:11:42.458 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-05-20T22:11:42.513 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.ir2a34rxrz 2024-05-20T22:11:42.513 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ir2a34rxrz 2024-05-20T22:11:42.670 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.ir2a34rxrz 2024-05-20T22:11:42.753 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.ir2a34rxrz is 498KB 2024-05-20T22:11:42.851 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.ir2a34rxrz 2024-05-20T22:11:42.866 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-20T22:11:42.866 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-20T22:11:42.869 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-20T22:11:42.953 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-20T22:11:42.986 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-20T22:11:42.986 INFO:tasks.ceph:Archiving crash dumps... 2024-05-20T22:11:42.988 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767/remote/smithi055/crash 2024-05-20T22:11:42.989 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-20T22:11:43.020 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767/remote/smithi138/crash 2024-05-20T22:11:43.021 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-20T22:11:43.049 INFO:tasks.ceph:Shutting down logrotate 2024-05-20T22:11:43.049 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:11:43.065 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T22:11:43.146 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-20T22:11:43.174 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-20T22:11:43.198 INFO:tasks.ceph:Compressing logs... 2024-05-20T22:11:43.199 DEBUG:teuthology.orchestra.run.smithi055:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-20T22:11:43.217 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-20T22:11:43.278 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-20T22:11:43.278 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37369.log 2024-05-20T22:11:43.278 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44342.log 2024-05-20T22:11:43.278 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.37369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37580.log 2024-05-20T22:11:43.278 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.1.44342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37620.log 2024-05-20T22:11:43.278 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37369.log.gz 2024-05-20T22:11:43.278 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-20T22:11:43.278 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.37580.log: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-20T22:11:43.278 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.37620.log: 90.6% -- replaced with /var/log/ceph/ceph-client.1.44342.log.gz 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37540.log 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37580.log.gz 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.37620.log.gz 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.37540.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37540.log.gz 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-20T22:11:43.279 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37500.log 2024-05-20T22:11:43.280 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.6.log: 93.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-20T22:11:43.289 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37409.log 2024-05-20T22:11:43.289 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.37500.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37500.log.gz 2024-05-20T22:11:43.301 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37460.log 2024-05-20T22:11:43.301 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.37409.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37409.log.gz 2024-05-20T22:11:43.317 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44796.log 2024-05-20T22:11:43.317 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.37460.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37460.log.gz 2024-05-20T22:11:43.317 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-20T22:11:43.317 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.3.44796.log: 91.5% -- replaced with /var/log/ceph/ceph-client.3.44796.log.gz 2024-05-20T22:11:43.317 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-20T22:11:43.338 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37329.log 2024-05-20T22:11:43.357 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.37329.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37329.log.gz 2024-05-20T22:11:43.357 INFO:teuthology.orchestra.run.smithi138.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-20T22:11:43.370 INFO:teuthology.orchestra.run.smithi138.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-20T22:11:43.412 INFO:teuthology.orchestra.run.smithi138.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58391.log 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50117.log 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52311.log 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52605.log 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58391.log.gz 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53375.log 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50117.log.gz 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52311.log.gz 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46442.log 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53375.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52605.log.gz 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46542.log 2024-05-20T22:11:43.416 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53375.log.gz 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56974.log 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46442.log.gz 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37665.log 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46542.log.gz 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46204.log 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56974.log: /var/log/ceph/ceph-client.admin.37665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51975.log 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56974.log.gz 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr: 87.7%/var/log/ceph/ceph-client.admin.46204.log: -- replaced with /var/log/ceph/ceph-client.admin.37665.log.gz 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55763.log 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46204.log.gz 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51167.log 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51975.log.gz 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.55763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55501.log 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51167.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55763.log.gz 2024-05-20T22:11:43.417 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52143.log 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51167.log.gz 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.55501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52227.log 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55501.log.gz 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57772.log 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52143.log.gz 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57366.log 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45161.log 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52227.log.gz 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57772.log.gz 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60398.log 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57366.log.gz 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46949.log 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45161.log.gz 2024-05-20T22:11:43.418 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57496.log 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60398.log.gz/var/log/ceph/ceph-client.admin.46949.log: 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52101.log 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57496.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46949.log.gz /var/log/ceph/ceph-client.admin.43485.log 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57496.log.gz 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50631.log 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52101.log: /var/log/ceph/ceph-client.admin.43485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46865.log 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52101.log.gz 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50631.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43485.log.gz 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54482.log 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50631.log.gz 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48996.log 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46865.log.gz 2024-05-20T22:11:43.419 INFO:teuthology.orchestra.run.smithi055.stderr:gzip/var/log/ceph/ceph-client.admin.54482.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50301.log 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.48996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57580.log 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54482.log.gz 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50301.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48996.log.gz 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49290.log 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50301.log.gz 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56718.log 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49290.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57580.log.gz 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53187.log 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56718.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49290.log.gz 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58815.log 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56718.log.gz 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51519.log 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53187.log.gz 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57706.log 2024-05-20T22:11:43.420 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58815.log.gz 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46991.log 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57706.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51519.log.gz 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46907.log 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54610.log 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46991.log.gz 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46907.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57706.log.gz 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52521.log 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.54610.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46907.log.gz 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51639.log 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54610.log.gz 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52353.log 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58564.log 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52521.log.gz 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52353.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-20T22:11:43.421 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51639.log.gz 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58564.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52353.log.gz 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50075.log 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58564.log.gz 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49332.log 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45859.log 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50075.log.gz 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60710.log 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% 85.5% -- replaced with /var/log/ceph/ceph-client.admin.49332.log.gz 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53229.log 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60710.log: gzip -5 --verbose 88.6% -- /var/log/ceph/ceph-client.admin.47033.log -- replaced with /var/log/ceph/ceph-client.admin.45859.log.gz 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60710.log.gz 2024-05-20T22:11:43.422 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53641.log 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53229.log.gz 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50673.log 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47033.log.gz 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49794.log 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59903.log 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50673.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53641.log.gz 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49794.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.50673.log.gz -5 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58498.log 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49794.log.gz 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr:gzip/var/log/ceph/ceph-client.admin.59903.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46443.log 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59903.log.gzgzip 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55637.log 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46443.log.gz 2024-05-20T22:11:43.423 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47453.log 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58498.log.gz 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.55637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56168.log 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47453.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55637.log.gz 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59204.log 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52934.log 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47453.log.gz 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59204.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56168.log.gz 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56126.log 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59204.log.gz 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49668.log 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52934.log.gz 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59483.log 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51597.log 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56126.log.gz 2024-05-20T22:11:43.424 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59483.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49668.log.gz 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45867.log 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59483.log.gz 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49374.log 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51597.log.gz 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46739.log 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58939.log 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45867.log.gz 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46739.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49374.log.gz 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57412.log 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46739.log.gz 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56813.log 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57412.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58939.log.gz 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51477.log 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56813.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57412.log.gz 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43441.log 2024-05-20T22:11:43.425 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56813.log.gz 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51477.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56042.log 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51477.log.gz 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43441.log.gz 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49931.log 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57282.log 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56042.log: /var/log/ceph/ceph-client.admin.49931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43230.log 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56042.log.gz 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr: 88.9%/var/log/ceph/ceph-client.admin.57282.log: -- replaced with /var/log/ceph/ceph-client.admin.49931.log.gz 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53557.log 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57282.log.gz 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59735.log 2024-05-20T22:11:43.426 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47285.log 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53557.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43230.log.gz 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59330.log 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53557.log.gz 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59735.log.gz 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60794.log 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47285.log.gz 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57454.log 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59330.log.gz 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52185.log 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44456.log 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60794.log.gz 2024-05-20T22:11:43.427 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52185.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57454.log.gz 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58275.log 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52185.log.gz 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.44456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60542.log 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60920.log 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60542.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58275.log.gz 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61004.log 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60542.log.gz 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43657.log 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60920.log.gz 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.61004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54525.log 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61004.log.gz 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43188.log 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43657.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.44456.log.gz 2024-05-20T22:11:43.428 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.54525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55916.log 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43657.log.gz 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43188.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54525.log.gz 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53473.log 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.55916.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43188.log.gz 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50967.log 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55916.log.gz 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52892.log 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53473.log.gz 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50427.log 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52892.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50967.log.gz 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52892.log.gz 2024-05-20T22:11:43.429 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53851.log 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50427.log.gz/var/log/ceph/ceph-osd.3.log: 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59777.log 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50385.log 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53851.log.gz 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45449.log 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59777.log.gz 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50547.log 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50385.log: /var/log/ceph/ceph-client.admin.45449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43357.log 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50385.log.gz 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50547.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45449.log.gz 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49416.log 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50547.log.gz 2024-05-20T22:11:43.430 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55679.log 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43357.log.gz 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56893.log 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49416.log: /var/log/ceph/ceph-client.admin.55679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46823.log 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49416.log.gz 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56893.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55679.log.gz 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45594.log 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56893.log.gz 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61172.log 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46823.log: /var/log/ceph/ceph-client.admin.45594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44951.log 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46823.log.gz 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.61172.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45594.log.gz 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49710.log 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61172.log.gz 2024-05-20T22:11:43.431 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49626.log 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.44951.log: /var/log/ceph/ceph-client.admin.49710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53019.log 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44951.log.gz 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49710.log.gz 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51393.log 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53145.log 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53019.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49626.log.gz 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51393.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53019.log.gz 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55595.log 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51393.log.gz 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50015.log 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53145.log: /var/log/ceph/ceph-client.admin.55595.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53145.log.gz 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55595.log.gz 2024-05-20T22:11:43.432 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45856.log 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50015.log.gz 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49542.log 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44017.log 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45856.log: /var/log/ceph/ceph-client.admin.49542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44683.log 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45856.log.gz 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49542.log.gz 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61046.log 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.44017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59063.log 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.44683.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44017.log.gz 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.61046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57143.log 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44683.log.gz 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61046.log.gz 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50589.log 2024-05-20T22:11:43.433 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57143.log: gzip -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59063.log.gz 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57187.log 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57143.log.gz 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46541.log 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50589.log.gz 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49584.log 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59861.log 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8%/var/log/ceph/ceph-client.admin.46541.log: -- replaced with /var/log/ceph/ceph-client.admin.57187.log.gz 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49584.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46541.log.gz 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46543.log 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49584.log.gz 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56550.log 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59861.log: /var/log/ceph/ceph-client.admin.46543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58691.log 2024-05-20T22:11:43.434 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59861.log.gz 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46543.log.gz 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49752.log 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56550.log.gz 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58691.log: /var/log/ceph/ceph-client.admin.49752.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58691.log.gz 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49752.log.gz 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56252.log 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50259.log 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.56252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47075.log 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56252.log.gz 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53271.log 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5/var/log/ceph/ceph-client.admin.47075.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50259.log.gz 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59651.log 2024-05-20T22:11:43.435 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53271.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47075.log.gz 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60006.log 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53271.log.gz 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45533.log 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr:gzip/var/log/ceph/ceph-client.admin.60006.log: -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59651.log.gz 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr: --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45533.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60006.log.gz 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45533.log.gz 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr: /var/log/ceph/ceph-client.admin.47201.log 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48094.log 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56592.log 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47201.log.gz 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.48094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37401.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48094.log.gz 2024-05-20T22:11:43.436 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56592.log: gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56592.log.gz 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr: -- /var/log/ceph/ceph-client.admin.55958.log 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.37401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61212.log 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr: 92.3%/var/log/ceph/ceph-client.admin.55958.log: -- replaced with /var/log/ceph/ceph-client.admin.37401.log.gz 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52976.log 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55958.log.gz 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48744.log 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.61212.log: /var/log/ceph/ceph-client.admin.52976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50217.log 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.48744.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52976.log.gz 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61212.log.gz 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48744.log.gz 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50883.log 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:gzip/var/log/ceph/ceph-client.admin.50217.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48912.log 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52689.log 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50217.log.gz 2024-05-20T22:11:43.437 INFO:teuthology.orchestra.run.smithi055.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50883.log.gz/var/log/ceph/ceph-client.admin.48912.log: 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48912.log.gz 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46206.log 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54219.log 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52689.log.gz 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47972.log 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3%/var/log/ceph/ceph-client.0.54219.log: -- replaced with /var/log/ceph/ceph-client.admin.46206.log.gz 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60584.log 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.0.47972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44993.log 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60584.log: 90.3% -- replaced with /var/log/ceph/ceph-client.0.47972.log.gz 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60584.log.gz 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50505.log 2024-05-20T22:11:43.438 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.44993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57101.log 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44993.log.gz 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58032.log 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57101.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50505.log.gz 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51041.log 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57101.log.gz 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.0.58032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43529.log 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51041.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51041.log.gz 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60836.log 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43529.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43529.log.gz 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45323.log 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60836.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60836.log.gz 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59001.log 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45323.log: 91.8% -- replaced with /var/log/ceph/ceph-client.0.58032.log.gz 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49206.log 2024-05-20T22:11:43.439 INFO:teuthology.orchestra.run.smithi055.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45323.log.gz 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56000.log 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49206.log: 89.2% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49206.log.gz 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59001.log.gz 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48954.log 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51681.log 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56000.log.gz 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.48954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50757.log 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51681.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48954.log.gz 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51681.log.gz 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56084.log 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50757.log: -- /var/log/ceph/ceph-client.admin.60500.log 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50757.log.gz 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56084.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56084.log.gz 2024-05-20T22:11:43.440 INFO:teuthology.orchestra.run.smithi055.stderr: -- /var/log/ceph/ceph-client.admin.53599.log 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52017.log 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60500.log.gz/var/log/ceph/ceph-client.admin.53599.log: 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43909.log 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53599.log.gz 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose 93.3% -- replaced with /var/log/ceph/ceph-client.0.54219.log.gz 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr: -- /var/log/ceph/ceph-client.admin.45870.log 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52017.log: /var/log/ceph/ceph-client.admin.43909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53061.log 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43909.log.gz 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52017.log.gz/var/log/ceph/ceph-client.admin.45870.log: 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45870.log.gz/var/log/ceph/ceph-client.admin.53061.log: 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56438.log 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53061.log.gz 2024-05-20T22:11:43.441 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50841.log 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.56438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60962.log 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56438.log.gz 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50841.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50841.log.gz 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51351.log 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48828.log 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60962.log.gz 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47369.log 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51351.log.gz 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.48828.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48828.log.gz 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45281.log 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58349.log 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47369.log.gz 2024-05-20T22:11:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45869.log 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58349.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45281.log.gz 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58349.log.gz 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51933.log 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59288.log 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45869.log.gz 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48786.log 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59288.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51933.log.gz 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59288.log.gz 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44893.log 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.48786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58753.log 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48786.log.gz 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.44893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61254.log 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58753.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44893.log.gz 2024-05-20T22:11:43.443 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58753.log.gz 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45491.log 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.61254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51765.log 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61254.log.gz 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56294.log 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51765.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45491.log.gz 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51765.log.gz 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56396.log 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51435.log 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56294.log.gz 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52269.log 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56396.log.gz 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51435.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51435.log.gz 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47243.log 2024-05-20T22:11:43.444 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45688.log 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52269.log.gz 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60272.log 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45688.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47243.log.gz 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45688.log.gz 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59441.log 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45854.log 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60272.log.gz 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.48470.log 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45854.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59441.log.gz 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45854.log.gz 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52059.log 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.2.48470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44809.log 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52059.log: 91.3% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52059.log.gz 2024-05-20T22:11:43.445 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-client.2.48470.log.gz 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51891.log 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.44809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48870.log 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44809.log.gz 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53893.log 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51891.log.gz 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.48870.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48870.log.gz 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56336.log 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43699.log 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56336.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53893.log.gz 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56336.log.gz 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59693.log 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46207.log 2024-05-20T22:11:43.446 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53313.log 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59693.log.gz 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46207.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46207.log.gz 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59964.log 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52563.log 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53313.log.gz 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59819.log 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52563.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59964.log.gz 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52563.log.gz 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50343.log 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46441.log 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59819.log.gz 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49080.log 2024-05-20T22:11:43.447 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46441.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50343.log.gz 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46441.log.gz 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55874.log 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49080.log.gz 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.55874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60752.log 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47327.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55874.log.gz 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46781.log 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43783.log 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60752.log.gz 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47411.log 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43783.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46781.log.gz 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43783.log.gz 2024-05-20T22:11:43.448 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46193.log 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45119.log 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47411.log.gz 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45035.log 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45119.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46193.log.gz 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45119.log.gz 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50715.log 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59246.log 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45035.log.gz 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61088.log 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50715.log.gz 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56210.log 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59246.log: /var/log/ceph/ceph-client.admin.61088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45864.log 2024-05-20T22:11:43.449 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56210.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59246.log.gz 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61088.log.gz 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49973.log 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56210.log.gz 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45864.log: gzip -5 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45864.log.gz 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48592.log 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47159.log 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49973.log.gz 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.48592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42901.log 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48592.log.gzgzip 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45407.log 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60878.log 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.42901.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47159.log.gz 2024-05-20T22:11:43.450 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45407.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42901.log.gz 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45407.log.gz 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr: /var/log/ceph/ceph-client.admin.52811.log 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57324.log 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60878.log.gz 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45865.log 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52811.log.gz/var/log/ceph/ceph-client.admin.57324.log: 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56508.log 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57324.log.gz 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43273.log 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56676.log 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56508.log: /var/log/ceph/ceph-client.admin.43273.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45865.log.gz 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56508.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43273.log.gz 2024-05-20T22:11:43.451 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60668.log 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58456.log 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56676.log.gz 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46539.log 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60668.log.gz 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49248.log 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58456.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49836.log 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60118.log 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49248.log: /var/log/ceph/ceph-client.admin.49836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59525.log 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46539.log.gz 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60118.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49248.log.gz 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49836.log.gz 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60118.log.gz 2024-05-20T22:11:43.452 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51125.log 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53515.log 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7%/var/log/ceph/ceph-client.admin.51125.log: -- replaced with /var/log/ceph/ceph-client.admin.59525.log.gz 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43867.log 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51125.log.gz 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53103.log 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53515.log: /var/log/ceph/ceph-client.admin.43867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46203.log 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53103.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53515.log.gz 89.0% 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43867.log.gz 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53103.log.gz 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53767.log 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52395.log 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53767.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46203.log.gz 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54652.log 2024-05-20T22:11:43.453 INFO:teuthology.orchestra.run.smithi055.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53767.log.gz 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43107.log 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58877.log 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.54652.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52395.log.gz 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46540.log 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43107.log.gz 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.58877.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.54652.log.gz 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58877.log.gz 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49458.log 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53430.log 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43399.log 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46540.log.gz 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53430.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49458.log.gz 2024-05-20T22:11:43.454 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51807.log 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51251.log 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53430.log.gz 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43399.log: /var/log/ceph/ceph-client.admin.51807.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43399.log.gz 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51807.log.gz 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51849.log 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2%/var/log/ceph/ceph-client.admin.43741.log: -- replaced with /var/log/ceph/ceph-client.admin.51251.log.gz 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37768.log 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47581.log 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53809.log 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.37768.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51849.log.gz 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44851.log 2024-05-20T22:11:43.455 INFO:teuthology.orchestra.run.smithi055.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37768.log.gz 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47581.log.gz 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53809.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53809.log.gz 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60356.log 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.44851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43011.log 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44851.log.gz 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57016.log 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60356.log.gz/var/log/ceph/ceph-client.admin.43011.log: 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43571.log 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43011.log.gz 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48634.log 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57016.log.gz 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43571.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-20T22:11:43.456 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43571.log.gz 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.48634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46199.log 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48634.log.gz/var/log/ceph/ceph-mds.a.log: 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45365.log 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46199.log: -- /var/log/ceph/ceph-client.admin.52647.log 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46199.log.gz 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45365.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45365.log.gz 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44767.log 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57231.log 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52647.log.gz 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.44767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60076.log 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57231.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44767.log.gz 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57231.log.gz 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43613.log 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49500.log 2024-05-20T22:11:43.457 INFO:teuthology.orchestra.run.smithi055.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60076.log.gz 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56634.log 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr: 88.0%/var/log/ceph/ceph-client.admin.49500.log: -- replaced with /var/log/ceph/ceph-client.admin.43613.log.gz 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49500.log.gz 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50925.log 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.56634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49038.log 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56634.log.gz 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50925.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50925.log.gz 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51723.log 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51209.log 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49038.log: /var/log/ceph/ceph-client.admin.51723.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49038.log.gz 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59162.log 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51723.log.gz 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52479.log 2024-05-20T22:11:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59162.log: /var/log/ceph/ceph-client.admin.51209.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51209.log.gz 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59162.log.gz 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54567.log 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52479.log: /var/log/ceph/ceph-client.admin.53959.log 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52479.log.gz 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.54567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53959.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54567.log.gz 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49164.log 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53959.log.gz 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.47117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60458.log 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49164.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50799.log 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49164.log.gz 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57664.log 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60458.log: /var/log/ceph/ceph-client.admin.50799.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60458.log.gz 2024-05-20T22:11:43.459 INFO:teuthology.orchestra.run.smithi055.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50799.log.gz 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51309.log 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57664.log: -- /var/log/ceph/ceph-client.admin.43825.log 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57664.log.gz 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52437.log 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51309.log.gz 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43825.log: 90.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46205.log 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43825.log.gz/var/log/ceph/ceph-client.admin.52437.log: 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53683.log 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52437.log.gz 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59567.log 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46205.log: /var/log/ceph/ceph-client.admin.53683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59609.log 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.59567.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46205.log.gz 2024-05-20T22:11:43.460 INFO:teuthology.orchestra.run.smithi055.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53683.log.gz 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45239.log 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59567.log.gz 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5/var/log/ceph/ceph-client.admin.59609.log: --verbose -- /var/log/ceph/ceph-client.admin.45077.log 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45239.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59609.log.gz 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44725.log 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45239.log.gz 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57058.log 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45077.log: /var/log/ceph/ceph-client.admin.44725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60314.log 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57058.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44725.log.gz 88.2% 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45077.log.gz 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53725.log 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57058.log.gz 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.60314.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31384.log 2024-05-20T22:11:43.461 INFO:teuthology.orchestra.run.smithi055.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60314.log.gz 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50159.log 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.53725.log: /var/log/ceph/ceph.tmp-client.admin.31384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46200.log 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31384.log.gz 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.50159.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53725.log.gz 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50159.log.gz 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr: /var/log/ceph/ceph-client.admin.61130.log 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.46200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49122.log 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46200.log.gz 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.61130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45644.log 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61130.log.gz 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.49122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43315.log 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49122.log.gz 2024-05-20T22:11:43.462 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.45644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51083.log 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45644.log.gz 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43951.log 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43315.log.gz 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55721.log 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.51083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57538.log 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.43951.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51083.log.gz/var/log/ceph/ceph-client.admin.55721.log: 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52731.log 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55721.log.gz 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57538.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43951.log.gz 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57622.log 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.52731.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60626.log 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57538.log.gz 2024-05-20T22:11:43.463 INFO:teuthology.orchestra.run.smithi055.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52731.log.gz 2024-05-20T22:11:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/ceph-client.admin.57622.log: /var/log/ceph/ceph-client.admin.60626.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57622.log.gz 2024-05-20T22:11:43.464 INFO:teuthology.orchestra.run.smithi055.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60626.log.gz 2024-05-20T22:11:43.496 INFO:teuthology.orchestra.run.smithi138.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-20T22:11:43.622 INFO:teuthology.orchestra.run.smithi055.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-20T22:11:44.003 INFO:teuthology.orchestra.run.smithi055.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-20T22:11:44.064 INFO:teuthology.orchestra.run.smithi138.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-20T22:11:44.079 INFO:teuthology.orchestra.run.smithi138.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-20T22:11:44.131 INFO:teuthology.orchestra.run.smithi055.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-20T22:11:44.160 INFO:teuthology.orchestra.run.smithi055.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-20T22:11:44.206 INFO:teuthology.orchestra.run.smithi055.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-20T22:11:44.208 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-05-20T22:11:44.208 INFO:teuthology.orchestra.run.smithi055.stderr:real 0m0.978s 2024-05-20T22:11:44.209 INFO:teuthology.orchestra.run.smithi055.stderr:user 0m3.837s 2024-05-20T22:11:44.209 INFO:teuthology.orchestra.run.smithi055.stderr:sys 0m0.373s 2024-05-20T22:11:44.211 INFO:teuthology.orchestra.run.smithi138.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-20T22:11:44.227 INFO:teuthology.orchestra.run.smithi138.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-20T22:11:44.229 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-05-20T22:11:44.229 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m0.975s 2024-05-20T22:11:44.229 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m3.821s 2024-05-20T22:11:44.229 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m0.148s 2024-05-20T22:11:44.229 INFO:tasks.ceph:Archiving logs... 2024-05-20T22:11:44.230 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767/remote/smithi055/log 2024-05-20T22:11:44.231 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-20T22:11:45.044 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767/remote/smithi138/log 2024-05-20T22:11:45.045 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-20T22:11:45.519 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-20T22:11:45.529 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-20T22:11:45.529 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-20T22:11:45.532 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-20T22:11:45.615 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-20T22:11:45.625 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-20T22:11:45.625 DEBUG:teuthology.orchestra.run.smithi055:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T22:11:45.628 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T22:11:45.642 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-05-20T22:11:45.643 INFO:teuthology.orchestra.run.smithi055.stderr:bash: line 1: ntpq: command not found 2024-05-20T22:11:45.646 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T22:11:45.646 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-05-20T22:11:45.646 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +850us[ +866us] +/- 55ms 2024-05-20T22:11:45.647 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 -188us[ -188us] +/- 67ms 2024-05-20T22:11:45.647 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -840us[ -840us] +/- 66ms 2024-05-20T22:11:45.647 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T22:11:45.648 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T22:11:45.648 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2024-05-20T22:11:45.648 INFO:teuthology.orchestra.run.smithi055.stdout:^* hv01.front.sepia.ceph.com 3 6 377 31 +828us[ +829us] +/- 55ms 2024-05-20T22:11:45.648 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -207us[ -205us] +/- 67ms 2024-05-20T22:11:45.649 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -854us[ -854us] +/- 66ms 2024-05-20T22:11:45.649 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T22:11:45.649 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-20T22:11:45.658 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-20T22:11:45.658 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-20T22:11:45.682 DEBUG:teuthology.orchestra.run.smithi055:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-20T22:11:45.769 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-20T22:11:45.835 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T22:11:45.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:11:45.867 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T22:11:45.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:11:45.895 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-20T22:11:45.904 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-20T22:11:45.923 INFO:teuthology.task.internal:Duration was 704.222836 seconds 2024-05-20T22:11:45.924 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-20T22:11:45.959 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-20T22:11:45.959 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-20T22:11:45.962 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-20T22:11:46.000 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T22:11:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T22:11:46.339 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-20T22:11:46.340 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi055.front.sepia.ceph.com 2024-05-20T22:11:46.340 DEBUG:teuthology.orchestra.run.smithi055:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-20T22:11:46.404 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-05-20T22:11:46.405 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-20T22:11:46.431 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-20T22:11:46.432 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-20T22:11:46.448 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-20T22:11:46.498 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-20T22:11:46.498 DEBUG:teuthology.orchestra.run.smithi055:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-20T22:11:46.518 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-20T22:11:46.640 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-20T22:11:46.651 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-20T22:11:46.652 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-20T22:11:46.678 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-20T22:11:46.711 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-20T22:11:46.721 DEBUG:teuthology.orchestra.run.smithi055:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-20T22:11:46.724 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-20T22:11:46.746 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = core 2024-05-20T22:11:46.775 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-05-20T22:11:46.788 DEBUG:teuthology.orchestra.run.smithi055:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-20T22:11:46.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:11:46.815 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-20T22:11:46.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T22:11:46.843 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-20T22:11:46.856 INFO:teuthology.task.internal:Transferring archived files... 2024-05-20T22:11:46.857 DEBUG:teuthology.misc:Transferring archived files from smithi055:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767/remote/smithi055 2024-05-20T22:11:46.857 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-20T22:11:46.893 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715767/remote/smithi138 2024-05-20T22:11:46.893 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-20T22:11:46.928 INFO:teuthology.task.internal:Removing archive directory... 2024-05-20T22:11:46.928 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-20T22:11:46.931 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-20T22:11:46.978 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-20T22:11:46.990 INFO:teuthology.task.internal:Not uploading archives. 2024-05-20T22:11:46.990 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-20T22:11:47.017 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-20T22:11:47.017 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-20T22:11:47.020 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-20T22:11:47.033 INFO:teuthology.orchestra.run.smithi055.stdout: 266029 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 20 22:11 /home/ubuntu/cephtest 2024-05-20T22:11:47.035 INFO:teuthology.orchestra.run.smithi138.stdout: 266037 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 20 22:11 /home/ubuntu/cephtest 2024-05-20T22:11:47.037 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-20T22:11:47.048 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-20T22:11:47.124 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/openfiletable} duration: 704.2228362560272 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-20T22:11:47.124 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-20T22:11:47.212 INFO:teuthology.run:pass